CN109324337B

CN109324337B - Unmanned aerial vehicle route generation and positioning method and device and unmanned aerial vehicle

Info

Publication number: CN109324337B
Application number: CN201710641017.9A
Authority: CN
Inventors: 刘鹏
Original assignee: Guangzhou Xaircraft Technology Co Ltd
Current assignee: Guangzhou Xaircraft Technology Co Ltd
Priority date: 2017-07-31
Filing date: 2017-07-31
Publication date: 2022-01-14
Anticipated expiration: 2037-07-31
Also published as: CN109324337A

Abstract

The embodiment of the invention provides a method and a device for generating and positioning a route of an unmanned aerial vehicle and the unmanned aerial vehicle, wherein the route generating method comprises the following steps: acquiring map data; extracting map data of a working area to be operated from the map data; acquiring first characteristic information from the map data of the operation area; and planning a route of the map data of the operation area based on the first characteristic information to obtain route information. The course planning process of the embodiment of the invention does not need GPS, so that the unmanned aerial vehicle can still carry out course planning under the condition of no GPS signal or shielding of the GPS signal, the dependence degree of the unmanned aerial vehicle on the GPS is reduced, and the course planning and navigation functions of the unmanned aerial vehicle are enriched.

Description

Unmanned aerial vehicle route generation and positioning method and device and unmanned aerial vehicle

Technical Field

The present invention relates to the field of unmanned aerial vehicle technology, and in particular, to a method for generating a route of an unmanned aerial vehicle, a method for positioning, a route generating device of an unmanned aerial vehicle, an aircraft, and a computer-readable storage medium.

Background

An Unmanned Aerial Vehicle (Unmanned Aerial Vehicle, UAV for short) is an Unmanned Aerial Vehicle. The unmanned aerial vehicle has wide application and is often applied to industries such as plant protection, city management, geology, meteorology, electric power, emergency and disaster relief, video shooting and the like.

The route task to be performed by the unmanned aerial vehicle is route information obtained by planning a route for a working object in a working area according to a certain rule, and the working area may be a closed area formed by a series of GPS (Global Positioning System) coordinates.

After the unmanned aerial vehicle obtains the route information, the unmanned aerial vehicle also needs to obtain its own position by positioning in the process of executing operation, and in the prior art, a position reference can be provided for the unmanned aerial vehicle based on a GPS or a Real-time kinematic (RTK) -GPS (Real-time kinematic differential method) with higher precision.

Therefore, the air route planning and the operation process in the prior art work based on the GPS, the air route planning and the operation process greatly depend on the GPS, and once the air route planning and the operation process have no GPS signals or are shielded, the unmanned aerial vehicle cannot work normally.

Disclosure of Invention

In view of the above problems, embodiments of the present invention are provided to provide a method of generating routes for an unmanned aerial vehicle, a method of positioning and a corresponding route generation apparatus for an unmanned aerial vehicle, an aircraft and a computer-readable storage medium that overcome or at least partially address the above problems.

In order to solve the above problem, an embodiment of the present invention discloses a method for generating a route of an unmanned aerial vehicle, where the method includes:

acquiring map data;

extracting map data of a working area to be operated from the map data;

acquiring first characteristic information from the map data of the operation area;

and planning a route of the map data of the operation area based on the first characteristic information to obtain route information.

Preferably, the step of extracting the map data of the work area to be worked from the map data includes:

acquiring a plurality of measurement points marked in the map data;

and carrying out image cutting on the closed area surrounded by the plurality of measuring points to obtain map data of the operation area.

Preferably, the first feature information at least includes a feature point, a feature descriptor corresponding to the feature point, and a feature dictionary corresponding to the feature point, and the step of obtaining the first feature information from the map data of the work area includes:

dividing the map data of the operation area into a plurality of block data;

respectively extracting a preset number of feature points from each block data;

generating a feature descriptor corresponding to the feature point;

and generating a feature dictionary corresponding to the feature points.

Preferably, the unmanned aerial vehicle comprises an onboard camera, and the track information comprises a waypoint and attribute information of the waypoint; the step of planning the route of the map data of the operation area based on the first characteristic information to obtain the route information comprises the following steps:

carrying out route planning on the map data of the operation area to obtain a plurality of waypoints;

respectively acquiring three-dimensional coordinate data of the waypoints;

for each waypoint, collecting a feature descriptor and a feature dictionary of an area which can be covered by the field angle of the airborne camera by taking the pixel coordinates of the waypoint as a center, and generating a feature set;

organizing the three-dimensional coordinate data and the feature set into attribute information of the waypoint.

Preferably, in the track information, the distance between two waypoints is determined by the overlapping degree of two images continuously acquired by the onboard camera.

Preferably, the step of respectively acquiring the three-dimensional coordinate data of the waypoints comprises:

acquiring the geographic position coordinates of the pixels corresponding to the waypoints as plane coordinates;

acquiring the height of a pixel corresponding to the waypoint;

acquiring the ground height of the unmanned aerial vehicle relative to the ground during planning;

and taking the height and the sum of the height to the ground as a height coordinate.

The embodiment of the invention also discloses a positioning method, which is applied to an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises an airborne camera, and the method comprises the following steps:

acquiring track information, wherein the track information comprises attribute information of a plurality of waypoints;

acquiring real-time image data acquired by the airborne camera according to a preset interval;

and determining the current position information of the unmanned aerial vehicle based on the flight path information and the real-time image data.

Preferably, the step of determining the current position information of the unmanned aerial vehicle based on the track information and the real-time image data comprises:

judging whether the unmanned aerial vehicle deviates from a route corresponding to the flight path information or not based on the flight path information and the real-time image data;

if the unmanned aerial vehicle does not deviate from the air route, acquiring temporary positioning information of the unmanned aerial vehicle;

and determining the current position information of the unmanned aerial vehicle based on the temporary positioning information.

Preferably, the attribute information includes a feature set associated with the waypoint, where the feature set includes a plurality of feature points and a first feature dictionary corresponding to each feature point;

the step of judging whether the unmanned aerial vehicle deviates from the route corresponding to the flight path information or not based on the flight path information and the real-time image data comprises the following steps:

extracting second feature information from the real-time image data, wherein the second feature information comprises a second feature dictionary;

calculating the matching degree of the second feature dictionary and each first feature dictionary in the feature set;

and if the first characteristic dictionary with the matching degree larger than the preset threshold value does not exist, judging that the unmanned aerial vehicle deviates from the route corresponding to the flight path information.

Preferably, the step of acquiring temporary positioning information of the unmanned aerial vehicle includes:

if the first feature dictionary with the matching degree larger than the preset threshold exists, obtaining feature points corresponding to the first feature dictionary with the matching degree larger than the preset threshold, and taking the region determined by the feature points as temporary positioning information of the unmanned aerial vehicle.

Preferably, the feature set further includes a first feature descriptor corresponding to each feature point, the second feature information includes a second feature descriptor, and the step of determining the current position information of the unmanned aerial vehicle based on the temporary positioning information includes:

determining neighborhood waypoints based on the temporary positioning information, and acquiring a first feature descriptor associated with the neighborhood waypoints;

matching the second feature descriptor with the first feature descriptor to determine a matched feature point sequence;

and determining the current position information of the unmanned aerial vehicle based on the matched feature point sequence.

Preferably, the step of determining the current position information of the unmanned aerial vehicle based on the matched feature point sequence comprises:

and calculating the matched characteristic point sequence by adopting a preset positioning algorithm, and determining the current position information of the unmanned aerial vehicle.

Preferably, the method further comprises:

determining the position offset of the current position information and the track information;

and correcting the current position information based on the position offset.

The embodiment of the invention also discloses a route generation device of the unmanned aerial vehicle, which comprises the following components:

the map data acquisition module is used for acquiring map data;

the operation area map data acquisition module is used for extracting operation area map data to be operated from the map data;

the first characteristic information acquisition module is used for acquiring first characteristic information from the map data of the operation area;

and the route planning module is used for planning routes of the map data of the operation area based on the first characteristic information to obtain route information.

Preferably, the work area map data acquisition module includes:

a measurement point marking sub-module for acquiring a plurality of measurement points marked in the map data;

and the image cutting sub-module is used for carrying out image cutting on the closed area formed by the plurality of measuring points to obtain the map data of the operation area.

Preferably, the first feature information at least includes a feature point, a feature descriptor corresponding to the feature point, and a feature dictionary corresponding to the feature point, and the first feature information acquiring module includes:

the map segmentation submodule is used for segmenting the map data of the operation area into a plurality of block data;

the characteristic point extraction submodule is used for extracting a preset number of characteristic points from each block data respectively;

the characteristic descriptor generation submodule is used for generating a characteristic descriptor corresponding to the characteristic point;

and the feature dictionary generating submodule is used for generating a feature dictionary corresponding to the feature points.

Preferably, the unmanned aerial vehicle comprises an onboard camera, and the track information comprises a waypoint and attribute information of the waypoint; the route planning module comprises:

the planning submodule is used for carrying out route planning on the map data of the operation area to obtain a plurality of waypoints;

the three-dimensional coordinate data acquisition submodule is used for respectively acquiring the three-dimensional coordinate data of the waypoints;

the characteristic set acquisition submodule is used for acquiring a characteristic descriptor and a characteristic dictionary of an area which can be covered by the field angle of the airborne camera by taking the pixel coordinates of the waypoint as the center for each waypoint to generate a characteristic set;

and the organizing submodule is used for organizing the three-dimensional coordinate data and the feature set into the attribute information of the waypoint.

Preferably, the three-dimensional coordinate data acquisition sub-module includes:

the plane coordinate acquisition unit is used for acquiring the geographic position coordinates of the pixels corresponding to the waypoints as plane coordinates;

the height coordinate acquisition unit is used for acquiring the height of the pixel corresponding to the waypoint; acquiring the ground height of the unmanned aerial vehicle relative to the ground during planning; and taking the height and the sum of the height to the ground as a height coordinate.

The embodiment of the invention also discloses an unmanned aerial vehicle, which comprises an airborne camera, and the unmanned aerial vehicle further comprises:

the flight path information acquisition module is used for acquiring flight path information, and the flight path information comprises attribute information of a plurality of flight points;

the real-time image acquisition module is used for acquiring real-time image data acquired by the airborne camera according to preset intervals;

and the positioning module is used for determining the current position information of the unmanned aerial vehicle based on the flight path information and the real-time image data.

Preferably, the positioning module comprises:

the flight path deviation judging submodule is used for judging whether the unmanned aerial vehicle deviates from a flight path corresponding to the flight path information or not based on the flight path information and the real-time image data;

the temporary positioning information acquisition submodule is used for acquiring temporary positioning information of the unmanned aerial vehicle if the unmanned aerial vehicle does not deviate from the air route;

and the current position information acquisition submodule is used for determining the current position information of the unmanned aerial vehicle based on the temporary positioning information.

the lane departure judgment submodule includes:

a second feature information extraction unit configured to extract second feature information from the real-time image data, the second feature information including a second feature dictionary;

the matching degree calculation unit is used for calculating the matching degree of the second feature dictionary and each first feature dictionary in the feature set;

and the deviation unit is used for judging that the unmanned aerial vehicle deviates from the route corresponding to the flight path information if the first characteristic dictionary with the matching degree larger than the preset threshold does not exist.

Preferably, the temporary positioning information obtaining sub-module is further configured to:

Preferably, the feature set further includes a first feature descriptor corresponding to each feature point, the second feature information includes a second feature descriptor, and the current location information obtaining sub-module includes:

the neighborhood waypoint determining unit is used for determining neighborhood waypoints based on the temporary positioning information and acquiring a first feature descriptor associated with the neighborhood waypoints;

the matching unit is used for matching the second feature descriptor with the first feature descriptor and determining a matched feature point sequence;

and the position determining unit is used for determining the current position information of the unmanned aerial vehicle based on the matched characteristic point sequence.

Preferably, the position determination unit is further configured to:

Preferably, the unmanned aerial vehicle further comprises:

the position offset determining module is used for determining the position offset of the current position information and the track information;

and the offset correction module is used for correcting the current position information based on the position offset.

The embodiment of the invention also discloses an aircraft, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and is characterized in that the steps of the method are realized when the processor executes the program.

The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program realizes the steps of the method when being executed by a processor.

The embodiment of the invention has the following advantages:

in the embodiment of the invention, the air route planning can be carried out according to the acquired map data, and the characteristic matching is carried out by combining the image data acquired by the airborne camera, so that the precise positioning of the unmanned aerial vehicle can be carried out under the condition of no GPS or no GPS signal.

Drawings

FIG. 1 is a flow chart illustrating the steps of an embodiment of a method for generating routes for an unmanned aerial vehicle according to the present invention;

FIG. 2 is a flow chart of the steps of one embodiment of a method of positioning of the present invention;

FIG. 3 is a block diagram of an embodiment of a route generation apparatus for an unmanned aerial vehicle according to the present invention;

fig. 4 is a block diagram of an embodiment of an unmanned aerial vehicle according to the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

In the unmanned aerial vehicle plant protection operation process, the unmanned aerial vehicle can be controlled by a flight control system (flying control for short) to complete the whole flight processes of takeoff, air flight, operation task execution, return flight and the like, the flying control is equivalent to the effect of a driver on human-computer interaction on the unmanned aerial vehicle, and the unmanned aerial vehicle is one of the most core technologies of the unmanned aerial vehicle.

Can install the airborne camera in unmanned vehicles, after unmanned vehicles starts, flight control system can control the airborne camera and shoot the image.

Referring to FIG. 1, a flow chart illustrating steps of an embodiment of a method for generating routes for an unmanned aerial vehicle of the present invention may include the steps of:

step 101, obtaining map data;

in the embodiment of the present invention, map data may be collected in advance, where the map data may be data including geographic location information (three-dimensional coordinate data), and the representation manner of the map data may include Tile (Tile) or TIFF (Tagged Image File Format).

In the implementation, the map data can be obtained through aerial photogrammetry and splicing processing, and the map data can be stored in a server at the cloud end, a database of the unmanned aerial vehicle, a ground station, a remote controller and other positions.

Therefore, the unmanned aerial vehicle can request map data from a server at the cloud end, extract the map data from a local database, receive the map data sent by a ground station or a remote controller, and the like.

In one embodiment, the map data may include high definition map data, where high definition map data refers to map data with a resolution of up to 1:500, or within a resolution of greater than 1: 1000. In practice, the resolution of the map data may affect the accuracy of the positioning of the unmanned aerial vehicle, and the higher the positioning accuracy, the higher the map resolution is required. For example, if the resolution of the map data is 1:500, the positioning accuracy may be greater than 5 cm.

102, extracting map data of a working area to be operated from the map data;

in the embodiment of the invention, after the map data is obtained, the map data of the operation area to be operated can be extracted from the map data according to the actual operation requirement.

In a preferred embodiment of the present invention, step 102 may further comprise the following sub-steps:

a substep S11 of acquiring a plurality of measurement points marked in the map data;

in one implementation, after determining the job task, the operator may perform dotting on a job land corresponding to the job task through the mapping device according to job requirements to determine a plurality of measurement points (s1, s2, s3, …, sn), and mark the plurality of measurement points in the map data.

In another implementation, the operator may also mark a plurality of measurement points directly in the map data using the naked eye in combination with actual needs (s1, s2, s3, …, sn).

For example, if the area to be worked is a field, a plurality of measurement points may be marked according to the distribution edge of the crop.

The plurality of measurement points are discrete sequential coordinate points, and the coordinate point sequence may be expressed as a point set S ═ S1, S2, S3, …, Sn }.

And a substep S12 of performing image segmentation on the closed region surrounded by the plurality of measurement points to obtain map data of the work region.

Specifically, the measurement points in the point set S are sequentially connected according to a preset sequence to obtain a closed area, and the image of the closed area is cut out to obtain map data of the operation area.

It should be noted that the embodiment of the present invention is not limited to the above-mentioned manner for determining the closed region, and it is all possible for a person skilled in the art to determine in other manners, for example, a plurality of target positions may be selected from the measurement points, and the closed region may be obtained by sequentially connecting the target positions.

Of course, in the working area, the measurement point may be located with respect to an obstacle such as a tree or a utility pole, and the no-fly area may be generated after the connection.

In implementation, each pixel point of the map data has a coordinate, so that the description of the map data of the operation area can be described by using two-dimensional coordinates in a certain order. The sequence may be a clockwise sequence or a counterclockwise sequence, which is not limited in the embodiment of the present invention.

Step 103, acquiring first characteristic information from the map data of the operation area;

after the work area map data is obtained, feature extraction may be performed on the work area map data to obtain first feature information.

As a preferable example of the embodiment of the present invention, the first feature information may include at least feature points, feature descriptors corresponding to the feature points, and feature dictionaries corresponding to the feature points.

In a preferred embodiment of the present invention, step 103 may comprise the following sub-steps:

a substep S21 of dividing the work area map data into a plurality of tile data;

for example, the image segmentation method may be adopted to segment the map data of the work area into N × M block data.

A substep S22 of extracting a preset number of feature points from each block data, respectively;

specifically, after obtaining N × M block data, t feature points may be extracted from each block data, respectively, to ensure that the feature points are uniformly distributed in the map data of the work area.

In practice, the N, M, t above may satisfy the following condition: a predetermined number of pixels corresponds to one feature point, for example, 100 pixels corresponds to one feature point.

Of course, besides the block, the feature point may also be directly extracted from the work area map data according to a preset feature extraction method, which is not limited in this embodiment of the present invention.

Feature points may refer to points whose location itself has a conventional attribute meaning, such as corner points (Fast corner points, Herries corner points, etc.), intersections, and so on.

In a specific implementation, a computer vision method may be adopted to extract feature points from each block data, for example, for Fast corner points, the extraction manner may include the following processes: and traversing each pixel point in the block data, selecting 16 surrounding pixels by taking the current pixel point as a center and 3 as a radius, sequentially comparing, marking if the gray difference value is greater than a preset threshold value, and taking the current point as a feature point if the number of the marks is greater than 12.

A substep S23 of generating a feature descriptor corresponding to the feature point;

after the feature points are obtained, a feature description is established for the feature points, and the feature description can be called a feature descriptor.

As an example, the feature descriptor may include a SURF (FAST up robust feature) descriptor, an orb (organized FAST and Rotated brief) descriptor, and the like.

In an implementation, the feature point may be described in combination with a feature descriptor, pixel coordinates of the feature point, and geographic location coordinates of the feature point, i.e., the description of the feature point may be denoted as x_i＝{d_i，c_i，f_iIn which d is_iA feature descriptor of the ith feature point, which is an n-dimensional vector; c. C_iPixel coordinates of the ith characteristic point; f. of_iThe geographical position coordinates of the ith feature point can be represented by a three-dimensional vector, namely three-dimensional position data.

The set of all feature points and corresponding feature descriptors in the work area map data may be: x ═ X₁，x₂，x₃，…，x_n}。

And a substep S24 of generating a feature dictionary corresponding to the feature points.

After the feature points and the corresponding feature descriptors are obtained, a feature dictionary corresponding to each feature point can be created, and the feature dictionary is used for rapidly detecting the current approximate position of the unmanned aerial vehicle, and further performing high-precision image matching and positioning of the unmanned aerial vehicle.

In particular implementations, a loop detection algorithm may be used to determine the feature dictionary corresponding to each feature point, for example, one loop detection algorithm may include bag of words (bag of words) of DBOW 2.

And 104, performing route planning on the map data of the operation area based on the first characteristic information to obtain route information.

Through steps 101-103, the region to be worked may include a piece of working region map data, a set of feature points and corresponding feature descriptors, and a feature dictionary corresponding to each feature point. Subsequently, a route planning may be performed for the area to be worked.

In specific implementation, a corresponding planning scheme can be selected according to operation requirements, and route planning can be directly performed on map data of an operation area. For example, if the operation requirement is that the operation of the unmanned aerial vehicle completely covers the area to be operated, a snake planning method can be adopted for route planning.

It should be noted that the embodiment of the present invention is not limited to the planning method of the route, and a series of discrete three-dimensional coordinate points (i.e. waypoints) and corresponding attribute information may be used to describe the planning result, i.e. the track information, regardless of the planning method.

In a preferred embodiment of the present invention, step 104 may further include the following sub-steps:

substep S31, performing route planning on the map data of the operation area to obtain a plurality of waypoints;

after the route planning is carried out on the operation area, a plurality of waypoints can be obtained, and the routes of the operation area can be obtained by connecting the plurality of waypoints.

The departure point of the flight path can be specified in the area to be operated according to any requirement, and the determining mode of the departure point is not limited in the embodiment of the invention.

A substep S32 of respectively obtaining three-dimensional coordinate data of the waypoints;

each waypoint may be described as a three-dimensional coordinate datum denoted as fi.

As an example, the three-dimensional coordinate data may include a plane coordinate and a height coordinate.

In a preferred embodiment of the present invention, the sub-step S32 further includes the following sub-steps: acquiring the geographic position coordinates of the pixels corresponding to the waypoints as plane coordinates; acquiring the height of a pixel corresponding to the waypoint; acquiring the ground height of the unmanned aerial vehicle relative to the ground during planning; and taking the height and the sum of the height to the ground as a height coordinate.

In order to realize the functions of positioning and navigation, the distance between two waypoints in the track information cannot be too sparse, and in one implementation mode, the distance between the two waypoints can be determined by the overlapping degree of two images continuously acquired by an airborne camera.

In practice, the above-mentioned overlap may be set to be more than 30%.

In a specific implementation, the relationship between the overlapping degree and the ground altitude of the unmanned aerial vehicle and the attribute of the onboard camera can be expressed as follows:

J＝Hw_pn_p(1-p)/f

wherein J is the time interval between two images shot by the onboard camera, H is the ground height of the unmanned aerial vehicle, and W_pIs the size of the pixel, n_pIs the number of pixels, P is the degree of overlap, and f is the focal length of the onboard camera.

The above-mentioned pixel size and the number of pixels refer to the pixel size and the number of pixels of the vertical and horizontal coordinates of the sensor parallel to the flight direction.

A substep S33 of, for each waypoint, collecting a feature descriptor and a feature dictionary of a region that can be covered by the field angle of the onboard camera with the pixel coordinates of the waypoint as the center, and generating a feature set;

in the embodiment of the invention, in order to meet the navigation requirement of the unmanned aerial vehicle, the characteristic set of the waypoint can be determined.

Specifically, for each waypoint, a feature descriptor and a feature dictionary of an area which can be covered by a field angle FOV of the onboard camera may be collected with a pixel coordinate of the waypoint as a center, and a set of the collected feature descriptor and feature dictionary is recorded as a feature set, which is di ═ xij, tij }, where xij is a feature descriptor of a jth feature point corresponding to the waypoint i, and tij is a feature dictionary of a jth feature point corresponding to the waypoint i.

And a substep S34 of organizing the three-dimensional coordinate data and the feature set into attribute information of the waypoint.

After obtaining the three-dimensional coordinate data and the feature set of each waypoint, the three-dimensional coordinate data and the feature set may be organized into attribute information of the waypoint, that is, the description of the waypoint is hi ═ di, fi }.

The track information obtained in step 104 can be described as H ═ { H1, H2, H3, …, hn }.

In the embodiment of the invention, the air route planning can be carried out on the operation area according to the operation area map data to be operated extracted from the map data and the first characteristic information acquired from the operation area map data, and the flight path information is acquired.

Referring to fig. 2, a flow chart of steps of an embodiment of the method of positioning of the present invention is shown, which when applied to the above-described unmanned aerial vehicle, may include the steps of:

step 201, acquiring track information, wherein the track information comprises attribute information of a plurality of waypoints;

as a preferred example of the embodiment of the present invention, the attribute information of the waypoint may include, but is not limited to: the navigation system comprises a feature set associated with a navigation point and three-dimensional coordinate data of each navigation point, wherein the feature set can comprise a plurality of feature points and a first feature dictionary corresponding to each feature point.

For the generation manner of the track information, reference may be made to the method in the embodiment of fig. 1, which is not described herein again.

In implementation, after generating the flight path information according to the embodiment of fig. 1, the flight path information may be stored in a server at the cloud end, a database of the unmanned aerial vehicle, a ground station, a remote controller, and the like.

Therefore, the unmanned aerial vehicle may request the track information from the cloud server, extract the track information from the local database, and receive the track information sent by the ground station or the remote controller, which is not limited in this embodiment of the present invention.

Step 202, acquiring real-time image data acquired by the airborne camera according to a preset interval;

in the embodiment of the invention, after the unmanned aerial vehicle is started, the onboard camera can be started, and the onboard camera is controlled to continuously acquire real-time image data according to the preset interval.

Step 203, determining the current position information of the unmanned aerial vehicle based on the flight path information and the real-time image data.

Aiming at the real-time image data acquired by the unmanned aerial vehicle each time, the unmanned aerial vehicle can be compared with the flight path information, so that the real-time position information of the unmanned aerial vehicle is determined according to the flight path information and the real-time image data.

In a preferred embodiment of the present invention, step 203 further comprises the following sub-steps:

a substep S41, judging whether the unmanned aerial vehicle deviates from a route corresponding to the flight path information based on the flight path information and the real-time image data;

firstly, the unmanned aerial vehicle can judge whether the current unmanned aerial vehicle is near a planned route according to the track information and the currently acquired real-time image data, namely whether the current unmanned aerial vehicle deviates from the route corresponding to the track information.

In a preferred embodiment of the present invention, the sub-step S41 further includes the following sub-steps:

substep S411, extracting second feature information from the real-time image data;

as a preferred embodiment of the present invention, the second feature information may include a plurality of second feature points, a second feature dictionary corresponding to each second feature point, a second feature descriptor, and the like.

For the process of extracting the second feature information from the real-time image data, reference may be made to the process of extracting the first feature information in step 103 in the embodiment of fig. 1, which is not described herein again.

Substep S412, calculating a matching degree between the second feature dictionary and each first feature dictionary in the feature set;

after second feature dictionaries corresponding to a plurality of second feature points in the real-time image data are obtained, each second feature dictionary can be matched with each first feature dictionary in the feature set respectively, so that the matching degree of each second feature dictionary with each first feature dictionary in each feature set is calculated.

In the embodiment of the present invention, the method for calculating the matching degree is not limited, and for example, the method for calculating the similarity may be used to calculate the matching degree between the first feature dictionary and the second feature dictionary.

And in the substep S413, if the first feature dictionary with the matching degree larger than the preset threshold does not exist, judging that the unmanned aerial vehicle deviates from the route corresponding to the flight path information.

After the matching degree of all the second feature dictionaries and each first feature dictionary in the feature set is calculated, if the matching degree which is larger than a preset threshold value does not exist, the route corresponding to the unmanned aerial vehicle deviation track information can be judged, wherein the deviation route means that the unmanned aerial vehicle deviates from at least one route belt.

A substep S42, if the unmanned aerial vehicle does not deviate from the air route, acquiring temporary positioning information of the unmanned aerial vehicle;

in a preferred embodiment of the present invention, the sub-step S42 further includes the following sub-steps:

If the matching degree is larger than the preset threshold value, the unmanned aerial vehicle can be judged to be on the route corresponding to the track information, namely the unmanned aerial vehicle is judged not to deviate from the route corresponding to the track information. At this time, feature points corresponding to the first feature dictionary with the matching degree greater than the preset threshold value may be determined as matching feature points, and the region determined by the matching feature points is used as temporary positioning information of the unmanned aerial vehicle, where the temporary positioning information is a rough flight position of the unmanned aerial vehicle.

It should be noted that, if the unmanned aerial vehicle flies according to the flight lines sequentially, because the adjacent waypoints to be matched are determined, the rough flight position of the unmanned aerial vehicle can be determined according to the waypoints already flown without using the feature dictionary.

And a substep S43 of determining current position information of the unmanned aerial vehicle based on the temporary positioning information.

After the temporary positioning information of the unmanned aerial vehicle is obtained, the accurate position information of the unmanned aerial vehicle can be obtained according to the rough positioning information.

In a preferred embodiment of the present invention, the sub-step S43 further includes the following sub-steps:

substep S431, determining neighborhood waypoints based on the temporary positioning information, and acquiring a first feature descriptor associated with the neighborhood waypoints;

substep S432, matching the second feature descriptor with the first feature descriptor, and determining a matched feature point sequence;

after the temporary positioning information of the unmanned aerial vehicle is obtained, the adjacent waypoints, namely the neighborhood waypoints, can be determined according to the temporary positioning information, and the first feature descriptors associated with the neighborhood waypoints are obtained.

Subsequently, the second feature descriptors corresponding to the real-time image data can be respectively matched with the first feature descriptors associated with the neighborhood waypoints, and a group of feature point sequences is obtained after matching is completed.

In one embodiment, the sequence of feature points may include two-dimensional coordinates of matched feature points of the real-time image data and three-dimensional coordinates of feature points of the corresponding matched work area map data.

And a substep S433, determining the current position information of the unmanned aerial vehicle based on the matched characteristic point sequence.

In a preferred embodiment of the present invention, the sub-step S433 may further include: and calculating the matched characteristic point sequence by adopting a preset positioning algorithm, and determining the current position information of the unmanned aerial vehicle.

In one embodiment, the predetermined positioning algorithm may include, but is not limited to, a PNP (camera pose estimation) algorithm.

In a preferred embodiment of the present invention, after obtaining the current position information of the unmanned aerial vehicle, the method may further include the following steps:

determining the position offset of the current position information and the track information; and correcting the current position information based on the position offset.

In a specific implementation, a position difference between the current position information and a predetermined waypoint position of the flight path information may be calculated as a position offset amount, and the position offset amount is input to the flight controller to correct the current deviation.

In the embodiment of the invention, the unmanned aerial vehicle is roughly positioned by using the feature dictionary, and more accurate current position information is obtained for the following feature descriptor matching and PNP algorithm, so that the positioning and navigation of the unmanned aerial vehicle when no GPS signal or the GPS signal is covered are realized, and the dependence of the unmanned aerial vehicle on the GPS is reduced.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 3, a block diagram of an embodiment of the route generation device of the unmanned aerial vehicle of the present invention is shown, which may include the following modules:

a map data acquisition module 301, configured to acquire map data;

a working area map data acquisition module 302, configured to extract working area map data to be worked from the map data;

a first feature information obtaining module 303, configured to obtain first feature information from the work area map data;

and the route planning module 304 is configured to perform route planning on the map data of the operation area based on the first feature information, so as to obtain route information.

In a preferred embodiment of the present invention, the operation area map data obtaining module 302 further includes the following sub-modules:

In a preferred embodiment of the present invention, the first feature information at least includes a feature point, a feature descriptor corresponding to the feature point, and a feature dictionary corresponding to the feature point, and the first feature information obtaining module 303 further may include the following sub-modules:

In a preferred embodiment of the present invention, the unmanned aerial vehicle includes an onboard camera, and the track information includes a waypoint and attribute information of the waypoint; the route planning module 304 may further include sub-modules to:

In a preferred embodiment of the present invention, in the track information, the distance between two waypoints is determined by the overlapping degree of two images continuously acquired by the onboard camera.

In a preferred embodiment of the present invention, the three-dimensional coordinate data obtaining sub-module further includes:

For the apparatus embodiment of fig. 3, since it is basically similar to the method embodiment described above, the description is simple, and for the relevant points, reference may be made to part of the description of the method embodiment.

Referring to fig. 4, a block diagram of an embodiment of an unmanned aerial vehicle according to the present invention is shown, where the unmanned aerial vehicle may include an onboard camera, and in the embodiment of the present invention, the unmanned aerial vehicle may further include the following modules:

a track information obtaining module 401, configured to obtain track information, where the track information includes attribute information of multiple waypoints;

a real-time image acquisition module 402, configured to acquire real-time image data acquired by the onboard camera at preset intervals;

and a positioning module 403, configured to determine current position information of the unmanned aerial vehicle based on the track information and the real-time image data.

In a preferred embodiment of the present invention, the positioning module 403 may include the following sub-modules:

In a preferred embodiment of the present invention, the attribute information includes a feature set associated with the waypoint, where the feature set includes a plurality of feature points and a first feature dictionary corresponding to each feature point;

the lane departure judging sub-module may further include the following units:

In a preferred embodiment of the present invention, the temporary positioning information obtaining sub-module is further configured to:

In a preferred embodiment of the present invention, the feature set further includes a first feature descriptor corresponding to each feature point, the second feature information includes a second feature descriptor, and the current location information obtaining sub-module further includes the following units:

In a preferred embodiment of the present invention, the position determining unit is further configured to:

In a preferred embodiment of the embodiments of the present invention, the unmanned aerial vehicle further includes the following modules:

For the aircraft embodiment of fig. 4, since it is substantially similar to the method embodiment described above, the description is relatively simple, and reference may be made to part of the description of the method embodiment for relevant points.

In addition, the embodiment of the present invention also discloses an aircraft, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the method of the above embodiment when executing the program, and the steps include:

In addition, the embodiment of the invention also discloses a computer readable storage medium, on which a computer program is stored, and the program is executed by a processor to realize the steps of the method of the embodiment.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The method and the device for generating and positioning the flight path of the unmanned aerial vehicle and the unmanned aerial vehicle are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A method of generating a course for an unmanned aerial vehicle, the unmanned aerial vehicle comprising an onboard camera, the method comprising:

acquiring map data;

extracting map data of a working area to be operated from the map data;

acquiring first characteristic information from the map data of the operation area; the first feature information at least comprises feature points, feature descriptors corresponding to the feature points and a feature dictionary corresponding to the feature points;

performing route planning on the map data of the operation area based on the first characteristic information to obtain route information, wherein the route information comprises route points and attribute information of the route points;

performing route planning on the map data of the operation area based on the first characteristic information to obtain route information, wherein the route planning comprises the following steps:

respectively acquiring three-dimensional coordinate data of the waypoints;

2. The method according to claim 1, wherein the step of extracting the work area map data to be worked from the map data includes:

acquiring a plurality of measurement points marked in the map data;

3. The method according to claim 1 or 2, wherein the step of acquiring first feature information from the work area map data includes:

dividing the map data of the operation area into a plurality of block data;

respectively extracting a preset number of feature points from each block data;

generating a feature descriptor corresponding to the feature point;

and generating a feature dictionary corresponding to the feature points.

4. The method according to claim 1, characterized in that in the track information, the distance between two waypoints is determined by the degree of overlap of two images successively acquired by the onboard camera.

5. The method according to claim 1 or 4, wherein the step of respectively acquiring three-dimensional coordinate data of the waypoints comprises:

acquiring the height of a pixel corresponding to the waypoint;

6. A method of positioning in an unmanned aerial vehicle, the unmanned aerial vehicle including an onboard camera, the method comprising:

determining current position information of the unmanned aerial vehicle based on the flight path information and the real-time image data;

wherein the step of generating the track information comprises:

acquiring map data;

extracting map data of a working area to be operated from the map data;

acquiring first feature information from the map data of the operation area, wherein the first feature information at least comprises feature points, feature descriptors corresponding to the feature points and a first feature dictionary corresponding to the feature points;

respectively acquiring three-dimensional coordinate data of the waypoints;

for each waypoint, collecting a feature descriptor and a first feature dictionary of an area which can be covered by the field angle of the airborne camera by taking the pixel coordinates of the waypoint as a center, and generating a feature set;

7. The method of claim 6, wherein the step of determining current position information of the UAV based on the track information and the real-time image data comprises:

8. The method of claim 7, wherein the step of determining whether the UAV deviates from a route corresponding to the track information based on the track information and the real-time image data comprises:

9. The method of claim 8, wherein the step of obtaining temporary positioning information for the UAV comprises:

10. The method according to claim 8 or 9, wherein the feature set further comprises a first feature descriptor corresponding to each feature point, the second feature information comprises a second feature descriptor, and the step of determining the current position information of the UAV based on the temporary positioning information comprises:

11. The method of claim 10, wherein the step of determining the current position information of the UAV based on the matched sequence of feature points comprises:

12. The method according to any one of claims 6-11, further comprising:

and correcting the current position information based on the position offset.

13. An airline generation apparatus of an unmanned aerial vehicle, the unmanned aerial vehicle including an onboard camera, the apparatus comprising:

the map data acquisition module is used for acquiring map data;

the first characteristic information acquisition module is used for acquiring first characteristic information from the map data of the operation area; the first feature information at least comprises feature points, feature descriptors corresponding to the feature points and a feature dictionary corresponding to the feature points;

the route planning module is used for planning routes of the map data of the operation area based on the first characteristic information to obtain route information, and the route information comprises route points and attribute information of the route points;

wherein the route planning module comprises:

14. The apparatus of claim 13, wherein the work area map data acquisition module comprises:

15. The apparatus according to claim 13 or 14, wherein the first feature information acquiring module comprises:

16. The apparatus according to claim 13, wherein in the track information, the distance between two waypoints is determined by the degree of overlap of two images continuously acquired by the onboard camera.

17. The apparatus of claim 13 or 16, wherein the three-dimensional coordinate data acquisition sub-module comprises:

18. An unmanned aerial vehicle, the unmanned aerial vehicle comprising an onboard camera, the unmanned aerial vehicle further comprising:

the positioning module is used for determining the current position information of the unmanned aerial vehicle based on the flight path information and the real-time image data;

the map data acquisition module is used for acquiring map data;

the first characteristic information acquisition module is used for acquiring first characteristic information from the map data of the operation area; the first feature information at least comprises feature points, feature descriptors corresponding to the feature points and a first feature dictionary corresponding to the feature points;

the route planning module comprises a planning submodule, a three-dimensional coordinate data acquisition submodule, a feature set acquisition submodule and an organization submodule, wherein,

the characteristic set acquisition submodule is used for acquiring a characteristic descriptor and a first characteristic dictionary of an area which can be covered by the field angle of the airborne camera by taking the pixel coordinates of the waypoint as the center for each waypoint to generate a characteristic set;

19. The UAV of claim 18 wherein the positioning module comprises:

20. The UAV of claim 19, wherein the attribute information comprises a feature set associated with the waypoint, the feature set comprising a plurality of feature points and a first feature dictionary corresponding to each feature point;

the lane departure judgment submodule includes:

21. The UAV of claim 20 wherein the temporary positioning information acquisition sub-module is further configured to:

22. The unmanned aerial vehicle of claim 20 or 21, wherein the feature set further comprises a first feature descriptor corresponding to each feature point, the second feature information comprises a second feature descriptor, and the current position information acquisition sub-module comprises:

23. The UAV of claim 22 wherein the position determination unit is further configured to:

24. The unmanned aerial vehicle of any of claims 18-23, further comprising:

25. An aircraft comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 and/or 6 to 12 when executing the program.

26. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5 and/or 6 to 12.