CN103822635A - Visual information based real-time calculation method of spatial position of flying unmanned aircraft - Google Patents

Abstract

The invention discloses a visual information-based real-time calculation method for a spatial position in flight of an unmanned aerial vehicle, belonging to the technical field of digital video image processing. This method is mainly based on the aerial image information and elevation information, and integrates the flight parameter information of the UAV to analyze and identify the aerial position of the UAV. Using the aerial image of the UAV, combined with the flight parameters such as the elevation information of the UAV, the image is corrected, compared with the prior information, and the current position information of the UAV is calculated from the aerial image. The invention aims at the characteristics of the unmanned aerial vehicle, fully utilizes the visual information, and improves the autonomy of the unmanned aerial vehicle.

Description

Real-time calculation method of spatial position in UAV flight based on visual information

technical field

The invention belongs to the technical field of digital video image processing, and in particular relates to a method for real-time calculation of the spatial position of an unmanned aerial vehicle based on visual information.

Background technique

In recent years, with the continuous development of drone technology, drones have not only been widely used in the military, but also gradually extended to civilian applications. In the military, it can be used for aerial reconnaissance, electronic jamming, communication relay, target positioning, battlefield surveillance and border patrol, etc. In civilian use, it can be used for aerial photography, disaster monitoring, geophysical prospecting, aerial photography, etc.

In the past, drones mainly relied on inertial navigation system (Inertial Navigation System, INS) and global positioning system (Global Position System, GPS) for navigation. It is not always available, and even when it is available, it is often not accurate enough for drone navigation.

In addition, the transmission of radio signals and GPS signals is easily blocked, and the anti-interference ability is not strong, so the advantages in military covert reconnaissance are not obvious. According to reports, Iran claimed that it had cracked the GPS signal of the US military, controlled the communication link, and successfully tricked and captured an RQ-170 "Sentinel" unmanned reconnaissance aircraft of the US military performing missions in Iran. On September 13, 2009, a U.S. military MQ-9 "Reaper" drone lost control while performing a mission in the mountains of northern Afghanistan. The U.S. military had no choice but to send fighter jets to shoot it down to prevent it from flying into the airspace of Tajikistan or China. . These accidents occurred because the UAV communication link was blocked or was deciphered and taken over by the enemy, receiving false navigation position information, which reduced its own reliability.

GPS signals are susceptible to interference and control by other countries, and the accuracy of inertial navigation/GPS integrated navigation is limited. It is precisely for these reasons that, in order to improve the autonomous flight and anti-spoofing capabilities of UAVs, people have a great research interest in UAV autonomous navigation, and it has become a hot spot in the recent research field of UAVs. The navigation autonomy is strong, the acquisition of navigation parameters does not depend on external equipment, the amount of information obtained is large, and it is completely provided by the aircraft itself. The anti-interference ability is strong, and the position positioning and identification are more accurate, which is very beneficial to the realization of the autonomous navigation of the aircraft.

At present, visual navigation is mainly based on relative position positioning. For example, when an aircraft is landing, it uses landmarks such as the runway sideline as a reference to continuously adjust the attitude of the aircraft to achieve the purpose of landing safely. There are relatively few methods for navigation absolute position positioning. During the flight of UAVs, GPS/INS integrated navigation is used, and vision-based auxiliary navigation is used to achieve more accurate navigation and positioning. However, in the case of communication link failure, the use of Visual navigation is more important for absolute position positioning.

Contents of the invention

The purpose of the present invention is to solve the above problems, aiming at the characteristics of the UAV itself, a real-time calculation method for the spatial position of the UAV in flight based on visual information is proposed, combined with image processing technology, using image features, comparing prior information, and obtaining no Air position information of man and machine to improve the autonomy of drones.

A method for real-time calculation of spatial position in UAV flight based on visual information, including the following steps:

Step 1, establishing the UAV flight route topography database;

Step 2, waypoint detection;

Step three, data collection;

Step 4, data matching;

Step five, obtain location information.

The advantages of the present invention are:

(1) Utilize the onboard resources and equipment of the UAV to perform absolute positioning of the UAV's spatial position;

(2) Using natural information such as visible light and infrared, it has good concealment;

Description of drawings

Fig. 1 is method flowchart of the present invention;

Fig. 2 is the flow chart of image information matching of the present invention;

Fig. 3 is a diagram of the geometric relationship between the shooting point of the drone and the central point of the landform according to the present invention.

Detailed ways

Specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

The real-time calculation method of the spatial position in the flight of the unmanned aerial vehicle based on visual information of the present invention, as shown in Figure 1, includes the following steps:

Step 1, establishing a geomorphological database of the flight route of the UAV;

According to the pre-planned route of the UAV, within the continuously observable field of view of the UAV, images and location information of various landform environments such as residential areas, vegetation, roads, waters, etc. are extracted, and the color, color, and location information of the landform image are calculated. Various features such as texture, straight line, corner point, SIFT, etc. are stored in the database;

Specifically: first collect waypoints for the planned route, that is, building landmarks such as signal lighthouses, overpasses or skyscrapers, and then calculate and store the point-line features of the waypoints. The steps are as follows:

(1) Obtain the waypoint of the UAV route;

According to the pre-set planning route of the UAV, select the waypoint on the UAV route, and use multi-load equipment to collect the information of the waypoint;

The waypoint types of UAV flight routes include the following: building landmarks such as signal lighthouses, overpasses or skyscrapers, and building landmarks have obvious characteristics and are difficult to replicate. reliability of the method.

According to the planned route, use multi-load equipment to collect various information of waypoints, including landmark image information, corresponding elevation information and navigation position information, etc.;

(2) Calculation features;

According to step (1), the landmark image information of the waypoint is obtained, and the SIFT point feature, Harris corner feature, texture feature, etc. of the waypoint are calculated;

(3) Obtain the geomorphic database;

Steps (1) and (2) are collected and calculated to obtain SIFT point features, Harris corner features, texture features, Hough line features, and location information of landmark images, and store them in the geomorphic database to complete the establishment of the geomorphic database.

The location information of the landmark image includes the waypoint map name, image size, and center coordinates.

According to the structure shown in Table 1, the information is stored in the geomorphic database to complete the establishment of the geomorphic database.

Table 1 Geomorphic database structure

Step 2: The UAV performs waypoint detection during the route flight.

During the flight of the UAV, the waypoints of the route are detected in real time, that is, landmarks such as signal lighthouses and buildings are detected. Since the Harris corner point detection method has the characteristics of high accuracy and good real-time performance, this detection method is used to detect the waypoint landmarks. Detection, matching the detected waypoint with the existing waypoint in the terrain database, if the match is successful, go to step 3, otherwise continue to the next waypoint detection, go to step 2;

Specifically include the following steps:

(1) Waypoint detection

When the UAV flies to a waypoint, the Harris corner detection method is used to obtain the Harris feature of the waypoint. The Harris corner detection method has the characteristics of high accuracy and good real-time performance, taking into account both efficiency and precision. Requirements, false detection rate is low.

(2) Waypoint matching

Match the Harris feature of the acquired waypoint with the Harris feature of the waypoint stored in the landform database. If the match is successful, proceed to step 3, otherwise proceed to the next waypoint detection and perform step 2;

Step 3, waypoint data collection.

For the successfully matched waypoints, the airborne CCD camera is used to collect data. The data includes high-definition aerial images and UAV aerial photography parameters. UAV aerial photography parameters include UAV height H, heading angle α, pitch angle β, roll angle γ and CCD camera platform angle η, azimuth λ;

Step four, data matching.

Match the collected data with the geomorphic database, access the geomorphic database, extract the geomorphic image data at the current location, use image features to detect and match feature points, and calculate the number of matching feature points N.

As shown in Figure 2, data matching includes the following steps:

(1) Aerial image preprocessing;

First of all, due to the influence of weather, temperature, humidity and other factors during the flight of the UAV, there are certain differences between the current aerial image and the airborne landform database. Therefore, the aerial image is first preprocessed, including median filter denoising, grayscale enhancement method, and calculate the SIFT point feature, Harris corner feature, and Hough line feature of the aerial image data;

(2) Access to the geomorphic database;

Access the geomorphic database, and extract the geomorphic image data of the current waypoint position according to the waypoint matching results, including SIFT point features, Harris corner point features and Hough line features;

(3) Image information matching;

Through the above two steps, the features of the aerial image of the waypoint and the landmark image (prior image information) in the landform database are obtained, and more prominent features are selected for matching, and the number of matching feature points N is obtained. If N is greater than the preset threshold, then If the matching is successful, go to step five; otherwise, if the matching fails, go back to step three;

Give examples of prominent features. For example, if Harris corner features are more dense, it means that there may be a residential area, or if a long parallel line appears, there may be a road below. In this case, Harris Corner features or Hough line features are more prominent. In the case that the general features are not outstanding, the SIFT point feature is used to calculate the number of matching feature points N. If N exceeds 10, the matching is considered successful, otherwise it is unsuccessful.

Step five, obtain the location information of the drone.

After the image matching is completed, according to the prior information of the aerial image, the latitude and longitude information of the UAV is reversely calculated to complete the work.

After the image matching is successful, it means that the UAV has reached the position shown in the landmark image (in the prior information) in the geomorphological database, and the geomorphic image is at the center of the aerial image. Read the latitude and longitude coordinates of the center point A of the current UAV (X, Y). The geometric relationship between the UAV shooting point P and the center point of the aerial image, as shown in Figure 3, P' is the projection of P point on the ground, and the formula for back-calculating the UAV latitude and longitude is as follows:

$\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&eta;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&gamma;</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

角为CCD摄像机相对于机体坐标系（北天东坐标系）的偏航角。则点A到点P'的x，y距离为：As shown in Formula 1 and Formula 2, the θ angle is the angle between the CCD camera and the center of gravity of the UAV.

Angle is the yaw angle of the CCD camera relative to the body coordinate system (North East coordinate system). Then the x, y distance from point A to point P' is:

The latitude and longitude coordinates of the UAV are expressed as:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{c}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ]</span>\\ <span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ]</span>\\ <span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ]</span>\\ <span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 111000</span>}<span\; class="notranslate">\; ]</span>\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

In Formula 4, P(x,y) represents the latitude and longitude coordinates of the UAV, point A in the center of the landform is the origin of the coordinates, and the point P' of the ground projection of the UAV may be located in any of the four quadrants of the coordinate system centered on point A , so Formula 4 is the calculation method corresponding to the four quadrants. Indicates the coordinate calculation formula in four different quadrants.

Aiming at the actual application requirements of UAVs, the present invention proposes a technical solution for real-time calculation of the spatial position of UAVs in flight based on visual information, which does not rely on data communication links. data, autonomous location recognition, and improved drone autonomy.

Claims (5)

1. the unmanned plane during flying spatial location real-time computing technique based on visual information, comprises following step:

Step 1, sets up the relief data storehouse in unmanned plane during flying course line;

According to the predefined planning of unmanned plane course line, choose the destination on unmanned plane course line, obtain the landmark image information of destination, calculate SIFT point feature, Harris Corner Feature, textural characteristics, the Hough linear feature of destination, by the positional information of above-mentioned feature and landmark image, be deposited into relief data storehouse;

Step 2, unmanned plane, in airline operation, carries out destination detection;

In unmanned plane during flying, detect in real time the destination in course line, by the destination detecting with in relief data storehouse, deposited destination and mated, carry out step 3 if the match is successful, detect execution step two otherwise proceed next destination;

Step 3, destination data acquisition;

To the destination that the match is successful, utilize airborne ccd video camera image data, data comprise high definition Aerial Images and the unmanned plane parameter of taking photo by plane, the unmanned plane parameter of taking photo by plane comprises unmanned plane height H, course angle α, angle of pitch β, roll angle γ and ccd video camera mesa corners η, position angle λ;

Step 4, Data Matching;

Storehouse is carried out respectively in the data that collect and relief data storehouse and mate, access landforms database, the landforms view data of extraction current location, utilizes characteristics of image to carry out feature point detection and mate, and calculates matching characteristic points N;

Step 5, obtains unmanned plane positional information;

Complete after images match, according to Aerial Images prior imformation, inverse unmanned plane latitude and longitude information, finishes the work.

2. the unmanned plane during flying spatial location real-time computing technique based on visual information according to claim 1, described step 1 specifically comprises following step:

(1) obtain the destination in unmanned plane course line;

According to the predefined planning of unmanned plane course line, choose the destination on unmanned plane course line, destination is chosen buildings terrestrial reference, utilizes the information of multi-load equipment collection destination, obtains the landmark image information of destination;

(2) calculated characteristics;

Obtain the landmark image information of destination according to step (1), calculate SIFT point feature, Harris Corner Feature, textural characteristics, the Hough linear feature of destination;

(3) obtain relief data storehouse;

Step (1), (2) are gathered to the positional information that calculates SIFT point feature, Harris Corner Feature, textural characteristics, Hough linear feature and landmark image, be deposited into relief data storehouse, complete the foundation in relief data storehouse;

The positional information of landmark image comprises destination figure title, picture size, centre coordinate.

3. the unmanned plane during flying spatial location real-time computing technique based on visual information according to claim 1, described step 2 specifically comprises following step:

(1) destination detects

In the time of unmanned plane during flying to destination, adopt Harris angular-point detection method, obtain the Harris Corner Feature of this destination;

(2) destination coupling

By the Harris Corner Feature of the destination obtaining, mate with the Harris Corner Feature of the destination of storing in relief data storehouse, carry out step 3 if the match is successful, detect execution step two otherwise proceed next destination.

4. the unmanned plane during flying spatial location real-time computing technique based on visual information according to claim 1, described step 4 specifically comprises following step:

(1) Aerial Images pre-service;

First, Aerial Images is carried out to pre-service, comprise that medium filtering denoising, gray scale strengthen, and calculate SIFT point feature, Harris Corner Feature, the Hough linear feature of Aerial Images data;

(2) access landforms database;

Access landforms database, according to destination matching result, extracts the landforms view data of current destination position, comprises SIFT point feature, Harris Corner Feature and Hough linear feature;

(3) image information coupling;

By above-mentioned two steps, obtain the feature of landmark image in the Aerial Images of destination and relief data storehouse, selected characteristic is mated, obtain matching characteristic points N, if N is greater than predetermined threshold value, the match is successful, enter step 5, otherwise it fails to match, return to step 3.

5. the unmanned plane during flying spatial location real-time computing technique based on visual information according to claim 1, described step 5 is specially:

After images match success, read current unmanned plane central point A latitude and longitude coordinates (X, Y), establishing P is unmanned plane shooting point, P' be P point in ground projection, inverse unmanned plane longitude and latitude formula is as follows:

$\mathrm{\&theta;}=\frac{\mathrm{\&pi;}}{2}-\mathrm{\&eta;}+\mathrm{\&gamma;}---\left(1\right)$

θ angle is that ccd video camera and unmanned plane center of gravity are pointed to angle,

angle is the crab angle of ccd video camera with respect to body axis system; Put the x of A to some P', y distance is:

Unmanned plane latitude and longitude coordinates is expressed as:

$P(x,y)=\left\{\begin{array}{c}[X-x\cos X/111000,Y-y/111000]\\ [X+x\cos X/111000,Y+y/111000]\\ [X+x\cos X/111000,Y-y/111000]\\ [X-x\cos X/111000,Y+y/111000]\end{array}\right.---\left(4\right)$

P (x, y) represents unmanned plane latitude and longitude coordinates, and formula (4) is four different quadrant internal coordinate computing formula.

Images