CN111637871A - Unmanned aerial vehicle camera steady self-checking method and device based on rotary flight - Google Patents

Abstract

The invention discloses a method and a device for robust self-checking and calibration of an unmanned aerial vehicle based on rotating flight, belonging to the field of aerial photogrammetry. The method performs a local area of the survey area at the same flight height before or after the formal photography is completed. The rotating flight takes a small number of images to form a dataset dedicated to camera calibration. Then, the self-calibration beam method adjustment is performed on this set of data separately, and the azimuth elements in the camera (including focal length, image principal point offset, and objective lens distortion parameters) are solved. The invention mainly aims at the problem of multiple solutions in the camera self-checking and calibration due to the strong correlation between the camera's inner orientation element and the image outer orientation element in the existing camera self-checking and calibration method, and obtains accurate and robust camera self-checking and calibration results. . At the same time, due to the small amount of image data involved in the self-calibration calculation, the computing memory required in the self-calibration process can be greatly reduced, the time for self-calibration processing is reduced, and high-precision self-calibration is provided for subsequent large-scale data processing. parameter.

Description

Robust self-checking method and device for drone camera based on rotating flight

technical field

The invention belongs to the field of aerial photogrammetry, and more particularly relates to a method and device for robust self-checking and calibration of UAV cameras based on rotational flight, in particular to rotational flight under a certain camera inclination.

Background technique

Photogrammetry is the science and technology of recovering three-dimensional information of objects from images, in which the internal orientation elements (camera focal length, image principal point coordinates) and external orientation elements (position and attitude) of the image are recovered through the area network adjustment. The premise and foundation of information extraction. At present, UAV photogrammetry has become a new type of photogrammetry and is widely used. However, UAV photogrammetry generally uses non-measuring cameras, and the camera lens often has a certain degree of optical distortion (generally radial distortion (k ₁ , k ₂ , k ₃ ) and tangential distortion (p ₁ , p ₂ )), at the same time, the camera's image principal point coordinates (x ₀ , y ₀ ) and the focal length f also often change with time and environment. Inaccurate in-camera orientation elements will cause errors in the measured or matched image point coordinates, which will affect the accuracy of image position and attitude parameter calculation. Therefore, accurate geometric calibration of the camera is a necessary part of image photogrammetry processing, and it is also the premise to ensure the accuracy of the adjustment solution.

The traditional close-range photogrammetry camera calibration method mainly uses the laboratory three-dimensional calibration field to calibrate the camera. The calibration accuracy of the 3D calibration yard is the highest, but the establishment cost of the calibration yard is too high, and there are many requirements, and instrument maintenance and high-precision measurement are required at intervals to ensure the accuracy of the calibration yard. For most small-scale, short-term projects, the establishment and maintenance of the inspection yard will far exceed the project budget. In order to reduce the cost of camera calibration, Zhang Zhengyou proposed that the checkerboard self-calibration method is widely used in camera calibration. The method assumes that the plane calibration board is placed on the horizontal plane of the world coordinate system where it is located, obtains the initial values of the camera parameters through the linear imaging model, then gives the objective function considering the nonlinear distortion based on the nonlinear imaging model, and finally uses the non-linear imaging model. Linear optimization method to obtain the optimal solution of camera parameters. This calibration method has good robustness and practicability, but the calibration accuracy is low. Regardless of the experimental field calibration method or the checkerboard calibration method, they are only suitable for cameras that shoot at short distances, not for drone cameras that focus at infinity. Therefore, an outdoor discrete target placement method was proposed. This method can make the scene larger, so as to ensure that the UAV camera can obtain a clear calibration image when focusing at infinity, but this method has high requirements for terrain. It is only suitable for large-scale production tasks, and has great limitations for data processing in urban areas.

The above scene-based camera calibration methods all require more time to perform camera calibration before data processing, which reduces the efficiency of data production and processing. Therefore, some camera self-calibration methods that do not need to calibrate the site have been introduced successively, such as the camera calibration method based on vanishing point in computer vision, the self-calibration method based on Kruppa equation, and the self-calibration method based on beam adjustment in photogrammetry. school method. These methods do not require control fields such as checkerboards and off-field targets, and only need to establish a constraint equation between two or more images, without considering the photoreconstruction process of the image sequence, so they are widely used in photogrammetry and computer software. , such as PhotoScan, ContextCapture, Pix4dMapper and other 3D modeling software. However, this method also has some defects. For example, when the amount of image data is large, the infinity plane consistency of all images relative to the determined projective space cannot be guaranteed, and the stability of the calibration algorithm will be affected. The calculation amount of the equation is too large, and the convergence after nonlinear optimization is not good. In particular, it should be pointed out that due to the strong correlation between camera calibration parameters, different methods for solving very large equations and different selection of initial values, different modeling software will result in camera self-calibration results for the same set of image data. The difference is large, which further reflects the instability of the solution of this type of method.

SUMMARY OF THE INVENTION

In view of the above defects or improvement needs of the prior art, the present invention proposes a robust self-checking method and device for a UAV camera based on rotating flight, thereby solving the problem of camera self-checking parameters in the existing camera self-checking method. There is a strong correlation between the camera and the external orientation elements, which leads to the technical problems of multiple solutions and unstable self-calibration results in the camera internal orientation elements.

In order to achieve the above object, according to one aspect of the present invention, a robust self-checking method for a UAV camera based on rotation flight is provided, including:

(1) Adjust the drone to the target height above a certain ground feature;

(2) setting the tilt angle of the camera when the drone is rotating and flying;

(3) Estimate the target radius of the rotating flight from the target height and the inclination angle, and construct a rotating flight path, wherein the center of the rotating flight path is the center of the feature, and the radius of the rotating flight path is the target radius;

(4) according to the rotating flight path, align the camera at the inclination angle with the feature to take a ring shot, and collect image data;

(5) Use the image data to perform camera self-calibration, and obtain relevant parameters of the camera robustly.

Preferably, step (1) includes:

Before or after the conventional image acquisition by the drone in the survey area, adjust the drone to the target height above the ground features in the survey area, wherein the target height needs to be able to ensure that the drone shoots ground features in the survey area.

Preferably, the target radius R of the rotating flight is estimated by R≈H*tan(θ), where H is the target height and θ is the inclination angle.

Preferably, step (4) includes:

According to the rotating flight path, the camera is aimed at the feature at the inclination angle, and the camera is adjusted to place the feature in the middle of the image, and image data is collected by taking a circle shot at a constant speed.

Preferably, step (5) includes:

Using the image data, the self-calibration beam adjustment method is used to perform camera self-calibration, and the relevant parameters of the camera are obtained robustly.

According to another aspect of the present invention, there is provided a robust self-calibration device for drone cameras based on rotational flight, comprising:

The setting module is used to adjust the drone to the target height above a certain ground feature, and set the tilt angle of the camera when the drone is rotating and flying;

A path construction module, used for estimating the target radius of the rotating flight from the target height and the inclination angle, and constructing a rotating flight path, wherein the center of the rotating flight path is the center of the feature, and the rotating flight path The radius of is the target radius;

an image acquisition module, configured to align the camera at the inclination angle with the characteristic object to perform a ring shot according to the rotating flight path, and collect image data;

The self-calibration module is used to perform camera self-calibration by using the image data, and obtain relevant parameters of the camera stably.

Preferably, the setting module is used to adjust the drone to a target height above the ground features in the survey area before or after the drone is used for conventional image acquisition in the survey area, wherein the target height is It needs to be able to ensure that the drone can photograph the ground features in the survey area.

Preferably, the target radius R of the rotating flight is estimated by R≈H*tan(θ), where H is the target height and θ is the inclination angle.

Preferably, the image acquisition module is specifically configured to align the camera with the feature at the tilt angle according to the rotating flight path, and adjust the camera to place the feature in the middle of the image , perform ring shooting at a constant speed, and collect image data.

According to another aspect of the present invention, there is provided a computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of any one of the above-mentioned methods are implemented.

In general, compared with the prior art, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

The invention is simple to operate, easy to implement, and only needs a small amount of data to solve the robust self-checking parameters, which can greatly reduce the memory consumed in the adjustment and solve, and improve the efficiency of self-checking processing. At the same time, the present invention fully considers the problem of strong correlation of unknowns in existing self-calibration methods, and uses tilt data based on rotating flight to reduce the correlation of unknowns in the equation and improve the precision of self-calibration.

Description of drawings

1 is a schematic flowchart of a method for robust self-checking and calibration of a UAV camera based on rotational flight provided by an embodiment of the present invention;

2 is a schematic diagram of the position and attitude of a camera when an unmanned aerial vehicle is flying around according to an embodiment of the present invention, wherein R is the rotating flight radius, H is the flying height of the unmanned aerial vehicle, and θ is the camera tilt angle;

Fig. 3 is a kind of UAV track distribution diagram during rotary flight provided by an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an apparatus provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

In the current self-calibration method, there is a strong correlation between the camera self-calibration parameters and the external orientation elements of the image, which leads to the multiple solutions of the camera's internal orientation elements and the instability of the self-calibration results. The present invention proposes a method. A robust self-calibration method for UAV cameras based on rotary flight. This method uses rotary flight to acquire a small amount of images at a certain ground point, and reduces the correlation between internal and external orientation elements in the process of self-calibration and adjustment calculation. In order to achieve the purpose of robustly solving the self-calibration parameters. The core idea is that before each flight mission or after the completion of the mission, at the height where the ground and objects in the survey area can be guaranteed to be photographed, a small amount of images are shot and obtained based on the rotation flight for the local area of the ground, and then this group is obtained. The self-calibration beam method adjustment is performed on the data of the rotating flight separately to obtain robust camera self-calibration parameters. Due to the large changes in the photographing direction and angle of the images obtained by the rotating flight, the strong correlation between the inner and outer azimuth elements of the image is broken, so the accuracy of the self-calibration and calculation of the azimuth elements in the camera is high.

Example 1

As shown in FIG. 1 is a schematic flowchart of a robust self-calibration method for a UAV camera based on rotation flight provided by an embodiment of the present invention. The method shown in FIG. 1 includes the following steps:

S1: Adjust the drone to the target height above a ground feature;

Further, step (1) includes:

Before or after the conventional image acquisition by the drone in the survey area, adjust the drone to the target height above the ground features in the survey area, where the target height needs to be able to ensure that the drone can capture the ground features of the survey area. thing.

S2: Set the tilt angle of the camera when the drone is rotating and flying;

In the embodiment of the present invention, the tilt angle may be determined according to the actual situation, and the embodiment of the present invention does not make a unique determination.

In the embodiment of the present invention, the preferred inclination angle is 30°.

S3: Estimate the target radius of the rotating flight from the target height and the inclination angle, and construct a rotating flight path, wherein the center of the rotating flight path is the center of the feature, and the radius of the rotating flight path is the target radius;

As shown in FIG. 2 , in this embodiment of the present invention, the target radius R of the rotating flight can be estimated by R≈H*tan(θ), where H is the target height and θ is the tilt angle, where ≈ in the formula represents The value of the radius R can be close to the value of H*tan(θ), that is to say, the difference between the value of R and the value of H*tan(θ) is within a preset range, and the preset range can be determined according to The actual situation is confirmed.

(4) According to the rotating flight path, align the camera at an oblique angle to the feature to take a ring shot, and collect image data;

As shown in Figure 3, in the embodiment of the present invention, step (4) includes:

According to the rotating flight path, the camera is aimed at the feature at an inclination angle θ, and the camera is adjusted to place the feature in the middle of the image, and the image data is collected at a constant speed.

Among them, the target characteristic object is mainly used as a reference mark in the embodiment of the present invention, so that the drone operator can easily judge whether the flight trajectory is deviated and whether the camera attitude is right at the position.

(5) Use image data to perform camera self-calibration, and obtain relevant parameters of the camera robustly.

In the embodiment of the present invention, step (5) includes:

Using image data, the self-calibration beam adjustment method is used to perform camera self-calibration, and the relevant parameters of the camera are obtained robustly, such as parameters including focal length, image principal point offset, and objective lens distortion.

The invention can effectively reduce the influence of the correlation between the self-calibration parameters and the azimuth elements outside the image during the adjustment calculation, and obtain accurate camera self-calibration results. At the same time, due to the small amount of image data involved in self-checking and calibration, it can greatly reduce the computing memory required in the self-checking and calibration process, reduce the processing time of self-checking and calibration, and provide high-precision self-checking and calibration parameters for subsequent large-scale data processing.

Embodiment 2

4 is a schematic structural diagram of a device provided by an embodiment of the present invention, including:

The setting module 401 is used to adjust the drone to the target height above a certain ground feature, and set the tilt angle of the camera when the drone is rotating and flying;

The path construction module 402 is used for estimating the target radius of the rotary flight by the target height and the inclination angle, and constructs the rotary flight path, wherein the center of the rotary flight path is the center of the feature, and the radius of the rotary flight path is the target radius;

The image acquisition module 403 is used for aligning the camera at an oblique angle with the feature object according to the rotating flight path to capture image data;

The self-calibration module 404 is configured to perform camera self-calibration using image data, and obtain relevant parameters of the camera stably.

Further, the above-mentioned setting module 401 is used to adjust the UAV to the target height above the ground features in the survey area before or after the conventional image acquisition by the UAV in the survey area, wherein the target height needs to be able to ensure no The man-machine photographed the ground features in the survey area.

Further, the target radius R of the rotating flight is estimated by R≈H*tan(θ), where H is the target height and θ is the inclination angle.

Further, the above-mentioned image acquisition module 403 is specifically used for aligning the camera at an oblique angle to the feature according to the rotating flight path, and adjusting the camera to place the feature in the middle of the image, performing circular shooting at a constant speed, and collecting image data.

For the specific implementation of each module, reference may be made to the descriptions of the foregoing method embodiments, which will not be repeated in the embodiments of the present invention.

Embodiment 4

The present application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disc, Server, App Store, etc., on which computer programs are stored, programs When executed by the processor, the robust self-checking method of the UAV camera based on rotation flight in the method embodiment is realized.

It should be pointed out that, according to the needs of implementation, the various steps/components described in this application may be split into more steps/components, or two or more steps/components or partial operations of steps/components may be combined into new steps/components to achieve the purpose of the present invention.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (10)

1. The utility model provides an unmanned aerial vehicle camera is steady self-checking school method based on rotatory flight which characterized in that includes:

(1) adjusting the unmanned aerial vehicle to a target height above a feature on a certain ground;

(2) setting the inclination angle of a camera of the unmanned aerial vehicle during rotary flight;

(3) estimating the target radius of the rotating flight according to the target height and the inclination angle, and constructing a rotating flight path, wherein the circle center of the rotating flight path is the center of the feature, and the radius of the rotating flight path is the target radius;

(4) according to the rotating flight path, aligning the camera to the feature object at the inclination angle for circular shooting, and acquiring image data;

(5) and performing camera self-calibration by using the image data, and stably acquiring relevant parameters of the camera.

2. The method of claim 1, wherein step (1) comprises:

before surveying district with unmanned aerial vehicle and carrying out conventional image acquisition or after gathering, adjust unmanned aerial vehicle to survey district ground feature overhead target height department, wherein, target height needs can guarantee that unmanned aerial vehicle shoots survey district ground feature.

3. Method according to claim 1 or 2, characterized in that the target radius R of the rotating flight is estimated by R ≈ H tan (θ), where H is the target height and θ is the inclination angle.

4. The method of claim 3, wherein step (4) comprises:

and aligning the camera to the feature object at the inclination angle according to the rotating flight path, adjusting the camera to place the feature object in the middle position of the image, performing ring shooting at a constant speed, and collecting image data.

5. The method of claim 4, wherein step (5) comprises:

and performing camera self-checking by using the image data and adopting a self-checking light beam adjustment method to robustly acquire relevant parameters of the camera.

6. The utility model provides an unmanned aerial vehicle camera is steady from checking device based on rotatory flight which characterized in that includes:

the device comprises a setting module, a control module and a control module, wherein the setting module is used for adjusting the unmanned aerial vehicle to a target height above a certain ground feature and setting the inclination angle of a camera when the unmanned aerial vehicle rotates and flies;

the path building module is used for estimating the target radius of the rotating flight according to the target height and the inclination angle and building a rotating flight path, wherein the circle center of the rotating flight path is the center of the feature, and the radius of the rotating flight path is the target radius;

the image acquisition module is used for aligning the camera to the feature object at the inclination angle for circular shooting according to the rotating flight path and acquiring image data;

and the self-checking module is used for performing camera self-checking by using the image data and steadily acquiring the related parameters of the camera.

7. The apparatus of claim 6, wherein the setting module is configured to adjust the drone to a target height above the survey area ground feature before or after conventional image capture by the drone for the survey area, wherein the target height is required to ensure that the drone captures the survey area ground feature.

8. The device according to claim 6 or 7, characterized in that the target radius R of the spinning flight is estimated by R ≈ H · (θ), where H is the target height and θ is the inclination angle.

9. The apparatus according to claim 8, wherein the image capturing module is specifically configured to align the camera with the feature at the tilt angle according to the rotating flight path, adjust the camera to place the feature in an intermediate position of an image, perform a circular shooting at a constant speed, and capture image data.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of the preceding claims 1 to 5.

Images