CN117252933A - Unmanned aerial vehicle-based camera internal parameter automatic calibration method, system and electronic equipment - Google Patents
Unmanned aerial vehicle-based camera internal parameter automatic calibration method, system and electronic equipment Download PDFInfo
- Publication number
- CN117252933A CN117252933A CN202311265715.5A CN202311265715A CN117252933A CN 117252933 A CN117252933 A CN 117252933A CN 202311265715 A CN202311265715 A CN 202311265715A CN 117252933 A CN117252933 A CN 117252933A
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- unmanned aerial
- camera
- checkerboard
- calibration plate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000009434 installation Methods 0.000 claims abstract description 43
- 238000005457 optimization Methods 0.000 claims abstract description 16
- 230000000007 visual effect Effects 0.000 claims abstract description 11
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000007476 Maximum Likelihood Methods 0.000 description 2
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 description 1
- 206010034719 Personality change Diseases 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Development Economics (AREA)
- Game Theory and Decision Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses an unmanned aerial vehicle-based camera internal parameter automatic calibration method, a system and electronic equipment, and relates to the technical field of camera parameter calibration, wherein the method comprises the following steps: arranging a calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera; in a calibration scene, a target unmanned aerial vehicle moves according to a preset track to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate; the preset track is determined according to the camera internal parameter error and the iterative optimization algorithm; according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera, parameters in the camera are calibrated.
Description
Technical Field
The invention relates to the technical field of camera parameter calibration, in particular to an unmanned aerial vehicle-based camera internal parameter automatic calibration method, a system and electronic equipment.
Background
The current camera internal parameter calibration method is based on detecting and matching angular points on a checkerboard, and estimating internal parameters and distortion parameters of a camera by minimizing a reprojection error. First, the checkerboard pattern on the calibration plate is photographed in a calibration image and the corner points are detected. The corner detection algorithm is typically based on feature point detection methods, such as Harris corner detection or Shi-Tomasi corner detection, after which the detected corner is matched with the theoretical corner on the calibration plate. The geometry and known dimensions of the calibration plate enable corner matching, and the internal and distortion parameters of the camera are estimated by minimizing the re-projection errors. The re-projection error refers to projecting theoretical corner points back to the image plane through calibration parameters, calculating the difference between the theoretical corner points and the actually detected corner points, and iteratively adjusting internal parameters and distortion parameters by using a nonlinear optimization algorithm (such as a Levenberg-Marquardt algorithm) so as to minimize the re-projection error.
During calibration, calibration images need to be taken by hand-holding the calibration plate, which leads to instability in the position and angle of the calibration plate. Since corner detection and matching depend on the exact position of the calibration plate, an unstable calibration plate position may introduce uncertainty and errors in the corner position. In addition, when a hand is holding the calibration plate, the relative posture between the camera and the calibration plate may change. For example, the shake of the hand, the rotation or inclination of the calibration plate, the placement position, and the like all cause the change of the posture. These attitude changes can affect the detection and matching process of the corner points, thereby affecting the estimation results of the internal parameters and distortion parameters. In addition, when the calibration plate is manually held by a person, since the person also needs to manually adjust the position and angle of the calibration plate, it takes a lot of manpower and time for a large number of cameras to complete the internal reference calibration. These problems add complexity and time-consuming to the calibration, limiting the efficiency and accuracy of the calibration.
Disclosure of Invention
The invention aims to provide an unmanned aerial vehicle-based camera internal parameter automatic calibration method, an unmanned aerial vehicle-based camera internal parameter automatic calibration system and electronic equipment, and solves the problem that a large number of camera internal parameters need human intervention in the calibration process.
In order to achieve the above object, the present invention provides the following solutions:
in a first aspect, the invention provides an unmanned aerial vehicle-based camera internal parameter automatic calibration method, which comprises the following steps:
determining the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle and the installation position of the camera in a world coordinate system;
arranging a calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera;
in a calibration scene, a target unmanned aerial vehicle moves according to a preset track to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate; the preset track is determined according to the camera internal parameter error and the iterative optimization algorithm;
and calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera.
Optionally, according to the size information of the checkerboard calibration plate and the visual field information of the camera, arranging calibration scenes specifically includes:
determining a target unmanned aerial vehicle placement range according to the size information of the checkerboard calibration plate and the visual field information of the camera; wherein, the checkerboard calibration board surface is level and smooth and not reflective, and checkerboard calibration board and unmanned aerial vehicle rigid connection.
Optionally, calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, pose data of the target unmanned aerial vehicle and photo data of the checkerboard calibration plate shot by the camera, specifically including:
determining coordinates of the checkerboard corner points in a pixel coordinate system and coordinates in a world coordinate system according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, pose data of the target unmanned aerial vehicle and photo data of the checkerboard calibration plate shot by a camera;
according to the installation position of the camera in the world coordinate system and the pose data of the target unmanned aerial vehicle, determining a rotation matrix and a translation vector of the target unmanned aerial vehicle relative to the camera;
and calculating an internal reference matrix and distortion parameters of the camera according to the rotation matrix and translation vector of the target unmanned aerial vehicle relative to the camera, the coordinate relation of the checkerboard corner points in the pixel coordinate system and the world coordinate system, and the coordinates of the checkerboard corner points in the pixel coordinate system and the coordinates in the world coordinate system.
Optionally, determining coordinates of the checkerboard corner in a pixel coordinate system and coordinates of the checkerboard corner in a world coordinate system according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, pose data of the target unmanned aerial vehicle and photo data of the checkerboard calibration plate shot by the camera specifically includes:
the photo data of the checkerboard calibration plate shot by the camera and the pose data of the unmanned aerial vehicle are subjected to time synchronization;
determining coordinates of the checkerboard corner points in a pixel coordinate system according to the photo data after time synchronization;
and determining coordinates of each angular point coordinate in the checkerboard calibration plate in a world coordinate system according to the pose data after time synchronization and the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle.
Optionally, the preset track determining process is as follows:
acquiring a running track of a target unmanned aerial vehicle determined by the current iteration times;
determining the camera internal parameters determined by the current iteration times according to the driving track of the target unmanned aerial vehicle determined by the current iteration times, and determining the camera internal parameter errors of the current iteration times according to the camera internal parameters determined by the current iteration times;
comparing the camera internal parameter error obtained by the current iteration times with the camera internal parameter error obtained by the last iteration times;
if the camera internal parameter error obtained by the current iteration number is better than the camera internal parameter error obtained by the last iteration number, reserving the running track of the target unmanned aerial vehicle determined by the current iteration number;
if the camera internal parameter error obtained by the last iteration number is better than the camera internal parameter error obtained by the current iteration number, reserving the driving track of the target unmanned aerial vehicle determined by the last iteration number;
judging whether the current iteration number reaches the preset iteration number or not;
if the current iteration number does not reach the preset iteration number, adding 1 to the current iteration number, updating the running track of the target unmanned aerial vehicle determined by the current iteration number, determining the running track of the target unmanned aerial vehicle reserved by the current iteration number as the running track of the target unmanned aerial vehicle determined by the last iteration number and the camera internal parameter error corresponding to the running track of the target unmanned aerial vehicle reserved by the current iteration number as the camera internal parameter error obtained by the last iteration number, returning the running track of the target unmanned aerial vehicle determined by the current iteration number, determining the camera internal parameter determined by the current iteration number, and determining the camera internal parameter error of the current iteration number according to the camera internal parameter determined by the current iteration number;
if the current iteration times reach the preset iteration times, ending the iteration optimization process, and determining the running track of the target unmanned aerial vehicle reserved by the current iteration times as a preset track.
Optionally, the determining process of the driving track of the target unmanned aerial vehicle is:
and determining the driving track of the target unmanned aerial vehicle according to the pose data of the target unmanned aerial vehicle by combining the view angle constraint and the front-rear distance constraint of the camera.
In a second aspect, the present invention provides an unmanned aerial vehicle-based camera internal parameter automatic calibration system, including:
the information acquisition module is used for determining the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle and the installation position of the camera in a world coordinate system;
the calibration scene arrangement module is used for arranging a calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera;
the pose data and photo data determining module is used for moving the target unmanned aerial vehicle according to a preset track in a calibration scene to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate; the preset track is determined according to the camera internal parameter error and the iterative optimization algorithm;
and the calibration module is used for calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera.
In a third aspect, the present invention provides an electronic device, including a memory and a processor, where the memory is configured to store a computer program, and the processor is configured to run the computer program to cause the electronic device to execute the method for automatically calibrating an internal reference of a camera based on an unmanned aerial vehicle according to the first aspect.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
aiming at the defects of the traditional camera internal parameter calibration, the invention provides an unmanned aerial vehicle-based camera internal parameter automatic calibration method, a system and electronic equipment. According to the invention, the automatic calibration of the camera internal parameters is realized by using the mode that the unmanned aerial vehicle carries the calibration plate, so that the calibration efficiency is improved, the error is reduced, and the manpower resource is saved. Simultaneously, iterative optimization of the unmanned plane planning route is introduced. By continuously optimizing the path planning of the unmanned aerial vehicle, the calibration plate is in the optimal pose for improving the calibration precision and the calibration efficiency of the internal parameters of the camera, so that the calibration error can be reduced to the greatest extent, the accuracy of the internal parameters of the camera is improved, and finally, the internal parameters of the camera are calibrated in a large batch with high precision and high efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a diagram of an automatic calibration method for camera internal parameters based on an unmanned aerial vehicle according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a calibration scenario provided in an embodiment of the present invention;
FIG. 3 is a schematic diagram of an internal reference calibration data acquisition process according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a calculation flow of camera internal parameter calibration data according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an optimization flow of a driving track of an unmanned aerial vehicle according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Example 1
When the sensing system based on the laser radar works, the focal length, the principal point position, distortion parameters and the like of the camera are needed to be used as internal parameters (camera internal reference matrix: physical lengths dx and dy of one pixel in the focal length f, x and y directions on a camera photosensitive plate, and a coordinate u of the center of the camera photosensitive plate under a pixel coordinate system 0 、v 0 The method comprises the steps of carrying out a first treatment on the surface of the Camera distortion parameters:radial distortion parameters k1, k2, k3, tangential distortion parameters p1, p 2). The process of adopting a Zhang Zhengyou calibration method and a Zhang Zhengyou calibration method is that firstly, a template is printed and attached to a plane, then a plurality of template images are shot from different angles, characteristic points in the images are detected, then the camera internal parameters and external parameters under ideal distortion-free conditions are solved, the accuracy is improved by maximum likelihood estimation, then the actual radial distortion coefficients are obtained by a least square method, the internal parameters, the external parameters and the distortion coefficients are synthesized, the maximum likelihood method is used for optimizing estimation, and the estimation accuracy is improved, and finally the camera internal parameter matrix and the camera distortion parameters are obtained. However, in the calibration process, manual movement and rotation of the calibration plate are required, and particularly when the accuracy requirement for the internal reference calibration of the camera is very high, the internal reference calibration accuracy can be improved by adopting a large-size calibration plate, at the moment, time and labor are wasted when the calibration plate is moved by adopting manpower, and time and effort are consumed when the calibration plate is manually placed and moved when a large number of cameras calibrate the internal reference.
Aiming at the problems, the embodiment provides an automatic calibration method for camera internal parameters based on an unmanned aerial vehicle, which comprises the following steps of.
Step 100: and determining the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle and the installation position of the camera in a world coordinate system.
In this embodiment, firstly, the size of the checkerboard calibration plate, for example, the size of each checkerboard, the specification of the corner points of the checkerboard, secondly, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, for example, rotation change and translation transformation, and finally, the installation position of the camera in the world coordinate system need to be confirmed.
Step 200: and arranging the calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera.
In this embodiment, as shown in fig. 2, the unmanned aerial vehicle (i.e., the target unmanned aerial vehicle) with the checkerboard calibration fixed is placed in the range of 0.5 m-1.5 m in front of the fixed camera (depending on the size of the checkerboard calibration plate selected and the camera field of view size used). The checkerboard calibration plate is required to be flat in surface, free of bending and non-reflective in surface, clear and complete imaging of the checkerboard on the checkerboard calibration plate in a camera is guaranteed, the checkerboard calibration plate is required to be rigidly connected with the unmanned aerial vehicle, and relative displacement and relative rotation cannot be generated in the motion process.
Step 300: in a calibration scene, a target unmanned aerial vehicle moves according to a preset track to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate.
In this embodiment, after the calibration scenario is arranged, raw data acquisition is performed, as shown in fig. 3. The target unmanned aerial vehicle moves in a camera fov range in a low-speed (less than 1 m/s) mode according to a preset track to acquire target unmanned aerial vehicle pose data required by calibration and picture data of a checkerboard calibration plate shot by the camera, in the process, enough checkerboard corner points are detected in x, y and z directions by a calibration program, a relatively horizontal rectangular network is required to be formed by the checkerboard corner points, and new movement track re-acquisition data is required to be re-planned under the condition that the error is increased due to the fact that too many pictures are shot at the same position of the camera.
Step 400: and calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera.
In this embodiment, as shown in fig. 4, step 400 specifically includes:
the first step: and determining coordinates of the checkerboard corner points in a pixel coordinate system and coordinates in a world coordinate system according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, pose data of the target unmanned aerial vehicle and photo data of the checkerboard calibration plate shot by the camera, wherein the detailed process is as follows.
1) And (3) synchronizing data, namely synchronizing the photo data of the checkerboard calibration plate shot by the camera with the pose data of the unmanned aerial vehicle in time.
2) And determining coordinates (u, v) of the corner points of the checkerboard in a pixel coordinate system according to the photo data after time synchronization, and determining coordinates (X, Y, Z) of the coordinates of each corner point in the checkerboard calibration plate in a world coordinate system according to the pose data after time synchronization and the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle.
Second, determining a rotation matrix and a translation vector (R, T) of the target unmanned aerial vehicle relative to the camera according to the installation position of the camera in the world coordinate system and pose data of the target unmanned aerial vehicle, wherein R represents the rotation matrix and T represents the translation vector
Thirdly, calculating an internal reference matrix A and distortion parameters of the camera according to a rotation matrix and a translation vector of the target unmanned aerial vehicle relative to the camera, a coordinate relation of the checkerboard corner points in a pixel coordinate system and a world coordinate system, and a coordinate of the checkerboard corner points in the pixel coordinate system and a coordinate of the checkerboard corner points in the world coordinate system.
The single point undistorted camera imaging model is as follows:
in the above formula, (X, Y, Z) is the physical coordinates of a point in the world coordinate system, and (u, v) is the pixel coordinates in the pixel coordinate system corresponding to the point, and Z is the scale factor.
The reference matrix called camera depends on the internal parameters of the camera, where f is the image distance, dx, dy represents the physical length of a pixel on the camera plate in the x, y direction (i.e. how many millimeters a pixel is on the plate), u 0 ，v 0 Respectively representing the coordinates of the center of the camera photosensitive plate in a pixel coordinate system, and theta represents the angle between the transverse side and the longitudinal side of the photosensitive plate, and will be +.>Called asAn external matrix of cameras.
When the world coordinate system is fixed on the checkerboard, the physical coordinate Z=0 of any point on the checkerboard, the internal reference matrix is marked as A, R1 and R2 are the first two columns of the rotation matrix R, so that the original single-point undistorted imaging model can be written as follows:
for different pictures, the internal reference matrix A is a fixed value, for the same picture, the internal reference matrix A and the external reference matrices (R1, R2 and T) are fixed values, and for single points on the same picture, the internal reference matrix A, the external reference matrices (R1, R2 and T) and the scale factor Z are fixed values. Let A (R1, R2, T) be the matrix H, H is the product of the internal and external reference matrices, and three columns of matrix H are (H1, H2, H3), then there are:
from the above formula:
in the above formula, X, Y may be output by the unmanned aerial vehicle, and u, v may be output by the camera, so that a matrix H may be obtained, and matrix h=a (R1, R2, T), wherein R1, R2, T output related data by the unmanned aerial vehicle, so that a camera internal reference matrix may be obtained. The number of pictures of the checkerboard calibration plate is generally 15 to 20, and the optimal internal reference matrix A is generally obtained by fitting through a least square method.
The radial distortion parameters k1, k2, k3 and tangential distortion parameters p1, p2 are calculated as follows:
the tangential distortion formula is as follows:
wherein (x) 0 ,y 0 ) The ideal undistorted normalized image coordinate can be obtained by outputting the coordinate of the calibration plate by the target unmanned aerial vehicle and then obtaining the normalized image coordinate after distortion by the original single-point undistorted imaging model, wherein (x, y) is the normalized image coordinate after distortion and can be directly obtained by outputting the coordinate of the angular point by the camera, namely (x) 0 ,y 0 ) (x, y) is known and r2=x2+y2, so that radial distortion parameters k1, k2, k3 and tangential distortion parameters p1, p2 can be obtained.
Fourth, through the second and third steps, camera internal parameters (camera internal parameter matrix: physical lengths dx, dy of one pixel in the focal length f, x, y direction on the camera photosheet, coordinates u of the center of the camera photosheet in the pixel coordinate system) 0 、v 0 The method comprises the steps of carrying out a first treatment on the surface of the Camera distortion parameters: radial distortion parameters k1, k2, k3, tangential distortion parameters p1, p 2) have all been found and calibration is complete.
Because the pose of the checkerboard calibration plate has a large influence on the calibration result in the camera internal reference calibration process, the optimal pose of the checkerboard calibration plate is optimized by iteratively optimizing the unmanned plane planning path, so that the effects of low error and high efficiency are achieved for mass camera internal reference calibration.
As shown in fig. 5, the preset number of iterations is first set to N. In each iteration, the target unmanned aerial vehicle performs a camera internal parameter calibration process according to the planned track, and performs camera internal parameter calibration error analysis.
The method comprises the following steps: acquiring a running track of a target unmanned aerial vehicle determined by the current iteration times; determining the camera internal parameters determined by the current iteration times according to the driving track of the target unmanned aerial vehicle determined by the current iteration times, and determining the camera internal parameter errors of the current iteration times according to the camera internal parameters determined by the current iteration times; comparing the camera internal parameter error obtained by the current iteration times with the camera internal parameter error obtained by the last iteration times; if the camera internal parameter error obtained by the current iteration number is better than the camera internal parameter error obtained by the last iteration number, reserving the running track of the target unmanned aerial vehicle determined by the current iteration number; if the camera internal parameter error obtained by the last iteration number is better than the camera internal parameter error obtained by the current iteration number, reserving the driving track of the target unmanned aerial vehicle determined by the last iteration number; judging whether the current iteration number reaches the preset iteration number or not; if the current iteration number does not reach the preset iteration number, adding 1 to the current iteration number, updating the running track of the target unmanned aerial vehicle determined by the current iteration number, determining the running track of the target unmanned aerial vehicle reserved by the current iteration number as the running track of the target unmanned aerial vehicle determined by the last iteration number and the camera internal parameter error corresponding to the running track of the target unmanned aerial vehicle reserved by the current iteration number as the camera internal parameter error obtained by the last iteration number, returning the running track of the target unmanned aerial vehicle determined by the current iteration number, determining the camera internal parameter determined by the current iteration number, and determining the camera internal parameter error of the current iteration number according to the camera internal parameter determined by the current iteration number; if the current iteration times reach the preset iteration times, ending the iteration optimization process, and determining the running track of the target unmanned aerial vehicle reserved by the current iteration times as a preset track.
By setting the preset iteration times and a selection mechanism of the optimized track, the embodiment can automatically select a better target unmanned aerial vehicle track in each iteration so as to furthest improve the calibration error of the internal parameters of the camera. By continuously executing the iterative optimization process, the accuracy and precision of the camera internal parameter calibration can be gradually improved. Therefore, the invention has higher efficiency and accuracy in the internal parameter calibration task of a large number of cameras, and can reduce the labor cost.
In order to realize constrained trajectory planning, so as to ensure that the checkerboard calibration plate cannot fly out of the field of view of the camera and meet the requirements of front and back distances, the method and the device for determining the unmanned aerial vehicle trajectory of the target unmanned aerial vehicle determine the driving trajectory of the target unmanned aerial vehicle according to pose data of the unmanned aerial vehicle and by combining the field angle FOV constraint of the camera and the front and back distance constraint.
The aim of the trajectory planning is to ensure that the target unmanned aerial vehicle does not have repeated pose in the flight process and meets the constraint condition.
Firstly, the space around the target unmanned aerial vehicle is divided into 27 small squares, wherein the small squares comprise squares where the target unmanned aerial vehicle is located. Then, among 26 blocks around the target unmanned aerial vehicle, numbers between 0 and 1 are randomly generated, and the target block for the next flight of the target unmanned aerial vehicle is determined according to the sizes of the random numbers. In this way, the track planning can have a certain randomness, and the diversity of the tracks is increased. Further, taking into account the variations in pitch angle, roll angle and yaw angle, numerals will be used to represent the pose of the target drone in these three orientations. For example, the pitch angle is 45 degrees, the roll angle is 20 degrees, and the yaw angle is 34 degrees, as indicated by numeral 452034. Through random generation in each azimuth, the gesture of the target unmanned aerial vehicle can have certain change in a certain range, and the track diversity is further increased. By combining the view angle constraint and the front-back distance constraint of the camera, the flight area of the unmanned aerial vehicle is limited in a specific area, so that the checkerboard calibration plate is always kept in the view range of the camera, the occurrence of the condition of too close or too far is avoided, and the recognition and the accuracy of angular points are ensured.
Example two
In order to execute the corresponding method of the above embodiment to achieve the corresponding functions and technical effects, an automatic calibration system for camera internal parameters based on an unmanned aerial vehicle is provided below.
The embodiment provides an automatic calibration system for camera internal parameters based on an unmanned aerial vehicle, which comprises:
and the information acquisition module is used for determining the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle and the installation position of the camera in the world coordinate system.
And the calibration scene arrangement module is used for arranging the calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera.
The pose data and photo data determining module is used for moving the target unmanned aerial vehicle according to a preset track in a calibration scene to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate; the preset track is determined according to the camera internal parameter error and the iterative optimization algorithm.
And the calibration module is used for calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera.
Example III
The embodiment of the invention provides electronic equipment which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor runs the computer program to enable the electronic equipment to execute the camera internal parameter automatic calibration method based on the unmanned aerial vehicle.
Alternatively, the electronic device may be a server.
In addition, the embodiment of the invention also provides a computer readable storage medium, which stores a computer program, and the computer program realizes the camera internal parameter automatic calibration method based on the unmanned aerial vehicle of the first embodiment when being executed by a processor.
Compared with the prior art, the invention has the innovation that:
1. the invention establishes an automatic calibration method for camera internal parameters by adopting a method that an unmanned aerial vehicle carries a calibration plate. 2. According to the invention, through the data output by the unmanned aerial vehicle and the relative positions of the calibration plate and the unmanned aerial vehicle, the conversion flow and model of the corner coordinates on the calibration plate from the world coordinate system to the pixel coordinate system are realized. 3. The invention adopts an iterative optimization unmanned aerial vehicle track method to optimize the camera internal parameter calibration precision. 4. The unmanned aerial vehicle is simple in recovery and arrangement scene, unmanned in process, labor-saving and efficiency-improving.
Compared with the prior art, the invention has the advantages that:
1. the traditional camera internal parameter calibration method needs to be manually participated, and the calibration process is tedious and takes a long time. The automatic calibration method and the automatic calibration device realize an automatic calibration process by using the way that the unmanned aerial vehicle carries the calibration plate, do not need to participate manually, save a great deal of manpower resources, and greatly shorten the calibration time, thereby improving the calibration efficiency.
2. According to the invention, the position and the posture of the calibration plate in the camera are calculated by utilizing the posture and the position information output by the unmanned aerial vehicle, so that the accuracy of the corner point is ensured. Through iterative optimization unmanned aerial vehicle planning route, can select the optimum track, make the demarcation board be in the optimum position gesture to the accuracy of demarcation has further been improved.
3. According to the invention, the unmanned aerial vehicle is used for carrying the calibration plate for automatic calibration, and manual operation is not required. Through iterative optimization unmanned aerial vehicle planning route, realize the automation process of the calibration of a large amount of camera internal parameters. The automatic characteristic saves human resources and improves the calibration efficiency and the batch processing capacity.
4. Compared with the traditional camera internal reference calibration method, the method adopts the unmanned aerial vehicle to recover and arrange the calibration plates, and simplifies the operation flow of calibration. Unmanned aerial vehicle need not artificial participation in carrying, arranging and retrieving the in-process of demarcation board, has reduced human cost and time consumption. The simplified operation flow greatly improves the calibration efficiency, and is particularly suitable for the large-scale camera internal parameter calibration task.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (8)
1. An unmanned aerial vehicle-based camera internal parameter automatic calibration method is characterized by comprising the following steps:
determining the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle and the installation position of the camera in a world coordinate system;
arranging a calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera;
in a calibration scene, a target unmanned aerial vehicle moves according to a preset track to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate; the preset track is determined according to the camera internal parameter error and the iterative optimization algorithm;
and calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera.
2. The automatic calibration method for camera internal parameters based on the unmanned aerial vehicle according to claim 1, wherein the calibration scene is arranged according to the size information of the checkerboard calibration plate and the visual field information of the camera, and specifically comprises the following steps:
determining a target unmanned aerial vehicle placement range according to the size information of the checkerboard calibration plate and the visual field information of the camera; wherein, the checkerboard calibration board surface is level and smooth and not reflective, and checkerboard calibration board and unmanned aerial vehicle rigid connection.
3. The automatic calibration method for camera internal parameters based on the unmanned aerial vehicle according to claim 1, wherein the calibration of the camera internal parameters according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in a world coordinate system, pose data of the target unmanned aerial vehicle and photo data of the checkerboard calibration plate shot by the camera specifically comprises:
determining coordinates of the checkerboard corner points in a pixel coordinate system and coordinates in a world coordinate system according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, pose data of the target unmanned aerial vehicle and photo data of the checkerboard calibration plate shot by a camera;
according to the installation position of the camera in the world coordinate system and the pose data of the target unmanned aerial vehicle, determining a rotation matrix and a translation vector of the target unmanned aerial vehicle relative to the camera;
and calculating an internal reference matrix and distortion parameters of the camera according to the rotation matrix and translation vector of the target unmanned aerial vehicle relative to the camera, the coordinate relation of the checkerboard corner points in the pixel coordinate system and the world coordinate system, and the coordinates of the checkerboard corner points in the pixel coordinate system and the coordinates in the world coordinate system.
4. The automatic calibration method for camera internal parameters based on unmanned aerial vehicle according to claim 3, wherein the determining of the coordinates of the checkerboard corner in the pixel coordinate system and the coordinates in the world coordinate system according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate taken by the camera specifically comprises:
the photo data of the checkerboard calibration plate shot by the camera and the pose data of the unmanned aerial vehicle are subjected to time synchronization;
determining coordinates of the checkerboard corner points in a pixel coordinate system according to the photo data after time synchronization;
and determining coordinates of each angular point coordinate in the checkerboard calibration plate in a world coordinate system according to the pose data after time synchronization and the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle.
5. The automatic calibration method for camera internal parameters based on the unmanned aerial vehicle according to claim 1, wherein the preset track determining process is as follows:
acquiring a running track of a target unmanned aerial vehicle determined by the current iteration times;
determining the camera internal parameters determined by the current iteration times according to the driving track of the target unmanned aerial vehicle determined by the current iteration times, and determining the camera internal parameter errors of the current iteration times according to the camera internal parameters determined by the current iteration times;
comparing the camera internal parameter error obtained by the current iteration times with the camera internal parameter error obtained by the last iteration times;
if the camera internal parameter error obtained by the current iteration number is better than the camera internal parameter error obtained by the last iteration number, reserving the running track of the target unmanned aerial vehicle determined by the current iteration number;
if the camera internal parameter error obtained by the last iteration number is better than the camera internal parameter error obtained by the current iteration number, reserving the driving track of the target unmanned aerial vehicle determined by the last iteration number;
judging whether the current iteration number reaches the preset iteration number or not;
if the current iteration number does not reach the preset iteration number, adding 1 to the current iteration number, updating the running track of the target unmanned aerial vehicle determined by the current iteration number, determining the running track of the target unmanned aerial vehicle reserved by the current iteration number as the running track of the target unmanned aerial vehicle determined by the last iteration number and the camera internal parameter error corresponding to the running track of the target unmanned aerial vehicle reserved by the current iteration number as the camera internal parameter error obtained by the last iteration number, returning the running track of the target unmanned aerial vehicle determined by the current iteration number, determining the camera internal parameter determined by the current iteration number, and determining the camera internal parameter error of the current iteration number according to the camera internal parameter determined by the current iteration number;
if the current iteration times reach the preset iteration times, ending the iteration optimization process, and determining the running track of the target unmanned aerial vehicle reserved by the current iteration times as a preset track.
6. The automatic calibration method for camera internal parameters based on the unmanned aerial vehicle according to claim 5, wherein the determination process of the driving track of the target unmanned aerial vehicle is as follows:
and determining the driving track of the target unmanned aerial vehicle according to the pose data of the target unmanned aerial vehicle by combining the view angle constraint and the front-rear distance constraint of the camera.
7. An unmanned aerial vehicle-based camera internal parameter automatic calibration system is characterized by comprising:
the information acquisition module is used for determining the size information of the checkerboard calibration plate, the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle and the installation position of the camera in a world coordinate system;
the calibration scene arrangement module is used for arranging a calibration scene according to the size information of the checkerboard calibration plate and the visual field information of the camera;
the pose data and photo data determining module is used for moving the target unmanned aerial vehicle according to a preset track in a calibration scene to obtain pose data of the target unmanned aerial vehicle required by calibration and photo data of a checkerboard calibration plate shot by a camera; the target unmanned aerial vehicle is an unmanned aerial vehicle with a fixed checkerboard calibration plate; the preset track is determined according to the camera internal parameter error and the iterative optimization algorithm;
and the calibration module is used for calibrating parameters in the camera according to the installation position of the checkerboard calibration plate relative to the unmanned aerial vehicle, the installation position of the camera in the world coordinate system, the pose data of the target unmanned aerial vehicle and the photo data of the checkerboard calibration plate shot by the camera.
8. An electronic device comprising a memory and a processor, the memory for storing a computer program, the processor running the computer program to cause the electronic device to perform an unmanned aerial vehicle-based camera intrinsic parameter automatic calibration method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311265715.5A CN117252933A (en) | 2023-09-27 | 2023-09-27 | Unmanned aerial vehicle-based camera internal parameter automatic calibration method, system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311265715.5A CN117252933A (en) | 2023-09-27 | 2023-09-27 | Unmanned aerial vehicle-based camera internal parameter automatic calibration method, system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117252933A true CN117252933A (en) | 2023-12-19 |
Family
ID=89127562
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311265715.5A Pending CN117252933A (en) | 2023-09-27 | 2023-09-27 | Unmanned aerial vehicle-based camera internal parameter automatic calibration method, system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117252933A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117519256A (en) * | 2023-12-25 | 2024-02-06 | 南京理工大学 | Monocular track reconstruction method for unmanned aerial vehicle platform |
-
2023
- 2023-09-27 CN CN202311265715.5A patent/CN117252933A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117519256A (en) * | 2023-12-25 | 2024-02-06 | 南京理工大学 | Monocular track reconstruction method for unmanned aerial vehicle platform |
| CN117519256B (en) * | 2023-12-25 | 2024-06-07 | 南京理工大学 | Monocular track reconstruction method for unmanned aerial vehicle platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109767476B (en) | Automatic focusing binocular camera calibration and depth calculation method | |
| CN107784672B (en) | Method and device for acquiring external parameters of vehicle-mounted camera | |
| CN107993266A (en) | Light-field camera intrinsic parameter caliberating device and method | |
| CN112308925A (en) | Binocular calibration method and device of wearable device and storage medium | |
| CN117252933A (en) | Unmanned aerial vehicle-based camera internal parameter automatic calibration method, system and electronic equipment | |
| CN111263142A (en) | Method, device, equipment and medium for testing optical anti-shake of camera module | |
| CN113793270A (en) | Aerial image geometric correction method based on unmanned aerial vehicle attitude information | |
| CN111123962A (en) | Rotor unmanned aerial vehicle repositioning photographing method for power tower inspection | |
| CN113920206B (en) | Calibration method of perspective tilt-shift camera | |
| US20220358679A1 (en) | Parameter Calibration Method and Apparatus | |
| CN115861445B (en) | Hand-eye calibration method based on three-dimensional point cloud of calibration plate | |
| CN109472778B (en) | Appearance detection method for towering structure based on unmanned aerial vehicle | |
| CN111461963A (en) | Fisheye image splicing method and device | |
| WO2020181409A1 (en) | Capture device parameter calibration method, apparatus, and storage medium | |
| CN111445537A (en) | Calibration method and system of camera | |
| CN115187612A (en) | Plane area measuring method, device and system based on machine vision | |
| CN115375775A (en) | Underwater camera calibration method and device based on forward projection refraction model | |
| CN114659523A (en) | Large-range high-precision attitude measurement method and device | |
| CN112182967B (en) | Automatic photovoltaic module modeling method based on thermal imaging instrument | |
| CN112465918B (en) | Microscopic vision calibration method based on Tsai calibration | |
| CN117190875A (en) | Bridge tower displacement measuring device and method based on computer intelligent vision | |
| CN109682312B (en) | Method and device for measuring length based on camera | |
| CN109389645B (en) | Camera self-calibration method and system, camera, robot and cloud server | |
| CN115239816A (en) | Camera calibration method, system, electronic device and storage medium | |
| CN115456870A (en) | Multi-image splicing method based on external parameter estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |