CN113781579A - Geometric calibration method for panoramic infrared camera - Google Patents

Geometric calibration method for panoramic infrared camera Download PDF

Info

Publication number
CN113781579A
CN113781579A CN202111060670.9A CN202111060670A CN113781579A CN 113781579 A CN113781579 A CN 113781579A CN 202111060670 A CN202111060670 A CN 202111060670A CN 113781579 A CN113781579 A CN 113781579A
Authority
CN
China
Prior art keywords
infrared camera
panoramic
infrared
coordinate system
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111060670.9A
Other languages
Chinese (zh)
Other versions
CN113781579B (en
Inventor
邓华健
王昊
金仲和
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202111060670.9A priority Critical patent/CN113781579B/en
Publication of CN113781579A publication Critical patent/CN113781579A/en
Application granted granted Critical
Publication of CN113781579B publication Critical patent/CN113781579B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image

Abstract

The invention discloses a geometric calibration method of a panoramic infrared camera, which comprises the steps of fixing one or more infrared targets in front of a turntable, and controlling the rotation of the panoramic infrared camera by using a high-precision turntable to realize target image acquisition at different angles; then, extracting the center of the infrared target pattern by adopting a gray scale centroid positioning algorithm to realize control point extraction; the installation position of the camera, the position of the installation matrix and the position of the target are used as external parameters, the principal point and the distortion coefficient of the camera are used as internal parameters, a complete model of the infrared target in the camera in the rotating process of the turntable is established, the precise estimation of the parameters is completed by adopting a two-step estimation method, and the precise calibration of the infrared omnidirectional camera is realized. The panoramic infrared camera calibration method is insensitive to the edge blur of target patterns universally existing in the infrared images, does not need initial information of external parameters, is not influenced by blind areas possibly existing in the camera, completely covers the whole target surface, can well complete the panoramic infrared camera calibration task, and achieves higher precision.

Description

Geometric calibration method for panoramic infrared camera
Technical Field
The invention relates to the technical field of geometric calibration of infrared cameras, in particular to a geometric calibration method of a panoramic infrared camera.
Background
The panoramic infrared camera can provide an ultra-large infrared view, can provide information about radiation and temperature of a target, and can also provide geometric shape information of a scene, so that the panoramic infrared camera plays an increasingly important role in the fields of security monitoring, target detection, tracking and identification. However, the ubiquitous severe optical distortion of the panoramic camera and the extremely low signal-to-noise ratio of the infrared camera greatly limit the high-precision application of the panoramic infrared camera, and particularly in the fields of machine vision, target positioning and the like which need high-precision angle detection, the geometric calibration precision will restrict the application of a sub-pixel image processing algorithm in an infrared image and limit the measurement precision of devices, so that the geometric calibration of the panoramic infrared camera is an indispensable and extremely important link.
The most popular infrared camera calibration scheme at present is the infrared plane calibration plate scheme. Unlike the plane calibration board used by the visible light camera, the infrared plane calibration board needs to provide a camera corresponding to the infrared characteristic detection wavelength, and the characteristics need to have a high-contrast edge or center symmetry point positioning high-precision control. Many researchers have worked to find a suitable flat calibration plate solution, such as using a geometric mask and a hot background, using a flat plate with illuminated bulbs, using a checkerboard of substrates and inks with different emissivity, etc. The use of an infrared plane calibration board to calibrate a panoramic infrared camera is generally a good choice, but also faces some challenges. The main challenge is the coverage of the oversized field of view, and in order to ensure that the calibration parameters are more consistent with the camera, the control points need to cover the full field of view of the camera as much as possible, and at the same time, it must be ensured that all the control points of the calibration plate appear on the image each time. However, in the case of a panoramic camera having a blind field of view, such as a Panoramic Annular Lens (PAL) camera, which has a circular blind area of ± 30 ° at the center thereof, the resolution is low, so that the placement of a plane calibration plate is extremely difficult. Other challenges are defocus and edge blur. Due to the limitations of low resolution of the infrared camera and low contrast of the target, the infrared image often has the problem of edge blurring, which needs an additional edge thinning algorithm to overcome. To cope with different tasks, the camera needs to adopt different focusing strategies, ranging from a few meters to hundreds of kilometers. In order to ensure image definition, the infrared plane calibration plate should be placed at a proper imaging position, but in most cases, the infrared plane calibration plate is almost impossible to realize, so that only edge blurring and reduction of control point extraction precision caused by defocusing in the imaging process can be accepted.
The use of a turntable to control the rotation of the camera is a common calibration solution for high precision cameras. The star sensor, as the most accurate attitude measurement device on the satellite, can be regarded as a narrow-view-field visible light camera, and the calibration task of the star sensor is widely researched. Wherein, Wen Xin nations and the like provide a star sensor calibration method based on internal and external parameter integrated modeling. The method utilizes the combined modeling of the alignment error of the star simulator, the installation error of the camera and the imaging process of the camera, so that the method does not depend on the installation alignment operation, the precision is effectively improved, and the operation is simplified. However, for panoramic infrared cameras with an oversized field of view, the application of the above method means that an infrared collimator of sufficient diameter is required, and the camera is required to be placed in the center of the turntable, with no obstruction to the full field of view, which is certainly difficult and extremely costly.
Disclosure of Invention
The invention provides a geometric calibration method of a panoramic infrared camera, which can effectively improve the geometric calibration precision of the panoramic infrared camera and provide a solid foundation for applying the panoramic infrared camera to the field of machine vision and completing tasks such as high-precision target positioning, angle detection and the like.
The purpose of the invention is realized by the following technical scheme:
a geometrical calibration method for a panoramic infrared camera comprises the following steps:
first, the panoramic infrared camera to which the present invention is directed responds to signals in the infrared band, requiring a suitable infrared target. Selecting a corresponding infrared target according to the response characteristic of the camera; and fixing one or more selected infrared targets in an area without shielding in front of the turntable, so that the infrared targets are opposite to the infrared camera.
And then, covering the image of the control point on the whole target surface of the infrared camera, controlling a shaft of the rotary table to rotate at fixed intervals, acquiring the rotation angle of the rotary table at each rotation position of the rotary table, photographing the infrared target by using the infrared camera, and extracting the control point according to the image of the infrared target. And controlling a shaft of the rotary table to rotate at fixed intervals, so that the image formed by the infrared target completely covers the full image surface of the panoramic infrared camera.
And preprocessing the infrared image shot by the panoramic infrared camera. For an image shot by the panoramic infrared camera, calculating 3.5 times of the global standard deviation of the image pixel gray level and adding the global average value as a threshold value to obtain a gray threshold value image:
Figure BDA0003256394220000031
wherein (u, v) represents coordinates of the pixel, F (u, v) represents an original gray level of the pixel, and F (u, v) represents a threshold gray level of the pixel; carrying out four-connected domain processing on the gray threshold value image, and extracting the region of the image formed by the infrared target; and (3) finishing control point extraction by adopting a centroid positioning algorithm:
Figure BDA0003256394220000032
wherein m and n are the total number of pixels in the u-direction and v-direction, respectively, (u)c,vc) Coordinates representing control points; for the combination of a plurality of infrared targets, identification and matching are required according to the relative position characteristics among the infrared targets.
Then, performing internal reference and external reference joint modeling on the complete imaging process of the infrared target in the panoramic infrared camera calibration process: establishing an external reference model for describing the position relation of the infrared target under a panoramic infrared camera coordinate system in the rotating process of the turntable by taking the mounting position of the panoramic infrared camera on the turntable, the mounting matrix and the position of the infrared target relative to the turntable as external references; and respectively establishing a rough imaging model according to the design principle of the panoramic infrared camera and a fine imaging model according to polynomial fitting and distortion parameter correction by taking principal points and distortion coefficients of the panoramic infrared camera as internal parameters, thereby describing the projection process of the infrared target to the image sensor under the coordinate system of the panoramic infrared camera.
The external participation modeling process comprises the following steps:
the rotary table is a double-shaft rotary table; establishing a double-shaft turntable rotating coordinate system P by taking the rotating center O of the double-shaft turntable as an original point and taking X, Y and Z axes of the double-shaft turntable; when the double-shaft turntable is located at an initial rotating position, defining a rotating coordinate system P at the moment as an inertial coordinate system B; with the optical centre O of the cameracLine of the image sensor as XcAxis, column of image sensor as YcAxis, ZcPerpendicular to XcAxis and YcThe axis is used for establishing a coordinate system C of the panoramic infrared camera;
on the rotation angle of the ith group of rotary tables, for the control point corresponding to the jth infrared target obtained by collection, the coordinate of the control point under the coordinate system C of the panoramic infrared cameraCXDij＝[XDijC,YDijC,ZDijC]TThe following relationships are present:
Figure BDA0003256394220000041
whereinBXDj＝[XDjB,YDjB,ZDjB]TRepresenting the position of the jth infrared target under an inertial coordinate system B;
Figure BDA0003256394220000042
representing a rotation matrix from a biaxial turntable fixed coordinate system P to a panoramic infrared camera coordinate system C,
Figure BDA0003256394220000043
a rotation matrix representing a fixed coordinate system P from the inertial coordinate system B to the double-shaft turntable;
Figure BDA0003256394220000044
describes the installation relationship of a panoramic infrared camera on a double-shaft turntable, adopts three groups of independent parameters alpha and beta,
Figure BDA0003256394220000045
to be described, the method has the advantages that,
Figure BDA0003256394220000046
the expression of (a) is:
Figure BDA0003256394220000047
Figure BDA0003256394220000048
describes the rotation process of a double-shaft turntable, and two groups of independent parameters theta are adopted according to the actually adopted double-shaft turntablexi,θziTo be described, the method has the advantages that,
Figure BDA0003256394220000049
the expression of (a) is:
Figure BDA0003256394220000051
the internal reference modeling process comprises the following steps:
image coordinate system with imaging center O on image sensorsThe pixel coordinate system takes the upper left corner of the image sensor as an origin, and the two plane coordinate systems both take the rows and the columns of the image sensor as an X axis and a Y axis;
according to the imaging principle of the panoramic infrared camera, the coordinates of the infrared target on the coordinate system C of the panoramic infrared camera areCXD＝[XDC,YDC,ZDC]TAnd projected on an image coordinate system as an image point p ' ═ u ', v ']TSpecifically, it is represented as:
λ·[XDC YDC ZDC]T＝g(u′,v′) (25)
where g denotes the projection imaging relationship and λ denotes the scaling factor.
In order to reduce the difficulty of parameter calculation, a coarse model and a fine model need to be established so as to realize initial parameter estimation and subsequent parameter optimization estimation.
a) Coarse model
Establishing a rough model according to an imaging principle designed by a panoramic infrared camera, wherein the imaging principle of the panoramic infrared camera comprises equidistant projection, orthogonal projection, equal solid angle projection and stereoscopic projection; the construction methods of rough models of the panoramic infrared cameras corresponding to the four projection principles are similar;
the construction method of the rough model of the panoramic infrared camera based on the equidistant projection imaging principle comprises the following steps:
the principle of isometric projection imaging is expressed as follows:
Figure BDA0003256394220000052
where f is the camera focal length, θ is the field angle corresponding to the target, and ρ represents the image height. Considering the distortion in the lens design process and the processing process, which generally does not deviate too much from the design imaging principle, the rough model is:
λ·[XDC YDC ZDC]T＝[u′sinθ v′sinθ ρcosθ]T (27)
in addition, the shift of the principal point is also considered, and the point p' in the image coordinate system is converted into p ═ u, v in the pixel coordinate system]TIt can be described as:
[u v]T＝[u′ v′]T+[u0 v0]T (28)
wherein [ u ]0,v0]TRepresenting the imaging center O on the image sensorsLocation in the pixel coordinate system.
b) Fine model
Here, a polynomial is used to perform an estimation fit on the projection imaging function g:
λ·[XDC YDC ZDC]T＝[u′ v′ a0+a2ρ2+a3ρ3+a4ρ4]T (29)
wherein a is0,a2,a3,a4Representing the coefficients of each order. Considering also the imaging process of the image sensor, point p' in the image coordinate system is converted into p ═ u, v in the pixel coordinate system]TThe method comprises the following steps:
Figure BDA0003256394220000061
where k denotes the image sensor pixel aspect ratio and s denotes the tilt angle of the image sensor row and column.
And finally, finishing the internal parameter and external parameter joint estimation in the model by adopting a two-step estimation method.
Step 1: performing preliminary parameter estimation; using a rough model, in combination with equations (22), (26), (27) and (28):
Figure BDA0003256394220000062
wherein:
Pj＝(XDjB,YDjB,ZDjB) (32)
Figure BDA0003256394220000063
in the above formula, the first and second carbon atoms are,
Figure BDA0003256394220000071
a measured value representing a control point is shown,
Figure BDA0003256394220000072
estimated value representing control point, e1ij,e2ijRepresenting the estimated residual of the control point, Pr,PjRepresenting a matrix of unknown parameters, fx,fyA non-linear function describing the above relationship is represented.
From this, a non-linear least squares estimation problem can be proposed, with minimized reprojection errors as the optimization objective:
Figure BDA0003256394220000073
wherein K represents the number of the infrared targets, and L represents the number of the rotation angle groups of the double-shaft turntable. The Levenberg-Marquardt algorithm can be used to solve this problem. Wherein the ratio of alpha, beta,
Figure BDA0003256394220000074
neglecting installation errors to give an initial estimate u according to the correspondence between coordinate systems0,v0Adopting the center of a pixel coordinate system as initial estimation, adopting a lens design value as initial estimation, and giving X according to the field situationDjB,YDjB,ZDjB,XCB,YCBIs estimated.
Step 2: optimizing parameters; on the basis of step 1, a fine model is adopted, and the combination of the formula (22), the formula (29) and the formula (30) is as follows:
Figure BDA0003256394220000075
wherein:
Figure BDA0003256394220000076
wherein P iseFor a matrix of unknown parameters, hx,hyA non-linear function describing the above relationship is represented.
A similar non-linear least squares problem can be derived:
Figure BDA0003256394220000077
the Levenberg-Marquardt algorithm can also be used to solve this problem. Wherein the parameters obtained in step 1 will be selected as initial estimates, and 1 and 0 are selected as initial estimates of k and s. For f obtained, the following relationship is utilized:
ρ/tan(ρ/f)＝a0+a2ρ2+a3ρ3+a4ρ4 (38)
selecting proper rho in a group of image ranges as input, substituting the rho into an equation (38), and fitting a polynomial to obtain a group of proper a0,a2,a3,a4As an initial estimate.
Therefore, the high-precision estimation of the geometric calibration parameters of the panoramic infrared camera can be completed.
The invention principle of the invention is as follows:
according to the geometric calibration method of the panoramic infrared camera, the center of the infrared target pattern is determined according to the energy distribution of the target by adopting a centroid positioning algorithm, and because edge information is not adopted in the method, the method is insensitive to edge blurring of the target image, so that high-precision positioning of a control point can be realized. The panoramic infrared camera is installed on the high-precision rotary table, the camera is controlled to rotate at high precision, the full coverage of the ultra-large field of view of the panoramic infrared camera is achieved, and the influence of a possible blind area of the camera is avoided. By establishing a complete imaging model of the infrared target in the camera rotation process and adopting a two-step estimation method, the accurate robust estimation of the parameters is realized. The calibration scheme greatly reduces the cost required by the calibration of the infrared camera, and provides a solid foundation for the high-precision application of the panoramic infrared camera in the field of machine vision.
Compared with the prior art, the invention has the following beneficial effects:
1. according to the invention, the gray scale centroid positioning algorithm based on energy distribution is adopted, so that the high-precision positioning of the corresponding control point of the infrared target image is realized, and the influence of the out-of-focus and edge blurring problems possibly existing in the target shooting of the infrared panoramic camera can be avoided because the edge information can not be used.
2. The double-shaft turntable applied in the invention controls the camera to rotate at high precision, and can realize the full coverage of the view field of the infrared panoramic camera by matching with the infrared target, thereby ensuring the full view field high-precision calibration effect of the camera.
3. According to the invention, an internal reference and external reference combined model is established in the imaging process of the infrared panoramic camera, and the high-precision estimation of parameters is completed by applying a two-step estimation method, so that the method does not depend on specially designed and high-precision aligned experimental equipment, and the calibration experimental requirements and difficulty are reduced.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a diagram of an infrared target formed by three ceramic heating sheets;
FIG. 2 is a grayscale image of the infrared target of FIG. 1 taken in an infrared panoramic camera;
FIG. 3 is a three-dimensional schematic view of a panoramic infrared camera calibration apparatus;
fig. 4 is a schematic diagram of the combination of control points located in the images taken at each turntable angle during the calibration process of the panoramic infrared camera, where a pentagram is a control point corresponding to the target 1, a triangle is a control point corresponding to the target 2, and a dot is a control point corresponding to the target 3; note that the blank area at the center is a circular blind area with a positive and negative 30 degrees in the center of the camera, and the blank area at the edge is a gap caused by the mismatch between the camera lens view field and the detector target surface. The full detector target surface is 384x288 pixels.
Wherein 1 is a double-shaft rotating platform, 2 is a panoramic infrared camera, and 3 is an infrared target.
Detailed Description
The invention provides a geometric calibration method of a panoramic infrared camera, which comprises the following implementation steps:
step 1, collecting calibration data; the panoramic infrared camera is arranged on a double-shaft or three-shaft rotary table, one or more circular ceramic heating sheets are used as infrared targets (shown in a real object picture 1) and fixed in a basically non-sheltered area in front of the rotary table, so that the circular ceramic heating sheets are suspended and directly exposed in the air, and the circular surfaces of the ceramic heating sheets are opposite to the infrared camera. And (3) electrifying and heating the infrared ceramic plate by adopting a voltage source, and waiting for the current of the infrared ceramic plate to be basically unchanged to achieve thermal balance. The ceramic heating plate with low price is used as the infrared target, the infrared signal with ultrahigh contrast can be stably provided for a long time, and the infrared imaging device can be easily usedThe ground excludes the interference of the environment. Taking the biaxial rotation stage shown in fig. 3 as an example, and taking a Panoramic Annular Lens (PAL) camera with a 180-degree field angle as an example, the turntable is first adjusted so that the position imaged by the ceramic heating plate is at the center of the image, and the Z-axis angle θ of the turntable is recordedz0Angle theta of X axisx0And generating dense turntable rotation angles alpha, beta by taking the step length around the Z axis of the turntable as 10 degrees and taking the step length around the X axis of the turntable as 5 degrees:
Figure BDA0003256394220000101
and controlling the rotating table to rotate according to the rotating angle of the intensive rotating table by using a computer, recording the angle of the rotating table, and simultaneously collecting the images shot by the panoramic infrared camera by using the computer.
Step 2, firstly, calculating 3.5 times of the global standard deviation of the image shot by the panoramic infrared camera and adding the global average value as a threshold value to obtain a gray threshold value image:
Figure BDA0003256394220000102
wherein (u, v) represents coordinates of the pixel, F (u, v) represents an original gray level of the pixel, and F (u, v) represents a threshold gray level of the pixel; then, carrying out four-connected domain processing on the gray threshold value image, extracting the area of the image formed by the infrared target according to the area and the gray of the area, and eliminating the image without the infrared target; and finally, finishing control point extraction by adopting a centroid positioning algorithm:
Figure BDA0003256394220000103
wherein m and n are the total number of pixels in the u-direction and v-direction, respectively, (u)c,vc) Representing the coordinates of the control points.
For the combination of a plurality of infrared targets, identification and matching are required according to the relative position characteristics among the infrared targets. A simple method is provided as a reference, and a plurality of targets keep different distances relative to the center of the camera according to the rotation process of the rotary table and leave gaps; and calculating the distance from the extracted control points to the right center of the image and sequencing the control points so as to realize the matching between the targets.
Step 3, performing combined modeling on internal and external parameters in the panoramic infrared camera calibration process; establishing an external reference model for describing the position relation of the infrared target under a panoramic infrared camera coordinate system in the rotating process of the turntable by taking the mounting position of the panoramic infrared camera on the turntable, the mounting matrix and the position of the infrared target relative to the turntable as external references; respectively establishing a rough imaging model according to the design principle of the panoramic infrared camera and a fine imaging model according to polynomial fitting and distortion parameter correction by taking principal points and distortion coefficients of the panoramic infrared camera as internal parameters;
step 4, estimating parameters of a two-step method; and according to the internal parameter and external parameter combined model established in the prior art, the internal parameter and external parameter combined estimation in the model is completed by adopting a two-step estimation method.
1): performing preliminary parameter estimation; using a rough model, in combination with equations (22), (26), (27) and (28):
Figure BDA0003256394220000111
wherein:
Pj＝(XDjB,YDjB,ZDjB) (43)
Figure BDA0003256394220000112
in the above formula, the first and second carbon atoms are,
Figure BDA0003256394220000113
a measured value representing a control point is shown,
Figure BDA0003256394220000114
estimated value representing control point, e1ij,e2ijRepresenting the estimated residual of the control point,Pr,Pjrepresenting a matrix of unknown parameters, fx,fyRepresenting a non-linear function describing the above relationship;
thus, a non-linear least squares estimation problem is proposed, with minimized reprojection errors as the optimization objective:
Figure BDA0003256394220000115
wherein K represents the number of the infrared targets, and L represents the number of the rotation angle groups of the double-shaft turntable; solving the problem of nonlinear least square estimation by adopting a Levenberg-Marquardt algorithm; wherein the ratio of alpha, beta,
Figure BDA0003256394220000116
according to the corresponding relation between the panoramic infrared camera coordinate system C and the double-shaft turntable rotating coordinate system P, neglecting installation errors to give initial estimation u0,v0Taking the center of a pixel coordinate system as initial estimation, and taking a lens design value of the panoramic infrared camera as initial estimation;
2): optimizing parameters; on the basis of step 1), a fine model is adopted, and the combination of the formula (22), the formula (29) and the formula (30) is as follows:
Figure BDA0003256394220000121
wherein:
Figure BDA0003256394220000122
wherein P iseFor a matrix of unknown parameters, hx,hyRepresenting a non-linear function describing the above relationship;
a similar non-linear least squares problem is thus obtained:
Figure BDA0003256394220000123
the Levenberg-Marquardt algorithm is also adopted to solve the problem; wherein the parameters obtained in step 1 are selected as initial estimates, and 1 and 0 are selected as initial estimates of k and s; for f obtained, the following relationship is utilized:
ρ/tan(ρ/f)＝a0+a2ρ2+a3ρ3+a4ρ4 (49)
selecting rho in a group of panoramic infrared camera image ranges as input, substituting the rho into a formula (38), and obtaining a group of alpha through polynomial fitting0,a2,a3,a4As an initial estimate; therefore, the high-precision estimation of the geometric calibration parameters of the panoramic infrared camera can be completed.

Claims (4)

1. A geometric calibration method for a panoramic infrared camera is characterized by comprising the following steps:
1) selecting a corresponding infrared target according to the response characteristic of the camera; fixing one or more selected infrared targets in an area without shielding in front of the turntable, so that the infrared targets are opposite to the infrared camera;
2) enabling the control point to cover the whole target surface of the infrared camera: controlling a shaft of the rotary table to rotate at fixed intervals, acquiring the rotation angle of the rotary table at each rotation position of the rotary table, photographing an infrared target by using an infrared camera, and extracting a control point according to an image of the infrared target;
3) performing internal reference and external reference joint modeling on the complete imaging process of the infrared target in the panoramic infrared camera calibration process: establishing an external reference model for describing the position relation of the infrared target under a panoramic infrared camera coordinate system in the rotating process of the turntable by taking the mounting position of the panoramic infrared camera on the turntable, the mounting matrix and the position of the infrared target relative to the turntable as external references; respectively establishing a rough imaging model according to the design principle of the panoramic infrared camera and a fine imaging model according to polynomial fitting and distortion parameter correction by taking principal points and distortion coefficients of the panoramic infrared camera as internal parameters;
4) and (3) finishing the internal parameter and external parameter joint estimation in the model by adopting a two-step method: the first step is to form a nonlinear least square problem according to the external reference model and the rough imaging model established in the step 3) to complete parameter preliminary estimation, and the second step is to form a nonlinear least square problem according to the external reference model and the fine imaging model established in the step 3) to complete parameter optimization; therefore, the high-precision estimation of the geometric calibration parameters of the panoramic infrared camera is completed.
2. The geometric calibration method for the panoramic infrared camera according to claim 1, wherein in the step 2), the control points are extracted according to the image of the infrared target, and specifically:
firstly, calculating 3.5 times of the global standard deviation of the image pixel gray level of an image shot by a panoramic infrared camera and adding a global average value as a threshold value T to obtain a gray threshold value image:
Figure FDA0003256394210000021
wherein (u, v) represents coordinates of the pixel, F (u, v) represents an original gray level of the pixel, and F (u, v) represents a threshold gray level of the pixel; then, carrying out four-connected domain processing on the gray threshold image, and extracting the region of the image formed by the infrared target; and finally, finishing control point extraction by adopting a centroid positioning algorithm:
Figure FDA0003256394210000022
wherein m and n are the total number of pixels in the u-direction and v-direction, respectively, (u)c,vc) Representing the coordinates of the control points.
3. The geometric calibration method for the panoramic infrared camera according to claim 1, wherein the step 3) is specifically as follows:
the modeling by utilizing external parameters specifically comprises the following steps:
the rotary table is a double-shaft rotary table; using the rotation center O of the two-axis turntable as the origin and the two-axis turntableEstablishing a double-shaft turntable rotating coordinate system P by X, Y and Z axes; when the double-shaft turntable is located at an initial rotating position, defining a rotating coordinate system P at the moment as an inertial coordinate system B; with the optical centre O of the cameracLine of the image sensor as XcAxis, column of image sensor as YcAxis, ZcPerpendicular to XcAxis and YcThe axis is used for establishing a coordinate system C of the panoramic infrared camera;
on the rotation angle of the ith group of double-shaft rotary tables, for the control point corresponding to the jth infrared target obtained by collection, the coordinate of the control point under the coordinate system C of the panoramic infrared cameraCXDij＝[XDijC,YDijC,ZDijC]TThe following relationships are present:
Figure FDA0003256394210000023
whereinBXDj＝[XDjB,YDjB,ZDjB]TRepresenting the position of the jth infrared target under an inertial coordinate system B;
Figure FDA0003256394210000031
representing a rotation matrix from a biaxial turntable fixed coordinate system P to a panoramic infrared camera coordinate system C,
Figure FDA0003256394210000032
a rotation matrix representing a fixed coordinate system P from the inertial coordinate system B to the double-shaft turntable;
Figure FDA0003256394210000033
describes the installation relationship of a panoramic infrared camera on a double-shaft turntable, adopts three groups of independent parameters alpha and beta,
Figure FDA0003256394210000034
to be described, the method has the advantages that,
Figure FDA0003256394210000035
the expression of (a) is:
Figure FDA0003256394210000036
Figure FDA0003256394210000037
describes the rotation process of a double-shaft turntable, and two groups of independent parameters theta are adopted according to the actually adopted double-shaft turntablexi,θziTo be described, the method has the advantages that,
Figure FDA0003256394210000038
the expression of (a) is:
Figure FDA0003256394210000039
modeling by utilizing internal parameters, specifically:
image coordinate system with imaging center O on image sensorsThe pixel coordinate system takes the upper left corner of the image sensor as an origin, and the two plane coordinate systems both take the rows and the columns of the image sensor as an X axis and a Y axis;
according to the imaging principle of the panoramic infrared camera, the coordinates of the infrared target on the coordinate system C of the panoramic infrared camera areCXD＝[XDC,YDC,ZDC]TAnd projected on an image coordinate system as an image point p ' ═ u ', v ']TSpecifically, it is represented as:
λ·[XDC YDC ZDC]T＝g(u′,v′). (6)
wherein g represents a projection imaging relationship and λ represents a scaling factor;
in order to reduce the difficulty of parameter calculation, a rough model and a fine model are required to be established so as to realize initial parameter estimation and subsequent parameter optimization estimation;
a) coarse imaging model
Establishing a rough model according to an imaging principle designed by a panoramic infrared camera, wherein the imaging principle of the panoramic infrared camera comprises equidistant projection, orthogonal projection, equal solid angle projection and stereoscopic projection; the construction methods of rough models of the panoramic infrared cameras corresponding to the four projection principles are similar;
the construction method of the rough model of the panoramic infrared camera based on the equidistant projection imaging principle comprises the following steps:
the principle of isometric projection imaging is expressed as follows:
Figure FDA0003256394210000041
wherein f is the focal length of the camera, theta is the angle of the field of view corresponding to the target, and rho represents the image height; the coarse model is then:
λ·[XDC YDC ZDC]T＝[u′sinθ v′sinθ ρcosθ]T (8)
considering the shift of the principal point, the point p' in the image coordinate system is converted into p ═ u, v in the pixel coordinate system]TThe description is as follows:
[u v]T＝[u′ v′]T+[u0 v0]T (9)
wherein [ u ]0,v0]TRepresenting the imaging center O on the image sensorsA position in a pixel coordinate system;
b) fine imaging model
And (3) estimating and fitting a projection imaging function g by using a polynomial:
λ·[XDC YDC ZDC]T＝[u′ v′ a0+a2ρ2+a3ρ3+a4ρ4]T (10)
wherein a is0,a2,a3,a4Representing coefficients of each order;
taking into account the imaging process of the image sensor, the image coordinate systemThe point p' below is converted to p ═ u, v in the pixel coordinate system]TThe method comprises the following steps:
Figure FDA0003256394210000042
where k denotes the image sensor pixel aspect ratio and s denotes the tilt angle of the image sensor row and column.
4. The geometric calibration method for the panoramic infrared camera according to claim 3, wherein the step 4) is specifically as follows:
step 1: performing preliminary parameter estimation; using a rough model, in combination with equations (3), (7), (8) and (9):
Figure FDA0003256394210000051
wherein:
Pj＝(XDjB,YDjB,ZDjB) (13)
Figure FDA0003256394210000052
in the above formula, the first and second carbon atoms are,
Figure FDA0003256394210000053
a measured value representing a control point is shown,
Figure FDA0003256394210000054
estimated value representing control point, e1ij,e2ijRepresenting the estimated residual of the control point, Pr,PjRepresenting a matrix of unknown parameters, fx,fyRepresenting a non-linear function describing the above relationship;
thus, a non-linear least squares estimation problem is proposed, with minimized reprojection errors as the optimization objective:
Figure FDA0003256394210000055
wherein K represents the number of the infrared targets, and L represents the number of the rotation angle groups of the double-shaft turntable; solving the problem of nonlinear least square estimation by adopting a Levenberg-Marquardt algorithm; wherein the ratio of alpha, beta,
Figure FDA0003256394210000056
according to the corresponding relation between the panoramic infrared camera coordinate system C and the double-shaft turntable rotating coordinate system P, neglecting installation errors to give initial estimation u0,v0Taking the center of a pixel coordinate system as initial estimation, and taking a lens design value of the panoramic infrared camera as initial estimation;
step 2: optimizing parameters; on the basis of step 1, a fine model is adopted, and the combination of the formula (3), the formula (10) and the formula (11) is as follows:
Figure FDA0003256394210000057
wherein:
Figure FDA0003256394210000058
wherein P iseFor a matrix of unknown parameters, hx,hyRepresenting a non-linear function describing the above relationship;
a similar non-linear least squares problem is thus obtained:
Figure FDA0003256394210000061
the Levenberg-Marquardt algorithm is also adopted to solve the problem; wherein the parameters obtained in step 1 are selected as initial estimates, and 1 and 0 are selected as initial estimates of k and s; for f obtained, the following relationship is utilized:
ρ/tan(ρ/f)＝a0+a2ρ2+a3ρ3+a4ρ4 (19)
selecting rho in a group of panoramic infrared camera image ranges as input, substituting into formula (19), and obtaining a group of a through polynomial fitting0,a2,a3,a4As an initial estimate;
therefore, the high-precision estimation of the geometric calibration parameters of the panoramic infrared camera can be completed.
CN202111060670.9A 2021-09-10 2021-09-10 Geometric calibration method for panoramic infrared camera Active CN113781579B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111060670.9A CN113781579B (en) 2021-09-10 2021-09-10 Geometric calibration method for panoramic infrared camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111060670.9A CN113781579B (en) 2021-09-10 2021-09-10 Geometric calibration method for panoramic infrared camera

Publications (2)

Publication Number Publication Date
CN113781579A true CN113781579A (en) 2021-12-10
CN113781579B CN113781579B (en) 2022-11-11

Family

ID=78842456

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111060670.9A Active CN113781579B (en) 2021-09-10 2021-09-10 Geometric calibration method for panoramic infrared camera

Country Status (1)

Country Link
CN (1) CN113781579B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114877871A (en) * 2022-05-06 2022-08-09 中国人民解放军国防科技大学 Attitude staring control method for deep space target observation by uncalibrated video satellite

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103646394A (en) * 2013-11-26 2014-03-19 福州大学 Mixed visual system calibration method based on Kinect camera
CN104732542A (en) * 2015-03-27 2015-06-24 安徽省道一电子科技有限公司 Image processing method for panoramic vehicle safety system based on multi-camera self calibration
US20160105649A1 (en) * 2014-10-10 2016-04-14 IEC Infrared Systems LLC Panoramic View Imaging System With Drone Integration
CN106548477A (en) * 2017-01-24 2017-03-29 长沙全度影像科技有限公司 A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target
US11080544B1 (en) * 2020-04-24 2021-08-03 StradVision, Inc. Method and device for calibrating pitch of camera on vehicle and method and device for continual learning of vanishing point estimation model to be used for calibrating the pitch

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103646394A (en) * 2013-11-26 2014-03-19 福州大学 Mixed visual system calibration method based on Kinect camera
US20160105649A1 (en) * 2014-10-10 2016-04-14 IEC Infrared Systems LLC Panoramic View Imaging System With Drone Integration
CN104732542A (en) * 2015-03-27 2015-06-24 安徽省道一电子科技有限公司 Image processing method for panoramic vehicle safety system based on multi-camera self calibration
CN106548477A (en) * 2017-01-24 2017-03-29 长沙全度影像科技有限公司 A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target
US11080544B1 (en) * 2020-04-24 2021-08-03 StradVision, Inc. Method and device for calibrating pitch of camera on vehicle and method and device for continual learning of vanishing point estimation model to be used for calibrating the pitch

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XU CHEN ET AL.: "A Structured-Light-Based Panoramic Depth Camera", 《2018 IEEE INTERNATIONAL CONFERENCE ON REAL-TIME COMPUTING AND ROBOTICS (RCAR)》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114877871A (en) * 2022-05-06 2022-08-09 中国人民解放军国防科技大学 Attitude staring control method for deep space target observation by uncalibrated video satellite
CN114877871B (en) * 2022-05-06 2024-01-09 中国人民解放军国防科技大学 Attitude staring control method for deep space target observation by uncalibrated video satellite

Also Published As

Publication number Publication date
CN113781579B (en) 2022-11-11

Similar Documents

Publication Publication Date Title
CN110296691B (en) IMU calibration-fused binocular stereo vision measurement method and system
Shah et al. A simple calibration procedure for fish-eye (high distortion) lens camera
Luhmann et al. Sensor modelling and camera calibration for close-range photogrammetry
US7479982B2 (en) Device and method of measuring data for calibration, program for measuring data for calibration, program recording medium readable with computer, and image data processing device
CN109029299B (en) Dual-camera measuring device and method for butt joint corner of cabin pin hole
CN107680139B (en) Universality calibration method of telecentric binocular stereo vision measurement system
CN109859272B (en) Automatic focusing binocular camera calibration method and device
Zhang et al. A robust and rapid camera calibration method by one captured image
CN110146038A (en) The distributed monocular camera laser measuring device for measuring and method of cylindrical member assembly corner
CN110966935B (en) Deflection measurement system integrated geometric calibration method based on mark points
CN110146030A (en) Side slope surface DEFORMATION MONITORING SYSTEM and method based on gridiron pattern notation
CN103017666A (en) Method and assembly for determining the position of a measurement point in geometric space
US20220230348A1 (en) Method and apparatus for determining a three-dimensional position and pose of a fiducial marker
CN111896221A (en) Alignment method of rotating optical measurement system for virtual coordinate system auxiliary camera calibration
CN106871900A (en) Image matching positioning method in ship magnetic field dynamic detection
CN111915685B (en) Zoom camera calibration method
CN109212497A (en) A kind of measurement of space six degree of freedom vehicle radar antenna pose deviation and interconnection method
US20220412795A1 (en) Sensor Arrangement
CN112767338A (en) Assembled bridge prefabricated part hoisting and positioning system and method based on binocular vision
CN113920206A (en) Calibration method of perspective tilt-shift camera
CN113724337A (en) Camera dynamic external parameter calibration method and device without depending on holder angle
Ding et al. A robust detection method of control points for calibration and measurement with defocused images
CN110827360B (en) Photometric stereo measurement system and method for calibrating light source direction thereof
CN113781579B (en) Geometric calibration method for panoramic infrared camera
Crispel et al. All-sky photogrammetry techniques to georeference a cloud field

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant