CN105678690A - Image registration method on the basis of optical imaging sensor internal and external parameters - Google Patents

Abstract

The invention discloses an image registration method on the basis of optical imaging sensor internal and external parameters. The method comprises following steps: S1. converting a pixel coordinate S(u1, v1) of an image to be registered obtained by a first optical imaging sensor to a world coordinate S(WX, WY, WZ); S2. using the pixel coordinate system of a reference image obtained by a second optical imaging sensor as parameters, converting the world coordinate S(WX, WY, WZ) of the image to be registered to a pixel coordinate S(u2, v2) under a pixel coordinate system of the reference image; S3. performing rounding and interpolation to the reference image and the image to be registered under a same pixel coordinate system to obtain a final registered image. By use of the method, the computational complexity of registration algorithm is reduced, the algorithm robustness is increased, and registration problems among images of different sources are effectively solved.

Description

A kind of method for registering images of optically-based imaging sensor inside and outside parameter

Technical field

The present invention relates to a kind of method for registering images, particularly to a kind of method utilizing optical imaging sensor inside and outside parameter to carry out image registration, be mainly used in aerospace digital image processing field.

Technical background

Image registration techniques is one of indispensable core technologies such as image co-registration, image mosaic, scene matching aided navigation, and only image registration problem is effectively solved, and follow-up work just can carry on. It is all widely used as assessment and understanding, remote sensing, computer vision, medical diagnosis, intelligent transportation, gis database renewal, environmental monitoring, wide area video supervision etc. are damaged in image object fusion detection, precision navigation, precise guidance, war on military, civilian.

Current automatic image registration method is broadly divided into 2 classes: based on the method for registering images of gray scale, typically directly utilize the half-tone information of the image similarity measurement to set up between image, then search makes the parameter value that similarity measurement is maximum, and then determine the solution of registration problems, common algorithms has cross-correlation method, fourier transform method, mutual information method etc.; The method for registering images of feature based sets up Matching Model by extracting in image the feature with invariance, common put the line feature such as feature and objective contour such as cross point, angle point, marginal point, some feature etc. and is all used in registration Algorithm.

Summary of the invention

The goal of the invention of the present invention is in that the method for registering images providing a kind of present invention to propose a kind of optically-based imaging sensor inside and outside parameter, using imaging sensor inside and outside parameter as the major parameter in image registration algorithm, by strict mathematical derivation, calculate the transformation model between image. The registration problems that present invention reduces the computation complexity of registration Algorithm, improve algorithm robustness, effectively solve between allos image.

The goal of the invention of the present invention is achieved through the following technical solutions:

The method for registering images of a kind of optically-based imaging sensor inside and outside parameter, comprises the steps of

Step one, the pixel coordinate S (u of image subject to registration that the first optical imaging sensor is obtained₁,v₁) be transformed into world coordinates S (^WX,^WY,^WZ):

$\left[\begin{array}{c}X_W\\ Y_W\\ Z_W\\ 1\end{array}\right]={Z_C}_1{M}_{11}^{-1}{M}_{12}{M}_{13}^{-1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]$

Wherein,^CZ₁For the distance of the zero of a S to first optical pickocff,

$M_{11}={\left[\begin{array}{cc}{R}_1& T_1\\ 0^T& 1\end{array}\right]}_{4\×4}$

$M_{12}={\left[\begin{array}{ccc}{f}_1^{-1}& 0& 0\\ 0& f_1^{-1}& 0\\ 0& 0& 1\\ 0& 0& \frac{1}{{Z_C}_1}\end{array}\right]}_{4\×3}$

$M_{13}={\left[\begin{array}{ccc}{s}_{1x}& 0& u_{10}\\ 0& s_{1y}& v_{10}\\ 0& 0& 1\end{array}\right]}_{3\×3}$

f₁It is the focal length of the first optical pickocff, (u₁₀,v₁₀) for the pixel coordinate of physical coordinates initial point of image subject to registration, s_1x,s_1yFor image as unit length number of pixels subject to registration, R₁It is the rotational coordinates utilizing the attitude information of the first optical imaging sensor to calculate, for

$R_1=\left[\begin{array}{ccc}{\mathrm{cos\γ}}_1& 0& -{\mathrm{sin\γ}}_1\\ 0& 1& 0\\ {\mathrm{sin\γ}}_1& 0& {\mathrm{cos\γ}}_1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& {\mathrm{cos\β}}_1& {\mathrm{sin\β}}_1\\ 0& -{\mathrm{sin\β}}_1& {\mathrm{cos\β}}_1\end{array}\right]\left[\begin{array}{ccc}{\mathrm{cos\α}}_1& -{\mathrm{sin\α}}_1& 0\\ {\mathrm{sin\α}}_1& {\mathrm{cos\α}}_1& 0\\ 0& 0& 1\end{array}\right],$

α₁、β₁、γ₁Respectively the first optical imaging sensor is relative to the azimuth of world coordinate system, the angle of pitch, roll angle, T₁It is the world coordinate system initial point translation vector to the first optical imaging sensor, for

$T_1=\left[\begin{array}{c}{x_W}_1\\ {y_W}_1\\ {z_W}_1\end{array}\right]$

O₁(^Wx₁,^Wy₁,^Wz₁) it is first optical imaging sensor position vector at world coordinate system;

Step 2, the pixel coordinate system of benchmark image obtained with the second optical imaging sensor for parameter, by the world coordinates S of image subject to registration (^WX,^WY,^WZ) the pixel coordinate S (u under the pixel coordinate system of benchmark image it is converted to into₂,v₂):

$\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=\frac{1}{{Z_C}_2}{M}_{24}{M}_{21}\left[\begin{array}{c}X_W\\ Y_W\\ Z_W\\ 1\end{array}\right]$

Wherein,^CZ₂For the distance of a S to second optical imaging sensor zero,

$M_{24}={\left[\begin{array}{cccc}{f}_2{s}_{2x}& 0& u_{20}& 0\\ 0& f_2{s}_{2y}& v_{20}& 0\\ 0& 0& 1& 0\end{array}\right]}_{3\×4}$

$M_{21}={\left[\begin{array}{cc}{R}_2& T_2\\ 0^T& 1\end{array}\right]}_{4\×4}$

f₂It is the focal length of the second optical imaging sensor, (u₂₀,v₂₀) for the pixel coordinate of benchmark image physical coordinates initial point, s_2x,s_2yFor benchmark image unit length number of pixels, R₂It is the rotational coordinates utilizing the attitude information of the second optical imaging sensor to calculate, for

$R_2=\left[\begin{array}{ccc}{\mathrm{cos\γ}}_2& 0& -{\mathrm{sin\γ}}_2\\ 0& 1& 0\\ {\mathrm{sin\γ}}_2& 0& {\mathrm{cos\γ}}_2\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& {\mathrm{cos\β}}_2& {\mathrm{sin\β}}_2\\ 0& -{\mathrm{sin\β}}_2& {\mathrm{cos\β}}_2\end{array}\right]\left[\begin{array}{ccc}{\mathrm{cos\α}}_2& -{\mathrm{sin\α}}_2& 0\\ {\mathrm{sin\α}}_2& {\mathrm{cos\α}}_2& 0\\ 0& 0& 1\end{array}\right],$

α₂、β₂、γ₂Respectively the second optical imaging sensor is relative to the azimuth of world coordinate system, the angle of pitch, roll angle,

T₂It is the world coordinate system initial point translation vector to the second optical imaging sensor, for

$T_2=\left[\begin{array}{c}{x_W}_2\\ {y_W}_2\\ {z_W}_2\end{array}\right],$

O₂(^Wx₂,^Wy₂,^Wz₂) it is second optical imaging sensor position vector at world coordinate system;

Step 3, the benchmark image that will be located under same pixel coordinate system and image subject to registration are by rounding and obtaining final registering images after interpolation.

According to features described above, described first optical imaging sensor and the second optical imaging sensor are foreign peoples's optical imaging sensor.

With prior art proportioning, the beneficial effects of the present invention is: the present invention utilizes imaging sensor inside and outside parameter as the major parameter in image registration algorithm, by strict mathematical derivation, calculate the transformation model between image, effectively reduce the computation complexity of registration Algorithm, improve algorithm robustness, the registration problems effectively solving between allos image.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet that a kind of present invention of the present invention proposes the method for registering images of a kind of optically-based imaging sensor inside and outside parameter.

The image schematic diagram subject to registration that Fig. 2 (a) collects for visible light sensor in embodiment;

The benchmark image schematic diagram that Fig. 2 (b) arrives for embodiment mid-infrared light sensor acquisition;

The Fig. 2 (c) the image schematic diagram for obtaining after adopting method for registering images of the present invention in embodiment.

Detailed description of the invention

Below according to drawings and Examples, the present invention is described in further detail.

As it is shown in figure 1, embodiment of the present invention flow process is as follows:

First optical imaging sensor is O in the position vector of world coordinate system₁(^Wx₁,^Wy₁,^Wz₁), relative to the attitude angle respectively azimuth angle alpha of world coordinate system₁, angle of pitch β₁, roll angle γ₁, focal length is f₁. The pixel coordinate of the physical coordinates initial point of the image subject to registration obtained by the first optical imaging sensor is (u₁₀,v₁₀), unit length number of pixels is s_1x,s_1y. Second optical imaging sensor is O in the position vector of world coordinate system₂(^Wx₂,^Wy₂,^Wz₂), relative to the attitude angle respectively azimuth angle alpha of world coordinate system₂, angle of pitch β₂, roll angle γ₂, focal length is f₂. The pixel coordinate of the physical coordinates initial point of benchmark image is (u₂₀,v₂₀), unit length number of pixels is s_2x,s_2y. In world coordinate system some coordinates in space be P (^Wx_p,^Wy_p,^Wz_p), this point projects to the first optical imaging sensor and the imaging plane of the second optical imaging sensor simultaneously. Assume that above variable all can obtain.

It is now assumed that the first optical imaging sensor obtains a certain pixel S (u of image subject to registration₁,v₁), transformed to according to transformation matrix Γ under the image pixel coordinates system of the second optical imaging sensor as S (u₂,v₂)

$\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=\mathrm{\Γ}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right].$

Transformation matrix Γ calculates by the following method:

Step one: by the pixel coordinate S (u of image subject to registration₁,v₁) be converted to world coordinates S (^WX,^WY,^WZ), for

$\left[\begin{array}{c}X_W\\ Y_W\\ Z_W\\ 1\end{array}\right]={Z_C}_1{M}_{11}^{-1}{M}_{12}{M}_{13}^{-1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]$

Wherein,^CZ₁For the distance of a S to first optical imaging sensor zero, R₁It is the rotational coordinates utilizing the attitude information of the first optical imaging sensor to calculate, for

$R_1=\left[\begin{array}{ccc}{\mathrm{cos\γ}}_1& 0& -{\mathrm{sin\γ}}_1\\ 0& 1& 0\\ {\mathrm{sin\γ}}_1& 0& {\mathrm{cos\γ}}_1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& {\mathrm{cos\β}}_1& {\mathrm{sin\β}}_1\\ 0& -{\mathrm{sin\β}}_1& {\mathrm{cos\β}}_1\end{array}\right]\left[\begin{array}{ccc}{\mathrm{cos\α}}_1& -{\mathrm{sin\α}}_1& 0\\ {\mathrm{sin\α}}_1& {\mathrm{cos\α}}_1& 0\\ 0& 0& 1\end{array}\right]$

T₁It is the world coordinate system initial point translation vector to the first optical imaging sensor, for

$T_1=\left[\begin{array}{c}{x_W}_1\\ {y_W}_1\\ {z_W}_1\end{array}\right]$

M₁₁,M₁₂And M₁₃The parameter being all based on the first optical imaging sensor respectively is calculated, for

$M_{11}={\left[\begin{array}{cc}{R}_1& T_1\\ 0^T& 1\end{array}\right]}_{4\×4}$

$M_{12}={\left[\begin{array}{ccc}{f}_1^{-1}& 0& 0\\ 0& f_1^{-1}& 0\\ 0& 0& 1\\ 0& 0& \frac{1}{{Z_C}_1}\end{array}\right]}_{4\×3}$

$M_{13}={\left[\begin{array}{ccc}{s}_{1x}& 0& u_{10}\\ 0& s_{1y}& v_{10}\\ 0& 0& 1\end{array}\right]}_{3\×3}$

Step 2: by the world coordinates S of image subject to registration (^WX,^WY,^WZ) the second optical imaging sensor it is converted to

Pixel coordinate S (u under coordinate system₂,v₂), for

$\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=\frac{1}{{Z_C}_2}{M}_{24}{M}_{21}\left[\begin{array}{c}X_W\\ Y_W\\ Z_W\\ 1\end{array}\right]$

Wherein,^CZ₂For the distance of a S to second optical imaging sensor zero, R₂It is the rotational coordinates utilizing the attitude information of the second optical imaging sensor to calculate, for

$R_2=\left[\begin{array}{ccc}{\mathrm{cos\γ}}_2& 0& -{\mathrm{sin\γ}}_2\\ 0& 1& 0\\ {\mathrm{sin\γ}}_2& 0& {\mathrm{cos\γ}}_2\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& {\mathrm{cos\β}}_2& {\mathrm{sin\β}}_2\\ 0& -{\mathrm{sin\β}}_2& {\mathrm{cos\β}}_2\end{array}\right]\left[\begin{array}{ccc}{\mathrm{cos\α}}_2& -{\mathrm{sin\α}}_2& 0\\ {\mathrm{sin\α}}_2& {\mathrm{cos\α}}_2& 0\\ 0& 0& 1\end{array}\right]$

T₂It is the world coordinate system initial point translation vector to the second optical imaging sensor, for

$T_2=\left[\begin{array}{c}{x_W}_2\\ {y_W}_2\\ {z_W}_2\end{array}\right]$

M₂₄And M₂₁The parameter being all based on the second optical imaging sensor respectively is calculated, for

$M_{24}={\left[\begin{array}{cccc}{f}_2{s}_{2x}& 0& u_{20}& 0\\ 0& f_2{s}_{2y}& v_{20}& 0\\ 0& 0& 1& 0\end{array}\right]}_{3\×4}$

$M_{21}={\left[\begin{array}{cc}{R}_2& T_2\\ 0^T& 1\end{array}\right]}_{4\×4}$

The original pixels coordinate S (u of image subject to registration₁,v₁) be converted to the second optical imaging sensor for reference

Image pixel coordinates S (u₂,v₂), for

$\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]=\frac{1}{{Z_C}_2}{M}_{24}{M}_{21}{Z_C}_1{M}_{11}^{-1}{M}_{12}{M}_{13}^{-1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]=\frac{{Z_C}_1}{{Z_C}_2}{M}_{24}{M}_{21}{M}_{11}^{-1}{M}_{12}{m}_{13}^{-1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]$

Therefore try to achieve the first optical imaging sensor and to the transformation matrix of the second optical imaging sensor be

$\mathrm{\Γ}=\frac{{Z_C}_1}{{Z_C}_2}{M}_{24}{M}_{21}{M}_{11}^{-1}{M}_{12}{M}_{13}^{-1}$

Image pixel coordinates subject to registration is sequentially passed through transformation matrix Γ and is converted under benchmark image pixel coordinate system;

By round with interpolation after obtain final registering images.

Shown in the effect of the present invention such as Fig. 2 (a), 2 (b), 2 (c), Fig. 2 (a) is the image schematic diagram subject to registration that the first optical imaging sensor (for visible light sensor in the present embodiment) collects; Fig. 2 (b) is the benchmark image schematic diagram that the first optical imaging sensor (for infrared light transducer[sensor in the present embodiment) collects; Fig. 2 (c) is the image schematic diagram that will be obtained after image subject to registration and benchmark image registration by the present invention.

It is understood that for those of ordinary skills, it is possible to it is equal to replacement according to technical scheme and inventive concept thereof or is changed, and all these are changed or replace the scope of the claims that all should belong to appended by the present invention.

Claims (2)

1. a method for registering images for optically-based imaging sensor inside and outside parameter, comprises the steps of

Step one, the pixel coordinate S (u of image subject to registration that the first optical imaging sensor is obtained₁,v₁) be transformed into world coordinates S (^WX,^WY,^WZ):

$\left[\begin{array}{c}X_W\\ Y_W\\ Z_W\\ 1\end{array}\right]={Z_C}_1{M}_{11}^{-1}{M}_{12}{M}_{13}^{-1}\left[\begin{array}{c}{u}_1\\ v_1\\ 1\end{array}\right]$

Wherein,^CZ₁For the distance of the zero of a S to first optical pickocff,

$M_{11}={\left[\begin{array}{cc}{R}_1& T_1\\ 0^T& 1\end{array}\right]}_{4\×4}$

$M_{12}={\left[\begin{array}{ccc}{f}_1^{-1}& 0& 0\\ 0& f_1^{-1}& 0\\ 0& 0& 1\\ 0& 0& \frac{1}{{Z_C}_1}\end{array}\right]}_{4\×3}$

$M_{13}={\left[\begin{array}{ccc}{s}_{1x}& 0& u_{10}\\ 0& s_{1y}& v_{10}\\ 0& 0& 1\end{array}\right]}_{3\×3}$

f₁It is the focal length of the first optical pickocff, (u₁₀,v₁₀) for the pixel coordinate of physical coordinates initial point of image subject to registration, s_1x,s_1yFor image as unit length number of pixels subject to registration, R₁It is the rotational coordinates utilizing the attitude information of the first optical imaging sensor to calculate, for

$R_1=\left[\begin{array}{ccc}{\mathrm{cos\γ}}_1& 0& -{\mathrm{sin\γ}}_1\\ 0& 1& 0\\ {\mathrm{sin\γ}}_1& 0& {\mathrm{cos\γ}}_1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& {\mathrm{cos\β}}_1& {\mathrm{sin\β}}_1\\ 0& -{\mathrm{sin\β}}_1& {\mathrm{cos\β}}_1\end{array}\right]\left[\begin{array}{ccc}{\mathrm{cos\α}}_1& -{\mathrm{sin\α}}_1& 0\\ {\mathrm{sin\α}}_1& {\mathrm{cos\α}}_1& 0\\ 0& 0& 1\end{array}\right],$

α₁、β₁、γ₁Respectively the first optical imaging sensor is relative to the azimuth of world coordinate system, the angle of pitch, roll angle, T₁It is the world coordinate system initial point translation vector to the first optical imaging sensor, for

$T_1=\left[\begin{array}{c}{x_W}_1\\ {y_W}_1\\ {z_W}_1\end{array}\right]$

O₁(^Wx₁,^Wy₁,^Wz₁) it is first optical imaging sensor position vector at world coordinate system;

Step 2, the pixel coordinate system of benchmark image obtained with the second optical imaging sensor for parameter, by the world coordinates S of image subject to registration (^WX,^WY,^WZ) the pixel coordinate S (u under the pixel coordinate system of benchmark image it is converted to into₂,v₂):

$\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=\frac{1}{{Z_C}_2}{M}_{24}{M}_{21}\left[\begin{array}{c}X_W\\ Y_W\\ Z_W\\ 1\end{array}\right]$

Wherein,^CZ₂For the distance of a S to second optical imaging sensor zero,

$M_{24}={\left[\begin{array}{cccc}{f}_2{s}_{2x}& 0& u_{20}& 0\\ 0& f_2{s}_{2y}& v_{20}& 0\\ 0& 0& 1& 0\end{array}\right]}_{3\×4}$

$M_{21}={\left[\begin{array}{cc}{R}_2& T_2\\ 0^T& 1\end{array}\right]}_{4\×4}$

f₂It is the focal length of the second optical imaging sensor, (u₂₀,v₂₀) for the pixel coordinate of benchmark image physical coordinates initial point, s_2x,s_2yFor benchmark image unit length number of pixels, R₂It is the rotational coordinates utilizing the attitude information of the second optical imaging sensor to calculate, for

$R_2=\left[\begin{array}{ccc}{\mathrm{cos\γ}}_2& 0& -{\mathrm{sin\γ}}_2\\ 0& 1& 0\\ {\mathrm{sin\γ}}_2& 0& {\mathrm{cos\γ}}_2\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& {\mathrm{cos\β}}_2& {\mathrm{sin\β}}_2\\ 0& -{\mathrm{sin\β}}_2& {\mathrm{cos\β}}_2\end{array}\right]\left[\begin{array}{ccc}{\mathrm{cos\α}}_2& -{\mathrm{sin\α}}_2& 0\\ {\mathrm{sin\α}}_2& {\mathrm{cos\α}}_2& 0\\ 0& 0& 1\end{array}\right],$

α₂、β₂、γ₂Respectively the second optical imaging sensor is relative to the azimuth of world coordinate system, the angle of pitch, roll angle,

T₂It is the world coordinate system initial point translation vector to the second optical imaging sensor, for

$T_2=\left[\begin{array}{c}{x_W}_2\\ {y_W}_2\\ {z_W}_2\end{array}\right],$

O₂(^Wx₂,^Wy₂,^Wz₂) it is second optical imaging sensor position vector at world coordinate system;

Step 3, the benchmark image that will be located under same pixel coordinate system and image subject to registration are by rounding and obtaining final registering images after interpolation.

2. method for registering images according to claim 1, it is characterised in that described first optical imaging sensor and the second optical imaging sensor are foreign peoples's optical imaging sensor.