WO2017161608A1

WO2017161608A1 - Geometric calibration processing method and device for camera

Info

Publication number: WO2017161608A1
Application number: PCT/CN2016/078905
Authority: WO
Inventors: 赵博
Original assignee: 完美幻境(北京)科技有限公司
Priority date: 2016-03-21
Filing date: 2016-04-08
Publication date: 2017-09-28
Also published as: CN105701837A

Abstract

Embodiments of the present invention provide a geometric calibration processing method and device for a camera. The method comprises: obtaining coordinates of a spatial point and coordinates of an image point, a panorama camera shoots the spatial point to obtain the image point corresponding to the spatial point, the spatial point being a point on a spatial coordinate system and the image point is a point on an image coordinate system; obtaining a panoramic camera imaging model, the panoramic camera imaging model being used for indicating a conversion relationship between a point on the spatial coordinate system and a point on an image coordinate system; and by using the coordinates of the spatial point, the coordinates of the image point and the panoramic camera imaging model, determining an external parameter of the panorama camera. In the solution, there is no defect that spatial points are coplanar according to a two-step calibration method and there is no defect that an initial value error is large because various distortion factors are not considered according to the calibration method put forwarded by Zhang Youzheng.

Description

Camera geometric calibration processing method and device

Technical field

The present invention relates to the field of image processing, and in particular, to a camera geometric calibration processing method and apparatus.

Background technique

Panoramic shooting usually refers to a method of shooting 360 degrees and 180 degrees vertically with a certain point as the center, and stitching a plurality of pictures taken into a single panoramic picture and a picture stitching method. In general, panoramic shooting can include at least a panoramic image and a panoramic video.

Usually, when stitching a plurality of original pictures taken into one panoramic picture, mapping and stitching are involved. The mapping can be understood as projecting the pixel points on the original picture to the corresponding positions of the panoramic picture, and the splicing can be understood as a fusion transition of the overlapping areas of the two adjacent original pictures.

In order to determine the relationship between the three-dimensional geometric position of a point on the surface of the space object and its corresponding point in the original picture, the camera parameters can be obtained by means of camera geometric calibration, so that the camera point can be subsequently used for pixel point projection. Typically, camera parameters may include the external parameters of the camera and the internal parameters of the camera.

At present, the commonly used external parameter estimation method mainly has two-step calibration method and Zhang Zhengyou calibration method.

For the two-step calibration method, the spatial points are not coplanar when solving the external parameters, and the external parameters cannot be obtained if the coplanar is coplanar. Therefore, for the coplanar flat calibration block, other methods must be used for external parameter estimation. In addition, the two-step calibration method assumes that the lens has only radial distortion, and the estimation error of the fisheye lens imaging is large.

For the Zhang Youzheng calibration method, instead of considering various distortions, all the points are substituted into the solution. However, the distortion of the pixels that are usually far from the center of the image is very large. If these pixels are also regarded as pixels without distortion, Substituting the solution will obviously increase the error of solving the initial value. Similarly, Zhang Youzheng's calibration method only considers radial distortion and does not apply to fisheye lenses.

Summary of the invention

The embodiment of the invention provides a camera geometric calibration processing method and device, which can effectively avoid the defects existing in the existing calibration scheme.

A camera geometric calibration processing method, the method comprising:

Obtaining coordinates of the spatial point and coordinates of the image point, and the panoramic camera captures the spatial point, and obtains the image point corresponding to the spatial point, where the spatial point is a point on a spatial coordinate system, and the image point is a point on the image coordinate system;

Obtaining a panoramic camera imaging model for representing a conversion relationship between a point on the spatial coordinate system and a point on the image coordinate system;

The outer parameters of the panoramic camera are determined using the coordinates of the spatial point, the coordinates of the image point, and the panoramic camera imaging model.

Preferably, the acquiring a panoramic camera imaging model comprises:

Performing a linear transformation on a point on the spatial coordinate system to obtain a point on a lens coordinate system of the panoramic camera;

Performing a nonlinear transformation on a point on the lens coordinate system to obtain a point on the sensor coordinate system of the lens;

Performing an affine transformation on a point on the sensor coordinate system to obtain a point on the image coordinate system;

The panoramic camera imaging model is established based on points on the spatial coordinate system and points on the image coordinate system.

Preferably, the points on the spatial coordinate system are converted to points on the lens coordinate system by the following formula:

Where (x _ij , y _ij , z _ij ) represents a point on the spatial coordinate system, and (u′′ _ij , v′′ _ij , f(u′′ _ij , v′′ _ij )) represent points on the lens coordinate system,

λ _ij denotes a normalization parameter, P ⁱ denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, and j denotes a j-th point on the coordinate system.

Preferably, the points on the lens coordinate system are converted to points on the sensor coordinate system by the following formula:

g(u" _ij ,v" _ij )=(u" _ij ,v" _ij ,f(u" _ij ,v" _ij )) ^T

Where (u" _ij , v" _ij , f ( u " _ij , v " _ij )) represent points on the lens coordinate system, (u" _ij , v" _ij ) represents points on the sensor coordinate system, and T represents Let i denote the i-th camera of the panoramic camera and j denote the j-th point on the coordinate system.

Preferably, the points on the sensor coordinate system are converted to points on the image coordinate system by the following formula:

u" _ij =Au' _ij +t ₁ ,v" _ij =Av' _ij +t ₁

Where (u" _ij , v" _ij ) represents a point on the sensor coordinate system, (u' _ij , v' _ij ) represents a point on the image coordinate system, i represents the ith camera of the panoramic camera, and j represents the coordinate system On the jth point, A represents the second rotation matrix, and t ₁ represents the translation matrix.

Preferably, the panoramic camera imaging model is established based on a point on the space coordinate system and a point on the image coordinate system, including:

Obtaining a correspondence between a point on the spatial coordinate system and a point on the image coordinate system:

(u' _ij , v' _ij ) denotes a point on the image coordinate system, λ _ij denotes a normalization parameter, P ⁱ denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, j Representing the jth point on the coordinate system, A represents the second rotation matrix, and t ₁ represents the translation matrix;

If z _ij is 0, the correspondence between the point on the space coordinate system and the point on the image coordinate system is:

If both ends of the equation are multiplied by p _ij at the same time, the correspondence between the point on the space coordinate system and the point on the image coordinate system is:

The corresponding relation obtained by the decomposition fork is obtained, and the imaging model of the i-th camera of the panoramic camera is obtained:

v' _j ·(r ₃₁ x _j +r ₃₂ y _j +t ₃ )-f(ρ _j )·(r ₂₁ x _j +r ₂₂ y _j +t ₂ )=0

f(ρ _j )·(r ₁₁ x _j +r ₁₂ y _j +t ₁ )-u' _j ·(r ₃₁ x _j +r ₃₂ y _j +t ₃ )=0.

u' _j ·(r ₂₁ x _j +r ₂₂ y _j +t ₂ )-v' _j ·(r ₁₁ x _j +r ₁₂ y _j +t ₁ )=0

A camera geometric calibration processing device, the device comprising:

a coordinate acquiring unit, configured to acquire coordinates of the spatial point and coordinates of the image point, and the panoramic camera captures the spatial point, and obtains the image point corresponding to the spatial point, where the spatial point is a point on the spatial coordinate system The image point is a point on the image coordinate system;

An imaging model acquisition unit, configured to acquire a panoramic camera imaging model, the panoramic camera imaging model is configured to represent a conversion relationship between a point on the spatial coordinate system and a point on the image coordinate system;

The outer parameter determining unit is configured to determine an outer parameter of the panoramic camera by using coordinates of the spatial point, coordinates of the image point, and the panoramic camera imaging model.

Preferably, the imaging model obtaining unit comprises:

a linear transformation unit configured to linearly transform a point on the spatial coordinate system to obtain a point on a lens coordinate system of the panoramic camera;

a nonlinear transform unit configured to perform nonlinear transformation on a point on the lens coordinate system to obtain a point on a sensor coordinate system of the lens;

An affine transformation unit, configured to perform affine transformation on a point on the sensor coordinate system to obtain a point on the image coordinate system;

An imaging model establishing unit configured to establish the panoramic camera imaging model based on a point on the spatial coordinate system and a point on the image coordinate system.

Preferably, the linear transformation unit is configured to convert a point on the spatial coordinate system to a point on the lens coordinate system by using the following formula:

The nonlinear transformation unit is configured to convert a point on the lens coordinate system to a point on the sensor coordinate system by using the following formula:

g(u" _ij ,v" _ij )=(u" _ij ,v" _ij ,f(u" _ij ,v" _ij )) ^T ;

The affine transformation unit is configured to convert a point on the sensor coordinate system to a point on the image coordinate system by using the following formula:

u" _ij =Au' _ij +t ₁ ,v" _ij =Av' _ij +t ₁

λ _ij denotes a normalization parameter, P ⁱ denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, and j denotes a j-th point on the coordinate system, (u′′ _ij , v′′ _Ij ) represents a point on the sensor coordinate system, T represents a transpose, (u′ _ij , v′ _ij ) represents a point on the image coordinate system, A represents a second rotation matrix, and t ₁ represents a translation matrix.

Preferably, an imaging model establishing unit is configured to obtain an imaging model of the i-th camera of the panoramic camera:

Compared with the prior art, the solution of the present invention obtains a panoramic camera imaging model representing a conversion relationship between a spatial point and an image point by studying a process of converting a spatial point into an image point, and simultaneously acquires a spatial point coordinate and a corresponding time in real time. Image point coordinates, which in turn determine the external parameters of the panoramic camera. In this scheme, there is no defect that the two-step calibration method requires that the spatial points are not coplanar, and there is no defect that the Zhangyouzheng calibration method does not consider various distortions and causes large initial value errors.

In addition, the inventive scheme considers the linear transformation from the space coordinate system to the lens coordinate system, the nonlinear transformation of the lens coordinate system to the sensor coordinate system, and the affine transformation from the sensor coordinate system to the image coordinate system when establishing the imaging model, not only considering Radial distortion, barrel distortion is also considered, which helps to improve the accuracy of the external parameter estimation of the present invention.

In addition, when solving the specific value of the external parameter of the panoramic camera, the least squares problem can be solved by the machine learning method, so that the calculation amount can be greatly reduced, and the solution of the present invention is applied to the real-time external parameter estimation of the embedded system.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are only It is only some embodiments of the present invention, and other drawings can be obtained from those skilled in the art without any inventive labor.

1 is a flow chart of a method for processing geometric calibration of a camera of the present invention;

2 is a flow chart of a method of establishing a panoramic camera imaging model in the present invention;

3 is a schematic structural view of a camera geometric calibration processing apparatus of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1, a flowchart of a camera geometric calibration processing method according to an embodiment of the present invention is shown, which may include the following steps:

S101. Acquire coordinates of a spatial point and coordinates of an image point. The panoramic camera captures the spatial point, and obtains the image point corresponding to the spatial point, where the spatial point is a point on a spatial coordinate system, and the image is The point is a point on the image coordinate system.

In order to achieve horizontal 360 degree and vertical 180 degree shooting, panoramic cameras typically include multiple cameras for multi-angle shooting. Each camera can capture spatial points on the spatial coordinate system to form corresponding image points on the image coordinate system. For example, the panoramic camera includes m cameras, and this step can obtain L sets of spatial point coordinates and image point coordinates for each camera, that is, for the panoramic camera, m*L group spatial point coordinates and image point coordinates can be obtained. In general, the value of L can be as large as possible, that is, as many as possible, space points and image points can be selected for each camera. As an example, L may be not less than 10.

S102. Acquire a panoramic camera imaging model, where the panoramic camera imaging model is used to represent a conversion relationship between a point on the spatial coordinate system and a point on the image coordinate system.

The panoramic camera imaging model in the present invention can be understood as the imaging model corresponding to each of the M cameras included in the panoramic camera, that is, the imaging models of the M cameras collectively form a panoramic camera imaging model. The process of establishing a camera imaging model can be found in Figure 2 below, which is not detailed here.

As an example, in the acquiring the panoramic camera imaging model in the present invention, the imaging model may be pre-established and saved, and directly read when external parameter estimation is needed, for example, the imaging model may be saved locally in the panoramic camera, or other The third party device that can communicate with the panoramic camera is not specifically limited in the present invention. In addition, in the acquisition of the panoramic camera imaging model in the present invention, it is also possible to establish an imaging model in real time when external parameter estimation is required.

It should be noted that, according to the solution of the present invention, the coordinates of the spatial point and the image point may be acquired first, and then the panoramic camera imaging model may be acquired; or the panoramic camera imaging model may be acquired first, and then the spatial point and the image are acquired. The coordinates of the point; in addition, two steps can be performed at the same time, which is not specifically limited in the present invention.

S103. Determine an external parameter of the panoramic camera by using coordinates of the spatial point, coordinates of the image point, and the panoramic camera imaging model.

It should be noted that the camera external camera plays an important role in the process of capturing the spatial point and generating the corresponding image point. Therefore, the panoramic camera imaging model of the present invention includes a camera external parameter, that is, the external parameter can be reflected by the imaging model. The role played by spatial points in the process of converting to image points.

The geometrical calibration is performed by the solution of the present invention. When the spatial point and the corresponding image point coordinates are known and the imaging model is known, the external reference of the panoramic camera can be determined. The specific process can be seen below, and will not be detailed here.

Referring to Figure 2, a flow chart of a method of establishing a panoramic camera imaging model in the present invention is shown. That is, the method of establishing an imaging model of each camera included in the panoramic camera may include the following steps:

S201, linearly transform a point on the space coordinate system to obtain a point on a lens coordinate system of the panoramic camera.

The conversion of the spatial point into the image point may include at least three conversion processes: the spatial coordinate system is converted into the lens coordinate system, the lens coordinate system is converted into the sensor coordinate system, and the sensor coordinate system is converted into the image coordinate system, which are explained below.

As an example, the conversion of the spatial coordinate system to the lens coordinate system is a linear transformation, which can be obtained by the first rotation matrix r and the three-dimensional translation vector t. Where r and t describe It is a parameter of the external scene of the camera, that is, the camera external reference in the present invention.

The conversion relationship between the space coordinate system and the lens coordinate system can be expressed as the following formula:

Where X represents a point on the space coordinate system, which can be embodied as (x _ij , y _ij , z _ij ); p _ij represents a point on the lens coordinate system, which can be embodied as (u′′ _ij , v′′ _ij , f (u" _ij , v" _ij )); P ⁱ denotes a first rotation matrix r and a three-dimensional translation vector t; λ _ij denotes a normalization parameter, i denotes an i-th camera of the panoramic camera, and j denotes a coordinate system j points.

S202, performing nonlinear transformation on a point on the lens coordinate system to obtain a point on a sensor coordinate system of the lens.

The light at the point X of the space passes through the optical center of the lens, and is refracted by the plurality of sets of lenses. The path of the light is bent, and the position of the image on the sensor is shifted. The transformation of this process is nonlinear. That is to say, the conversion of the lens coordinate system to the Sensor coordinate system is a nonlinear transformation.

As an example, a Taylor polynomial can be used to represent a generalized model of the lens projection. Specifically, the conversion relationship between the lens coordinate system and the Sensor coordinate system can be embodied as follows:

p _ij =g(u" _ij ,v" _ij )=(u" _ij ,v" _ij ,f(u" _ij ,v" _ij )) ^T (2)

Where (u" _ij , v" _ij , f ( u " _ij , v " _ij )) represent points on the lens coordinate system, (u" _ij , v" _ij ) represents points on the sensor coordinate system, and T represents Set,

The value of N in the present invention may not be specifically limited, and N will be cancelled out in the subsequent processing.

It should be noted that f(u" _ij , v" _ij ) is a nonlinear function with respect to ρ, so it can also be expressed as f(ρ), where ρ represents a point (u" _ij , v" _ij ) to the sensor The distance from the origin of the coordinates. Thus, the imaging model of the present invention allows both radial distortion and barrel distortion to be considered, which helps to improve the accuracy of the external parameter estimation of the present invention.

S203: Perform affine transformation on a point on the sensor coordinate system to obtain a point on the image coordinate system.

As an example, the coordinate system established on the virtual sensor imaging plane is in physical unit mm, and the final image coordinate system is in pixel units, and the coordinate origin positions of the two are different. Therefore, the sensor coordinate system can be converted to the image coordinate system by affine transformation.

The conversion relationship between the sensor coordinate system and the image coordinate system can be expressed as the following formula:

u′′ _ij =Au′ _ij +t ₁ ,v′′ _ij =Av′ _ij +t ₁ (3)

Where (u" _ij , v" _ij ) represents a point on the sensor coordinate system, (u' _ij , v' _ij ) represents a point on the image coordinate system, A represents a second rotation matrix, and t ₁ represents a translation matrix, A And t ₁ depends mainly on the camera chosen.

S204. The panoramic camera imaging model is established based on a point on the spatial coordinate system and a point on the image coordinate system.

Specifically, based on the above three conversion formulas, a correspondence between points on the space coordinate system and points on the image coordinate system can be obtained:

As an example, some special spatial points may be selected without loss of generality in order to obtain the imaging model of the present invention. For example, if a spatial point where z _ij is 0 is selected, the correspondence between the point on the spatial coordinate system and the point on the image coordinate system may be:

As an example, in order to facilitate the calculation of the imaging model, p _ij can be multiplied at both ends of the equation of equation (5) to make the equation 0:

That is, the correspondence between the points on the space coordinate system and the points on the image coordinate system can be:

Finally, by decomposing the formula (7), an imaging model of each camera included in the panoramic camera can be obtained. For example, the imaging model of the i-th camera can be:

f(ρ _j )·(r ₁₁ x _j +r ₁₂ y _j +t ₁ )-u' ₁ ·(r ₃₁ x _j +r ₃₂ y _j +t ₃ )=0.

In summary, an imaging model of each camera can be obtained, thereby obtaining a panoramic camera imaging model in the present invention.

As described above at S103, in the case where the spatial point coordinates, image point coordinates, and imaging model corresponding to each camera are known, the camera external reference of the present invention can be determined based on these known information. The following explains the solution process of each camera external parameter.

First, the imaging model of the camera is rewritten into a vector form: M·H=0, where

After the model is rewritten, a super-positive equation can be obtained, which can be solved by the least squares method, ie, min||M·H|| ² is solved.

After solving the values of r ₁₁ , r ₁₂ , r ₂₁ , r ₂₂ , t ₁ , t ₂ by the least squares method, it is considered that the first rotation matrix r has the following characteristics: |r ₁ r ₂ r ₃ |=1, Therefore, it is possible to continue to solve the values of r ₃₁ and r ₃₂ by using the solved r ₁₁ , r ₁₂ , r ₂₁ , and r ₂₂ . Thus, the specific values of the camera external parameters r ₁₁ , r ₁₂ , r ₂₁ , r ₂₂ , r ₃₁ , r ₃₂ , t ₁ , t ₂ can be obtained.

As an example, the SVD (English: Singular Value Decomposition) algorithm can be used to solve the system of equations by using the least squares method; or it can be implemented by the following machine learning method, which is not Make specific limits.

Compared with the SVD algorithm, solving the least squares problem based on the machine learning method can greatly reduce the amount of calculation, and the scheme of the present invention is applicable to the real-time external parameter estimation of the embedded system. This is explained below.

It should be noted that the method mainly determines a set including a plurality of iterative parameters by means of machine learning, and then gradually iteratively optimizes the iterative parameters based on the initial value of the camera, and finally determines Better camera external parameters.

As an example, the initial value of the external parameter can be set in combination with the actual operating experience; or, considering that the prior value of the camera parameter is usually a better value, the initial value of the external parameter can also be set according to the prior value, for example, The a priori value is determined as the initial value of the external reference; or, the random disturbance can be added on the basis of the prior value to obtain the initial value of the external reference. For example, the random disturbance value may be set according to the actual operation experience, or the random disturbance value may be set to ± (prior value / 100), which is not specifically limited in the present invention.

In the solution of the present invention, the relationship between the external parameter x _k-1 before iteration and the external parameter x _k after iteration can be embodied as the following iterative formula: x _k = x _k-1 + P _k-1 M·H ( x _k-1 ) + Q _k-1 , therefore, solving the least squares problem translates to solving the iterative parameters P _k-1 and Q _k-1 .

Specifically, the process of solving the iterative parameters can be divided into a learning phase and a verification phase.

Learning phase

The training samples are selected, and through the idea of supervised learning in machine learning, the following iterative parameter sets are obtained using training samples: {P ₀ , P ₁ ,..., P _k-1 , P _k } and {Q ₀ , Q ₁ , ..., Q _k-1 , Q _k }.

It should be noted that the parameters of the training sample in the present invention may be as follows: the identity code of the sample camera included in the sample panoramic camera; the spatial point coordinates and image point coordinates corresponding to each sample camera; the initial value of the external reference of each sample camera The estimated value of the external parameter of each sample camera, that is, the optimal camera external parameter.

2. Verification phase

Usually, when k=5, the iterative parameter set is {P ₀ , P ₁ , P ₂ , P ₃ , P ₄ , P ₅ } and {Q ₀ , Q ₁ , Q ₂ , Q ₃ , Q ₄ , Q ₅ }, iteratively based on the initial value of the camera external reference, can obtain the optimal external parameter estimation value of the camera. The present invention does not specifically limit the number of iterations, and may be determined in combination with actual application conditions.

Corresponding to the method described above, the embodiment of the present invention further provides a camera geometric calibration processing device. Referring to FIG. 3, the device may include:

a coordinate acquiring unit 301, configured to acquire coordinates of a spatial point and coordinates of an image point, and a panoramic phase Taking the space point to obtain the image point corresponding to the space point, the space point is a point on a space coordinate system, and the image point is a point on an image coordinate system;

An imaging model acquiring unit 302, configured to acquire a panoramic camera imaging model, where the panoramic camera imaging model is used to represent a conversion relationship between a point on the spatial coordinate system and a point on the image coordinate system;

The external parameter determining unit 303 is configured to determine an external parameter of the panoramic camera by using coordinates of the spatial point, coordinates of the image point, and the panoramic camera imaging model.

Optionally, the imaging model acquiring unit includes:

Optionally,

The linear transformation unit is configured to convert a point on the spatial coordinate system to a point on the lens coordinate system by using the following formula:

g(u" _ij ,v" _ij )=(u" _ij ,v" _ij ,f(u" _ij ,v" _ij )) ^T ;

u" _ij =Au' _ij +t ₁ ,v" _ij =Av' _ij +t ₁

Where (x _ij , y _ij , z _ij ) represents a point on the space coordinate system, and (u′′ _ij , v′′ _ij , f(u′′ _ij , v′′ _ij )) represent points on the lens coordinate system,

Optionally, an imaging model establishing unit is configured to obtain an imaging model of the i-th camera of the panoramic camera:

It should be noted that each embodiment in the specification is described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same similar parts between the embodiments are referred to each other. can. For the device type embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

Finally, it is also to be understood that the term "comprises", "comprising" or any other variants thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device comprising a plurality of elements includes Those elements, but also other elements not explicitly listed, or elements that are inherent to such a process, method, item or equipment. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The foregoing provides a detailed description of the solution provided by the present invention. The principles and embodiments of the present invention are described herein by using specific examples. The description of the above embodiments is only for helping to understand the method and the core idea of the present invention; The present invention is not limited by the scope of the present invention, and the details of the present invention are not limited by the scope of the present invention.

Claims

A camera geometric calibration processing method, characterized in that the method comprises:

Obtaining coordinates of the spatial point and coordinates of the image point, and the panoramic camera captures the spatial point, and obtains the image point corresponding to the spatial point, where the spatial point is a point on a spatial coordinate system, and the image point is a point on the image coordinate system;

Obtaining a panoramic camera imaging model for representing a conversion relationship between a point on the spatial coordinate system and a point on the image coordinate system;

The outer parameters of the panoramic camera are determined using the coordinates of the spatial point, the coordinates of the image point, and the panoramic camera imaging model.
The method of claim 1 wherein said acquiring a panoramic camera imaging model comprises:

Performing a linear transformation on a point on the spatial coordinate system to obtain a point on a lens coordinate system of the panoramic camera;

Performing a nonlinear transformation on a point on the lens coordinate system to obtain a point on the sensor coordinate system of the lens;

Performing an affine transformation on a point on the sensor coordinate system to obtain a point on the image coordinate system;

The panoramic camera imaging model is established based on points on the spatial coordinate system and points on the image coordinate system.
The method of claim 2, wherein the points on the spatial coordinate system are converted to points on the lens coordinate system by the following formula:

Where (x ij , y ij , z ij ) represents a point on the spatial coordinate system, and (u′′ ij , v′′ ij , f(u′′ ij , v′′ ij )) represent points on the lens coordinate system,

λ ij denotes a normalization parameter, P i denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, and j denotes a j-th point on the coordinate system.
The method of claim 2 wherein the points on the lens coordinate system are converted to points on the sensor coordinate system by the following formula:

g(u" ij ,v" ij )=(u" ij ,v" ij ,f(u" ij ,v" ij )) T

Where (u" ij , v" ij , f ( u " ij , v " ij )) represent points on the lens coordinate system, (u" ij , v" ij ) represents points on the sensor coordinate system, and T represents Let i denote the i-th camera of the panoramic camera and j denote the j-th point on the coordinate system.
The method of claim 2 wherein the points on the sensor coordinate system are converted to points on the image coordinate system by the following formula:

u" ij =Au' ij +t 1 ,v" ij =Av' ij +t 1

Where (u" ij , v" ij ) represents a point on the sensor coordinate system, (u' ij , v' ij ) represents a point on the image coordinate system, i represents the ith camera of the panoramic camera, and j represents the coordinate system On the jth point, A represents the second rotation matrix, and t 1 represents the translation matrix.
The method according to claim 2, wherein the panoramic camera imaging model is established based on a point on the spatial coordinate system and a point on the image coordinate system, including:

Obtaining a correspondence between a point on the spatial coordinate system and a point on the image coordinate system:

Where (x ij , y ij , z ij ) represents a point on the spatial coordinate system, and (u′′ ij , v′′ ij , f(u′′ ij , v′′ ij )) represent points on the lens coordinate system,

(u' ij , v' ij ) denotes a point on the image coordinate system, λ ij denotes a normalization parameter, P i denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, j Representing the jth point on the coordinate system, A represents the second rotation matrix, and t 1 represents the translation matrix;

If z ij is 0, the correspondence between the point on the space coordinate system and the point on the image coordinate system is:

If both ends of the equation are multiplied by p ij at the same time, the correspondence between the point on the space coordinate system and the point on the image coordinate system is:

The corresponding relation obtained by the decomposition fork is obtained, and the imaging model of the i-th camera of the panoramic camera is obtained:

v' j ·(r 31 x j +r 32 y j +t 3 )-f(ρ j )·(r 21 x j +r 22 y j +t 2 )=0

f(ρ j )·(r 11 x j +r 12 y j +t 1 )-u' j ·(r 31 x j +r 32 y j +t 3 )=0.

u' j ·(r 21 x j +r 22 y j +t 2 )-v' j ·(r 11 x j +r 12 y j +t 1 )=0
A camera geometric calibration processing device, characterized in that the device comprises:

a coordinate acquiring unit, configured to acquire coordinates of the spatial point and coordinates of the image point, and the panoramic camera captures the spatial point, and obtains the image point corresponding to the spatial point, where the spatial point is a point on the spatial coordinate system The image point is a point on the image coordinate system;

An imaging model acquisition unit, configured to acquire a panoramic camera imaging model, the panoramic camera imaging model is configured to represent a conversion relationship between a point on the spatial coordinate system and a point on the image coordinate system;

The outer parameter determining unit is configured to determine an outer parameter of the panoramic camera by using coordinates of the spatial point, coordinates of the image point, and the panoramic camera imaging model.
The apparatus according to claim 7, wherein the imaging model acquisition unit comprises:

a linear transformation unit configured to linearly transform a point on the spatial coordinate system to obtain a point on a lens coordinate system of the panoramic camera;

a nonlinear transform unit configured to perform nonlinear transformation on a point on the lens coordinate system to obtain a point on a sensor coordinate system of the lens;

An affine transformation unit, configured to perform affine transformation on a point on the sensor coordinate system to obtain a point on the image coordinate system;

An imaging model establishing unit configured to establish the panoramic camera imaging model based on a point on the spatial coordinate system and a point on the image coordinate system.
The device of claim 8 wherein:

The linear transformation unit is configured to convert a point on the spatial coordinate system to a point on the lens coordinate system by using the following formula:

The nonlinear transformation unit is configured to convert a point on the lens coordinate system to a point on the sensor coordinate system by using the following formula:

g(u" ij ,v" ij )=(u" ij ,v" ij ,f(u" ij ,v" ij )) T ;

The affine transformation unit is configured to convert a point on the sensor coordinate system to a point on the image coordinate system by using the following formula:

u" ij =Au' ij +t 1 ,v" ij =Av' ij +t 1

Where (x ij , y ij , z ij ) represents a point on the spatial coordinate system, and (u′′ ij , v′′ ij , f(u′′ ij , v′′ ij )) represent points on the lens coordinate system,

λ ij denotes a normalization parameter, P i denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, and j denotes a j-th point on the coordinate system, (u′′ ij , v′′ Ij ) represents a point on the sensor coordinate system, T represents a transpose, (u′ ij , v′ ij ) represents a point on the image coordinate system, A represents a second rotation matrix, and t 1 represents a translation matrix.
The apparatus according to claim 8, wherein the imaging model establishing unit is configured to obtain an imaging model of the i-th camera of the panoramic camera:

Obtaining a correspondence between a point on the spatial coordinate system and a point on the image coordinate system:

Where (x ij , y ij , z ij ) represents a point on the spatial coordinate system, and (u′′ ij , v′′ ij , f(u′′ ij , v′′ ij )) represent points on the lens coordinate system,

(u' ij , v' ij ) denotes a point on the image coordinate system, λ ij denotes a normalization parameter, P i denotes a first rotation matrix r and a three-dimensional translation vector t, i denotes an i-th camera of the panoramic camera, j Representing the jth point on the coordinate system, A represents the second rotation matrix, and t 1 represents the translation matrix;

If z ij is 0, the correspondence between the point on the space coordinate system and the point on the image coordinate system is:

If both ends of the equation are multiplied by p ij at the same time, the correspondence between the point on the space coordinate system and the point on the image coordinate system is:

The corresponding relation obtained by the decomposition fork is obtained, and the imaging model of the i-th camera of the panoramic camera is obtained:

v' j ·(r 31 x j +r 32 y j +t 3 )-f(ρ j )·(r 21 x j +r 22 y j +t 2 )=0

f(ρ j )·(r 11 x j +r 12 y j +t 1 )-u' j ·(r 31 x j +r 32 y j +t 3 )=0.

u' j ·(r 21 x j +r 22 y j +t 2 )-v' j ·(r 11 x j +r 112 y j +t 1 )=0