CN103644897A - Three-dimensional surveying method based on super-resolution image reconstruction - Google Patents

Abstract

The invention relates to a three-dimensional surveying method based on super-resolution image reconstruction. The three-dimensional surveying method comprises the steps of generating a super-resolution three-dimensional image according to three-dimensional images shot by multiple stations; checking a three-dimensional camera by using a three-dimensional checking field, establishing an east-north-heaven right hand coordinate system by using a left image of a first station as a reference, performing multiple-base line bundle balancing on the basis of checking the relative gesture of the three-dimensional camera, and determining elements of orientation of all three-dimensional images; regarding the super-resolution image being shot by a virtual camera, checking the virtual three-dimensional camera by using three-dimensional coordinates obtained through forward intersection of multiple-base line three-dimensional images as true values; obtaining re-checking affine transformation coefficient by using a least square method according internal and external parameters obtained by the virtual three-dimensional camera through checking; and performing forward intersection on image point coordinates of a target to be surveyed, the internal and external parameters of the virtual three-dimensional camera and the re-checking coefficient to obtain three-dimensional coordinates of the target to be surveyed. The three-dimensional surveying method can be widely applied to three-dimensional surveying of super-resolution images of terrestrial photogrammetric survey.

Description

A kind of three-dimensional measuring method based on super-resolution image reconstruction

Technical field

The present invention relates to a kind of three-dimensional measuring method of image reconstruction, particularly about a kind of three-dimensional measuring method based on super-resolution image reconstruction.

Background technology

In photogrammetric application, high-resolution image is localizing objects accurately.For reducing CCD(Charge-coupled Device, charge coupled cell) constraint of resolution, can use super-resolution (Super Resolution, SR) reconstruction technique to obtain high-resolution image.Technique of Super-resolution Image Construction be utilize about Same Scene have Displacement and containing multiframe low resolution (the Low Resolution of redundant information, LR) image reconstruction goes out high resolving power (High Resolution, HR) technology of image, its ultimate principle is to utilize nonredundancy information that the fine difference between sequence of low resolution pictures frame and frame comprises to improve the resolution of image.In close-range photogrammetry, can adopt many Baselines to improve the three-dimensional measuring precision of image, but high-resolution image can not be provided.

Summary of the invention

For the problems referred to above, the object of this invention is to provide a kind ofly can provide high resolution image and the high three-dimensional measuring method based on super-resolution image reconstruction of three-dimensional measuring precision.

For achieving the above object, the present invention takes following technical scheme: a kind of three-dimensional measuring method based on super-resolution image reconstruction, it comprises the following steps: 1) at four, take the photograph station and use respectively stereoscopic camera to treat to measure target and photograph, obtain four couples of stereopsis (L of target to be measured ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄); To four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) carry out obtaining after Image registration and image restoration the super-resolution stereopsis (SR_L, SR_R) of target to be measured; 2) use stereoscopic camera to take three-dimensional calibration field, according to the stereopsis (Calib_L, Calib_R) of three-dimensional calibration field, calculate the relative position T (V between the camera of left and right in the elements of interior orientation of stereoscopic camera and stereoscopic camera _x, V _y, V _z) and attitude R; 3) with the left image L of target first stop to be measured ₁center be initial point O, the left image L of first stop ₁east to being X-axis,, to being Y-axis, take and point into the sky direction as Z axis perpendicular to XOY plane in its north, sets up O-XYZ coordinate system, under O-XYZ coordinate system, the left image L of first stop ₁position, outer orientation and outer orientation attitude be respectively (0,0,0) and unit matrix; According to the left image L of first stop ₁position, outer orientation and the relative position T (V between the left and right camera of outer orientation attitude and stereoscopic camera _x, V _y, V _z) and attitude R, obtain the right image R of first stop ₁position, outer orientation and outer orientation attitude be respectively T (V _x, V _y, V _z) and R; By the left image L of first stop ₁position, outer orientation and outer orientation attitude and the right image R of first stop ₁position, outer orientation and outer orientation attitude obtain first stop stereopsis (L ₁, R ₁) position, outer orientation and outer orientation attitude; 4) according to first stop stereopsis (L ₁, R ₁) position, outer orientation and outer orientation attitude and four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on unique point of the same name, calculate stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude; 5) the super-resolution stereopsis (SR_L, SR_R) of target to be measured is considered as by virtual three-dimensional camera, take and obtain, at the upper selected equally distributed reference mark of super-resolution stereopsis (SR_L, SR_R); To be positioned at four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) upper and corresponding with the upper reference mark of super-resolution stereopsis (SR_L, SR_R) corresponding image points coordinate and stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃), (L ₄, R ₄) position, revised outer orientation and attitude substitution collinearity equation carry out many baselines intersection calculation, obtains the three-dimensional coordinate at the upper reference mark of super-resolution stereopsis (SR_L, SR_R); Utilize three-dimensional coordinate and the upper corresponding picpointed coordinate of super-resolution stereopsis (SR_L, SR_R) at these reference mark to carry out calibration to virtual three-dimensional camera, obtain elements of interior orientation, position, outer orientation and the outer orientation attitude of virtual three-dimensional camera; 6) at super-resolution stereopsis (SR_L, SR_R) three above reference mark of upper selection, utilize the three-dimensional coordinate at reference mark and the elements of interior orientation of virtual three-dimensional camera, position, outer orientation and outer orientation attitude respectively left image SR_L and right image SR_R to be set up to the affine Transform Model of calibration again, and utilize least square method to calculate the coefficient of calibration again of affine Transform Model; 7) by the coefficient of the calibration again a of the affine Transform Model of left image SR_L and right image SR_R in elements of interior orientation, position, outer orientation and the outer orientation attitude of the virtual three-dimensional camera calibration being obtained by step 5), the super-resolution stereopsis (SR_L, SR_R) that obtained by step 6) ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂and carry out forward intersection computing in the coordinate substitution collinearity equation for the treatment of gauge point on super-resolution stereopsis (SR_L, SR_R), obtain the three-dimensional coordinate of target to be measured.

Described step 2) in, the relative position T (V in the elements of interior orientation of acquisition stereoscopic camera and stereoscopic camera between the camera of left and right _x, V _y, V _z) and attitude R, it comprises the following steps: 1. use stereoscopic camera to take three-dimensional calibration field, obtain the stereopsis (Calib_L of three-dimensional calibration field, Calib_R), utilize characteristic point matching method to obtain the picpointed coordinate (x at the middle reference mark of stereopsis (Calib_L, Calib_R) of three-dimensional calibration field _tL, y _tL) and (x _iR, y _iR), t > 6; 2. by the known three-dimensional coordinate (X in reference mark in the stereopsis (Calib_L, Calib_R) of three-dimensional calibration field _t, Y _t, Z _t) and the picpointed coordinate (x at reference mark _tL, y _tL) and (x _tR, y _tR) substitution collinearity equation, by the picpointed coordinate (x at reference mark _tL, y _tL) and (x _tR, y _tR) distortion model of substitution pinhole imaging system, and by the distortion model of collinearity equation and pinhole imaging system, the two result after processing is carried out the compensating computation of entire light method, obtains position, the outer orientation T of the elements of interior orientation of stereoscopic camera, left image Calib_L _{calib_L}(X _ls, Y _ls, Z _ls) and attitude R _{calib_L}and position, the outer orientation T of right image Calib_R _{calib_R}(X _rs, Y _rs, Z _rs) and attitude R _{calib_R}; 3. by position, the outer orientation T of left image Calib_L _{calib_L}(X _ls, Y _ls, Z _ls) and attitude R _{calib_L}, right image Calib_R position, outer orientation T _{calib_R}(X _rs, Y _rs, Z _rs) and attitude R _{calib_R}substitution formula

$\left\{\begin{array}{c}R=R_{\mathrm{Calib}\_L}^{-1}\·R_{\mathrm{Calib}\_R}\\ T=R_{\mathrm{Calib}\_L}^{-1}\·(T_{\mathrm{Calib}\_R}-T_{\mathrm{Calib}\_L})\end{array}\right.,$

Calculate the relative position T (V between the camera of left and right in stereoscopic camera _x, V _y, V _z) and attitude R.

In described step 4), obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude, it specifically comprises the following steps: four couples of stereopsis (L that 1. extract target to be measured ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on unique point of the same name, obtain unique point at stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on picpointed coordinate; 2. by the elements of interior orientation of stereoscopic camera, first stop stereopsis (L ₁, R ₁) picpointed coordinate and the first stop stereopsis (L of upper unique point ₁, R ₁) position, outer orientation and outer orientation attitude substitution collinearity equation carry out forward intersection computing, obtain first stop stereopsis (L ₁, R ₁) three-dimensional coordinate of upper some unique points; 3. to first stop stereopsis (L ₁, R ₁) three-dimensional coordinate and the left image L of upper some unique points ₂, L ₃and L ₄on the picpointed coordinate of unique point carry out photogrammetric resection computing, obtain successively left image L ₂, L ₃and L ₄position, outer orientation with outer orientation attitude

initial value, i=2,3,4; 4. according to left image L ₂, L ₃and L ₄position, outer orientation with outer orientation attitude

initial value and the relative position T (V between the camera of left and right _x, V _y, V _z) and attitude R, obtain right image R ₂, R ₃and R ₄position, outer orientation

with outer orientation attitude initial value be respectively:

$\left\{\begin{array}{c}{T}_i^{\mathrm{right}}=R_i^{\mathrm{left}}\·T+T_i^{\mathrm{left}}\\ R_i^{\mathrm{right}}=R_i^{\mathrm{left}}\·R\end{array}\right.;$

By left image L ₂, L ₃and L ₄position, outer orientation

with outer orientation attitude

and right image R ₂, R ₃and R ₄position, outer orientation

with outer orientation attitude obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the initial value of position, outer orientation

initial value with outer orientation attitude 5. adopt many baselines bundle adjustment method to stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the initial value of position, outer orientation

initial value with outer orientation attitude

revise, obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude.

In described step 6), to super-resolution stereopsis (SR_L, SR_R) in, left image SR_L and right image SR_R set up the process of affine Transform Model, and it comprises the following steps: 1. in the upper selection of super-resolution stereopsis (SR_L, SR_R) and at stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the upper corresponding unique point of unique point of the same name of extracting, and using these unique points of the same name as reference mark, utilize the forward intersection of many baselines to calculate the three-dimensional coordinate at these reference mark; 2. by super-resolution stereopsis (SR_L, SR_R) the upper three-dimensional coordinate at reference mark and the position, outer orientation of virtual three-dimensional camera and outer orientation attitude substitution collinearity equation, calculate coordinate under the photo coordinate system oxy of reference mark on super-resolution stereopsis (SR_L, SR_R) (x ' _l, y ' _l) and (x' _r, y' _r); 3. by the elements of interior orientation of virtual three-dimensional camera, reference mark the coordinate under the photo coordinate system oxy on super-resolution stereopsis (SR_L, SR_R) (x ' _l, y ' _l) and (x' _r, y' _r) the camera distortion model of substitution pinhole imaging system, use Newton iteration method to calculate the coordinate (m under the collimation mark coordinate system o'mn of reference mark on super-resolution stereopsis (SR_L, SR_R) _l, n _l) and (m _r, n _r); 4. according to reference mark the measurement coordinate under the collimation mark coordinate system o'mn on (SR_L, SR_R) image (m ' _l, n ' _l) and (m' _r, n' _r) and the collimation mark coordinate system o'mn of the reference mark that calculates in 3. of step on super-resolution stereopsis (SR_L, SR_R) under coordinate (m _l, n _l) and (m _r, n _r), left image SR_L in super-resolution stereopsis (SR_L, SR_R) and right image SR_R are set up respectively to affine Transform Model:

$\left\{\begin{array}{c}{m}_l^{\′}-m_l=a_0+a_1{m}_l+a_2{n}_l\\ n_l^{\′}-n_l=b_0+b_1{m}_l+b_2{n}_l\end{array}\right.,$

$\left\{\begin{array}{c}{m}_r^{\′}-m_r=c_0+c_1{m}_r+c_2{n}_r\\ n_r^{\′}-n_r=d_0+d_1{m}_r+d_2{n}_r\end{array}\right.,$

In formula, a ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂it is respectively the coefficient of calibration again of left image SR_L and right image SR_R in super-resolution stereopsis (SR_L, SR_R).

In described step 7), obtain the three-dimensional coordinate of target to be measured, it specifically comprises the following steps: 1. according to the coefficient of the calibration again a of the affine Transform Model of left image SR_L and right image SR_R in super-resolution stereopsis (SR_L, SR_R) ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂and the picpointed coordinate that measures under collimation mark coordinate system o'mn of left image SR_L and right image SR_R with

calculate the coordinate after correcting under measuring the collimation mark coordinate system o'mn of target at left image SR_L and right image SR_R

with

be respectively:

$\left\{\begin{array}{c}{x}_{\mathrm{lrow}}^{\′\mathrm{Tar}}=x_{\mathrm{lrow}}^{\mathrm{Tar}}+a_0+a_1{x}_{\mathrm{lrow}}^{\mathrm{Tar}}+a_2{y}_{\mathrm{lcol}}^{\mathrm{Tar}}\\ y_{\mathrm{lcol}}^{\′\mathrm{Tar}}=y_{\mathrm{lcol}}^{\mathrm{Tar}}+b_0+b_1{x}_{\mathrm{lrow}}^{\mathrm{Tar}}+b_2{y}_{\mathrm{lcol}}^{\mathrm{Tar}}\end{array}\right.,$

$\left\{\begin{array}{c}{x}_{\mathrm{rrow}}^{\′\mathrm{Tar}}=x_{\mathrm{rrow}}^{\mathrm{Tar}}+c_0+c_1{x}_{\mathrm{rrow}}^{\mathrm{Tar}}+c_2{y}_{\mathrm{rcol}}^{\mathrm{Tar}}\\ y_{\mathrm{rcol}}^{\′\mathrm{Tar}}=y_{\mathrm{rcol}}^{\mathrm{Tar}}+d_0+d_1{x}_{\mathrm{rrow}}^{\mathrm{Tar}}+d_2{y}_{\mathrm{rcol}}^{\mathrm{Tar}}\end{array}\right.;$

2. use the elements of interior orientation of pinhole imaging system distortion model and the calibration of virtual three-dimensional camera, respectively by the coordinate after the correction under left image SR_L collimation mark coordinate system o'mn until gauge point with the coordinate after correction under right image SR_R collimation mark coordinate system o'mn

be converted to the point coordinate under left image SR_L photo coordinate system oxy with the point coordinate under right image SR_R photo coordinate system oxy

3. by the picpointed coordinate of the position, outer orientation of virtual three-dimensional camera calibration and outer orientation attitude and the left image SR_L of super-resolution stereopsis (SR_L, SR_R)

picpointed coordinate with right image SR_R carry out forward intersection computing, obtain the three-dimensional coordinate of target to be measured under O-XYZ coordinate system.

The present invention is owing to taking above technical scheme, it has the following advantages: 1, the present invention is owing to using three-dimensional calibration field to carry out calibration to stereoscopic camera, on the basis of calibration stereoscopic camera relative attitude, carry out many baselines bundle adjustment, determine the element of orientation of every width stereopsis of target to be measured, the stereopsis of taking according to multistation generates super-resolution stereopsis, super-resolution image is considered as to a pair of virtual camera and takes acquisition, the three-dimensional coordinate that the multi-base stereo image forward intersection of take obtains is true value, virtual three-dimensional camera is carried out to calibration, by the picpointed coordinate of target to be measured, the intrinsic parameter of virtual three-dimensional camera, outer parameter and again calibration coefficient carry out forward intersection computing, obtain the three-dimensional coordinate of target to be measured, therefore the present invention can guarantee that three-dimensional measuring has higher precision, and can provide high resolution image 2, the present invention is owing to using the unique point obtaining from multi-base stereo image as the elements of interior orientation and the elements of exterior orientation that obtain super-resolution stereopsis with reference to reference mark, therefore the present invention does not need extra measurement control point coordinate, be convenient to practical application.Based on above advantage, the present invention can be widely used in the three-dimensional measuring of terrestrial photogrammetry super-resolution image.

Accompanying drawing explanation

Fig. 1 is the three-dimensional measuring method flow schematic diagram of super-resolution image reconstruction of the present invention;

Fig. 2 is the conceptual schematic view of super-resolution image reconstruction of the present invention; Wherein, O-XYZ coordinate system represents east-north-sky right-handed coordinate system, and SR_L and SR_R represent respectively left image and the right image of the super-resolution stereopsis (SR_L, SR_R) of target to be measured; L ₁, L ₂, L ₃and L ₄the left image that represents respectively target first stop to be measured, second station, the 3rd station and the 4th stand body image, R ₁, R ₂, R ₃and R ₄the right image that represents respectively target first stop to be measured, second station, the 3rd station and the 4th stand body image.

Embodiment

Below in conjunction with drawings and Examples, the present invention is described in detail.

As shown in Figure 1, the three-dimensional measuring method based on super-resolution image reconstruction of the present invention, it comprises the following steps:

1) as shown in Figure 2, at four, take the photograph station and use respectively stereoscopic camera to treat to measure target and photograph, obtain four couples of stereopsis (L of target to be measured ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄); To four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) carry out obtaining after Image registration and image restoration the super-resolution stereopsis (SR_L, SR_R) of target to be measured.

2) use stereoscopic camera to take three-dimensional calibration field, according to the stereopsis (Calib_L, Calib_R) of three-dimensional calibration field, calculate the relative position T (V between the camera of left and right in the elements of interior orientation of stereoscopic camera and stereoscopic camera _x, V _y, V _z) and attitude R, it comprises the following steps:

1. use stereoscopic camera to take three-dimensional calibration field, obtain the stereopsis (Calib_L of three-dimensional calibration field, Calib_R), utilize characteristic point matching method to obtain the picpointed coordinate (x at the middle reference mark of stereopsis (Calib_L, Calib_R) of three-dimensional calibration field _tL, y _tL) and (x _tR, y _tR), wherein (t > 6).

2. by the known three-dimensional coordinate (X in reference mark in the stereopsis (Calib_L, Calib_R) of three-dimensional calibration field _t, Y _t, Z _t) and the picpointed coordinate (x at reference mark _tL, y _tL) and (x _tR, y _tR) substitution collinearity equation, by the picpointed coordinate (x at reference mark _tL, y _tL) and (x _tR, y _tR) distortion model of substitution pinhole imaging system, and by the distortion model of collinearity equation and pinhole imaging system, the two result after processing is carried out the compensating computation of entire light method, obtains position, the outer orientation T of the elements of interior orientation of stereoscopic camera, left image Calib_L _{calib_L}(X _ls, Y _ls, Z _ls) and attitude R _{calib_L}and position, the outer orientation T of right image Calib_R _{calib_R}(X _rs, Y _rs, Z _rs) and attitude R _{calib_R}

3. by position, the outer orientation T of left image Calib_L _{calib_L}(X _ls, Y _ls, Z _ls) and attitude R _{calib_L}, right image Calib_R position, outer orientation T _{calib_R}(X _rs, Y _rs, Z _rs) and attitude R _{calib_R}substitution following formula:

$\left\{\begin{array}{c}R=R_{\mathrm{Calib}\_L}^{-1}\·R_{\mathrm{Calib}\_R}\\ T=R_{\mathrm{Calib}\_L}^{-1}\·(T_{\mathrm{Calib}\_R}-T_{\mathrm{Calib}\_L})\end{array}\right.---\left(1\right)$

Calculate the relative position T (V between the camera of left and right in stereoscopic camera _x, V _y, V _z) and attitude R.

3) with the left image L of first stop in four pairs of stereopsis of target to be measured ₁center be initial point O, the left image L of first stop ₁east to being X-axis,, to being Y-axis, take and point into the sky direction as Z axis perpendicular to XOY plane in its north, sets up east-north-day right-handed coordinate system (being O-XYZ coordinate system); Under O-XYZ coordinate system, the left image L of first stop ₁position, outer orientation and outer orientation attitude be respectively (0,0,0) and unit matrix; According to the left image L of first stop ₁position, outer orientation and outer orientation attitude and stereoscopic camera in relative position T (V between the camera of left and right _x, V _y, V _z) and attitude R, obtain the right image R of first stop ₁position, outer orientation and outer orientation attitude be respectively T (V _x, V _y, V _z) and R.

By the left image L of first stop ₁position, outer orientation and outer orientation attitude and the right image R of first stop ₁position, outer orientation and outer orientation attitude obtain first stop stereopsis (L ₁, R ₁) position, outer orientation and outer orientation attitude.

4) according to first stop stereopsis (L ₁, R ₁) position, outer orientation and outer orientation attitude and four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on unique point of the same name, calculate stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude, it specifically comprises the following steps:

1. extract four couples of stereopsis (L of target to be measured ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on unique point of the same name, obtain unique point at stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on picpointed coordinate.

2. by step 2) in the stereoscopic camera elements of interior orientation, the first stop stereopsis (L that obtain ₁, R ₁) picpointed coordinate and the first stop stereopsis (L of upper unique point ₁, R ₁) position, outer orientation and outer orientation attitude substitution collinearity equation carry out forward intersection computing, obtain first stop stereopsis (L ₁, R ₁) three-dimensional coordinate of upper some unique points.

3. to first stop stereopsis (L ₁, R ₁) three-dimensional coordinate and the left image L of upper some unique points ₂, L ₃and L ₄on the picpointed coordinate of unique point carry out photogrammetric resection computing, obtain successively left image L ₂, L ₃and L ₄position, outer orientation

(i=2,3,4) and outer orientation attitude

initial value.

4. according to left image L ₂, L ₃and L ₄position, outer orientation

with outer orientation attitude

initial value and the relative position T (V between the camera of left and right _x, V _y, V _z) and attitude R, obtain right image R ₂, R ₃and R ₄position, outer orientation

with outer orientation attitude

initial value be respectively:

$\left\{\begin{array}{c}{T}_i^{\mathrm{right}}=R_i^{\mathrm{left}}\·T+T_i^{\mathrm{left}}\\ R_i^{\mathrm{right}}=R_i^{\mathrm{left}}\·R\end{array}\right.---\left(2\right)$

By left image L ₂, L ₃and L ₄position, outer orientation

with outer orientation attitude

and right image R ₂, R ₃and R ₄position, outer orientation

with outer orientation attitude obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the initial value of position, outer orientation

initial value with outer orientation attitude

5. for to improve the position, outer orientation of target stereopsis to be measured and the precision of outer orientation attitude, adopt many baselines bundle adjustment method to stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the initial value of position, outer orientation

initial value with outer orientation attitude

revise, obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude.

5) the super-resolution stereopsis (SR_L, SR_R) of target to be measured is considered as by virtual three-dimensional camera, take and obtain, at the upper selected equally distributed reference mark of super-resolution stereopsis (SR_L, SR_R);

To be positioned at four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) upper and corresponding with the upper reference mark of super-resolution stereopsis (SR_L, SR_R) corresponding image points coordinate and stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃), (L ₄, R ₄) position, outer orientation and attitude respectively substitution collinearity equation carry out many baselines intersection calculation, obtain the three-dimensional coordinate at the upper reference mark of super-resolution stereopsis (SR_L, SR_R);

Utilize three-dimensional coordinate and the upper corresponding picpointed coordinate of super-resolution stereopsis (SR_L, SR_R) at these reference mark to carry out calibration to virtual three-dimensional camera, obtain elements of interior orientation, position, outer orientation and the outer orientation attitude of virtual three-dimensional camera.

6) at super-resolution stereopsis (SR_L, SR_R) three above reference mark of upper selection, utilize the three-dimensional coordinate at reference mark and the elements of interior orientation of virtual three-dimensional camera, position, outer orientation and outer orientation attitude respectively left image SR_L and right image SR_R to be set up to the affine Transform Model of calibration again, and utilize least square method to calculate the coefficient of calibration again of affine Transform Model, it specifically comprises the following steps:

1. stereopsis (the L extracting in the upper selection of super-resolution stereopsis (SR_L, SR_R) and step 4) ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on the corresponding unique point of unique point of the same name, and using these unique points of the same name as reference mark, utilize the forward intersection of many baselines to calculate the three-dimensional coordinate at these reference mark.

2. by super-resolution stereopsis (SR_L, SR_R) the upper three-dimensional coordinate at reference mark and the position, outer orientation of virtual three-dimensional camera and outer orientation attitude substitution collinearity equation, calculate coordinate under the photo coordinate system oxy of reference mark on super-resolution stereopsis (SR_L, SR_R) (x ' _l, y ' _l) and (x' _r, y' _r).

3. by the elements of interior orientation of virtual three-dimensional camera, reference mark the coordinate under the photo coordinate system oxy on super-resolution stereopsis (SR_L, SR_R) (x ' _l, y ' _l) and (x' _r, y' _r) the camera distortion model of substitution pinhole imaging system, use Newton iteration method to calculate the coordinate (m under the collimation mark coordinate system o'mn of reference mark on super-resolution stereopsis (SR_L, SR_R) _l, n _l) and (m _r, n _r).

4. according to reference mark the measurement coordinate under the collimation mark coordinate system o'mn on (SR_L, SR_R) image (m ' _l, n ' _l) and (m' _r, n' _r) and the collimation mark coordinate system o'mn of the reference mark that calculates in 3. of step on super-resolution stereopsis (SR_L, SR_R) under coordinate (m _l, n _l) and (m _r, n _r), left image SR_L in super-resolution stereopsis (SR_L, SR_R) and right image SR_R are set up respectively to affine Transform Model:

$\left\{\begin{array}{c}{m}_l^{\′}-m_l=a_0+a_1{m}_l+a_2{n}_l\\ n_l^{\′}-n_l=b_0+b_1{m}_l+b_2{n}_l\end{array}\right.---\left(3\right)$

$\left\{\begin{array}{c}{m}_r^{\′}-m_r=c_0+c_1{m}_r+c_2{n}_r\\ n_r^{\′}-n_r=d_0+d_1{m}_r+d_2{n}_r\end{array}\right.---\left(4\right)$

In formula (3), a ₀, a ₁, a ₂, b ₀, b ₁, b ₂it is the coefficient of calibration again of left image SR_L in super-resolution stereopsis (SR_L, SR_R); In formula (4), c ₀, c ₁, c ₂, d ₀, d ₁, d ₂it is the coefficient of calibration again of right image SR_R in super-resolution stereopsis (SR_L, SR_R).

5. utilize least square method to calculate the coefficient of the calibration again a of the affine Transform Model of left image SR_L and right image SR_R in super-resolution stereopsis (SR_L, SR_R) ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂.

7) by the coefficient of the calibration again a of the affine Transform Model of left image SR_L and right image SR_L in elements of interior orientation, position, outer orientation and the outer orientation attitude of the virtual three-dimensional camera calibration being obtained by step 5), the super-resolution stereopsis (SR_L, SR_R) that obtained by step 6) ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂and carry out forward intersection computing in the coordinate substitution collinearity equation for the treatment of gauge point on super-resolution stereopsis (SR_L, SR_R), and obtaining the three-dimensional coordinate of target to be measured, it specifically comprises the following steps:

1. according to the coefficient of the calibration again a of the affine Transform Model of left image SR_L in the super-resolution stereopsis (SR_L, SR_R) being obtained by step 5) and right image SR_R ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂and the picpointed coordinate that in super-resolution stereopsis (SR_L, SR_R), left image SR_L measures under collimation mark coordinate system o'mn the picpointed coordinate that right image SR_R measures under collimation mark coordinate system o'mn

calculate the coordinate after the correction under the collimation mark coordinate system o'mn that measures target left image SR_L in super-resolution stereopsis with the coordinate after correction under the collimation mark coordinate system o'mn of right image SR_R in super-resolution stereopsis (SR_L, SR_R)

for:

$\left\{\begin{array}{c}{x}_{\mathrm{lrow}}^{\′\mathrm{Tar}}=x_{\mathrm{lrow}}^{\mathrm{Tar}}+a_0+a_1{x}_{\mathrm{lrow}}^{\mathrm{Tar}}+a_2{y}_{\mathrm{lcol}}^{\mathrm{Tar}}\\ y_{\mathrm{lcol}}^{\′\mathrm{Tar}}=y_{\mathrm{lcol}}^{\mathrm{Tar}}+b_0+b_1{x}_{\mathrm{lrow}}^{\mathrm{Tar}}+b_2{y}_{\mathrm{lcol}}^{\mathrm{Tar}}\end{array}\right.---\left(5\right)$

$\left\{\begin{array}{c}{x}_{\mathrm{rrow}}^{\′\mathrm{Tar}}=x_{\mathrm{rrow}}^{\mathrm{Tar}}+c_0+c_1{x}_{\mathrm{rrow}}^{\mathrm{Tar}}+c_2{y}_{\mathrm{rcol}}^{\mathrm{Tar}}\\ y_{\mathrm{rcol}}^{\′\mathrm{Tar}}=y_{\mathrm{rcol}}^{\mathrm{Tar}}+d_0+d_1{x}_{\mathrm{rrow}}^{\mathrm{Tar}}+d_2{y}_{\mathrm{rcol}}^{\mathrm{Tar}}\end{array}\right.---\left(6\right)$

2. use the elements of interior orientation of pinhole imaging system distortion model and the calibration of virtual three-dimensional camera, by the coordinate after the correction under left image SR_L collimation mark coordinate system o'mn until gauge point

be converted to the point coordinate under left image SR_L photo coordinate system oxy

by the coordinate after the correction under right image SR_R collimation mark coordinate system o'mn until gauge point

be converted to the point coordinate under right image SR_R photo coordinate system oxy

3. by the picpointed coordinate of the position, outer orientation of the virtual three-dimensional camera calibration being obtained by step 5) and outer orientation attitude and the left image SR_L of super-resolution stereopsis (SR_L, SR_R)

picpointed coordinate with right image SR_R

carry out forward intersection computing, obtain the three-dimensional coordinate of target to be measured under O-XYZ coordinate system.

The various embodiments described above are only for illustrating the present invention; wherein the structure of each parts, connected mode and method step etc. all can change to some extent; every equivalents of carrying out on the basis of technical solution of the present invention and improvement, all should not get rid of outside protection scope of the present invention.

Claims (5)

1. the three-dimensional measuring method based on super-resolution image reconstruction, it comprises the following steps:

1) at four, take the photograph station and use respectively stereoscopic camera to treat to measure target and photograph, obtain four couples of stereopsis (L of target to be measured ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄); To four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) carry out obtaining after Image registration and image restoration the super-resolution stereopsis (SR_L, SR_R) of target to be measured;

2) use stereoscopic camera to take three-dimensional calibration field, according to the stereopsis (Calib_L, Calib_R) of three-dimensional calibration field, calculate the relative position T (V between the camera of left and right in the elements of interior orientation of stereoscopic camera and stereoscopic camera _x, V _y, V _z) and attitude R;

3) with the left image L of target first stop to be measured ₁center be initial point O, the left image L of first stop ₁east to being X-axis,, to being Y-axis, take and point into the sky direction as Z axis perpendicular to XOY plane in its north, sets up O-XYZ coordinate system, under O-XYZ coordinate system, the left image L of first stop ₁position, outer orientation and outer orientation attitude be respectively (0,0,0) and unit matrix; According to the left image L of first stop ₁position, outer orientation and the relative position T (V between the left and right camera of outer orientation attitude and stereoscopic camera _x, V _y, V _z) and attitude R, obtain the right image R of first stop ₁position, outer orientation and outer orientation attitude be respectively T (V _x, V _y, V _z) and R; By the left image L of first stop ₁position, outer orientation and outer orientation attitude and the right image R of first stop ₁position, outer orientation and outer orientation attitude obtain first stop stereopsis (L ₁, R ₁) position, outer orientation and outer orientation attitude;

4) according to first stop stereopsis (L ₁, R ₁) position, outer orientation and outer orientation attitude and four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on unique point of the same name, calculate stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude;

5) the super-resolution stereopsis (SR_L, SR_R) of target to be measured is considered as by virtual three-dimensional camera, take and obtain, at the upper selected equally distributed reference mark of super-resolution stereopsis (SR_L, SR_R);

To be positioned at four couples of stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) upper and corresponding with the upper reference mark of super-resolution stereopsis (SR_L, SR_R) corresponding image points coordinate and stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃), (L ₄, R ₄) position, revised outer orientation and attitude substitution collinearity equation carry out many baselines intersection calculation, obtains the three-dimensional coordinate at the upper reference mark of super-resolution stereopsis (SR_L, SR_R);

Utilize three-dimensional coordinate and the upper corresponding picpointed coordinate of super-resolution stereopsis (SR_L, SR_R) at these reference mark to carry out calibration to virtual three-dimensional camera, obtain elements of interior orientation, position, outer orientation and the outer orientation attitude of virtual three-dimensional camera;

6) at super-resolution stereopsis (SR_L, SR_R) three above reference mark of upper selection, utilize the three-dimensional coordinate at reference mark and the elements of interior orientation of virtual three-dimensional camera, position, outer orientation and outer orientation attitude respectively left image SR_L and right image SR_R to be set up to the affine Transform Model of calibration again, and utilize least square method to calculate the coefficient of calibration again of affine Transform Model;

7) by the coefficient of the calibration again a of the affine Transform Model of left image SR_L and right image SR_R in elements of interior orientation, position, outer orientation and the outer orientation attitude of the virtual three-dimensional camera calibration being obtained by step 5), the super-resolution stereopsis (SR_L, SR_R) that obtained by step 6) ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂and carry out forward intersection computing in the coordinate substitution collinearity equation for the treatment of gauge point on super-resolution stereopsis (SR_L, SR_R), obtain the three-dimensional coordinate of target to be measured.

2. a kind of three-dimensional measuring method based on super-resolution image reconstruction as claimed in claim 1, is characterized in that: described step 2), obtain the relative position T (V between the camera of left and right in the elements of interior orientation of stereoscopic camera and stereoscopic camera _x, V _y, V _z) and attitude R, it comprises the following steps:

1. use stereoscopic camera to take three-dimensional calibration field, obtain the stereopsis (Calib_L of three-dimensional calibration field, Calib_R), utilize characteristic point matching method to obtain the picpointed coordinate (x at the middle reference mark of stereopsis (Calib_L, Calib_R) of three-dimensional calibration field _tL, y _tL) and (x _iR, y _iR), t > 6;

2. by the known three-dimensional coordinate (X in reference mark in the stereopsis (Calib_L, Calib_R) of three-dimensional calibration field _t, Y _t, Z _t) and the picpointed coordinate (x at reference mark _tL, y _tL) and (x _tR, y _tR) substitution collinearity equation, by the picpointed coordinate (x at reference mark _tL, y _tL) and (x _tR, y _tR) distortion model of substitution pinhole imaging system, and by the distortion model of collinearity equation and pinhole imaging system, the two result after processing is carried out the compensating computation of entire light method, obtains position, the outer orientation T of the elements of interior orientation of stereoscopic camera, left image Calib_L _{calib_L}(X _ls, Y _ls, Z _ls) and attitude R _{calib_L}and position, the outer orientation T of right image Calib_R _{calib_R}(X _rs, Y _rs, Z _rs) and attitude R _{calib_R};

3. by position, the outer orientation T of left image Calib_L _{calib_L}(X _ls, Y _ls, Z _ls) and attitude R _{calib_L}, right image Calib_R position, outer orientation T _{calib_R}(X _rs, Y _rs, Z _rs) and attitude R _{calib_R}substitution formula

$\left\{\begin{array}{c}R=R_{\mathrm{Calib}\_L}^{-1}\·R_{\mathrm{Calib}\_R}\\ T=R_{\mathrm{Calib}\_L}^{-1}\·(T_{\mathrm{Calib}\_R}-T_{\mathrm{Calib}\_L})\end{array}\right.,$

Calculate the relative position T (V between the camera of left and right in stereoscopic camera _x, V _y, V _z) and attitude R.

3. a kind of three-dimensional measuring method based on super-resolution image reconstruction as claimed in claim 1 or 2, is characterized in that: in described step 4), obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude, it specifically comprises the following steps:

1. extract four couples of stereopsis (L of target to be measured ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on unique point of the same name, obtain unique point at stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) on picpointed coordinate;

2. by the elements of interior orientation of stereoscopic camera, first stop stereopsis (L ₁, R ₁) picpointed coordinate and the first stop stereopsis (L of upper unique point ₁, R ₁) position, outer orientation and outer orientation attitude substitution collinearity equation carry out forward intersection computing, obtain first stop stereopsis (L ₁, R ₁) three-dimensional coordinate of upper some unique points;

3. to first stop stereopsis (L ₁, R ₁) three-dimensional coordinate and the left image L of upper some unique points ₂, L ₃and L ₄on the picpointed coordinate of unique point carry out photogrammetric resection computing, obtain successively left image L ₂, L ₃and L ₄position, outer orientation

with outer orientation attitude initial value, i=2,3,4;

4. according to left image L ₂, L ₃and L ₄position, outer orientation with outer orientation attitude

initial value and the relative position T (V between the camera of left and right _x, V _y, V _z) and attitude R, obtain right image R ₂, R ₃and R ₄position, outer orientation

with outer orientation attitude

initial value be respectively:

$\left\{\begin{array}{c}{T}_i^{\mathrm{right}}=R_i^{\mathrm{left}}\·T+T_i^{\mathrm{left}}\\ R_i^{\mathrm{right}}=R_i^{\mathrm{left}}\·R\end{array}\right.;$

By left image L ₂, L ₃and L ₄position, outer orientation

with outer orientation attitude

and right image R ₂, R ₃and R ₄position, outer orientation

with outer orientation attitude

obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the initial value of position, outer orientation initial value with outer orientation attitude

5. adopt many baselines bundle adjustment method to stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the initial value of position, outer orientation

initial value with outer orientation attitude

revise, obtain stereopsis (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) position, revised outer orientation and outer orientation attitude.

4. a kind of three-dimensional measuring method based on super-resolution image reconstruction as claimed in claim 1 or 2, it is characterized in that: in described step 6), to super-resolution stereopsis (SR_L, SR_R) in, left image SR_L and right image SR_R set up the process of affine Transform Model, and it comprises the following steps:

1. in the upper selection of super-resolution stereopsis (SR_L, SR_R) and at stereopsis (L ₁, R ₁), (L ₂, R ₂), (L ₃, R ₃) and (L ₄, R ₄) the upper corresponding unique point of unique point of the same name of extracting, and using these unique points of the same name as reference mark, utilize the forward intersection of many baselines to calculate the three-dimensional coordinate at these reference mark;

2. by super-resolution stereopsis (SR_L, SR_R) the upper three-dimensional coordinate at reference mark and the position, outer orientation of virtual three-dimensional camera and outer orientation attitude substitution collinearity equation, calculate coordinate under the photo coordinate system oxy of reference mark on super-resolution stereopsis (SR_L, SR_R) (x ' _l, y ' _l) and (x ' _r, y' _r);

3. by the elements of interior orientation of virtual three-dimensional camera, reference mark the coordinate under the photo coordinate system oxy on super-resolution stereopsis (SR_L, SR_R) (x ' _l, y ' _l) and (x' _r, y' _r) the camera distortion model of substitution pinhole imaging system, use Newton iteration method to calculate the coordinate (m under the collimation mark coordinate system o'mn of reference mark on super-resolution stereopsis (SR_L, SR_R) _l, n _l) and (m _r, n _r);

4. according to reference mark the measurement coordinate under the collimation mark coordinate system o'mn on (SR_L, SR_R) image (m ' _l, n ' _l) and (m' _r, n' _r) and the collimation mark coordinate system o'mn of the reference mark that calculates in 3. of step on super-resolution stereopsis (SR_L, SR_R) under coordinate (m _l, n _l) and (m _r, n _r), left image SR_L in super-resolution stereopsis (SR_L, SR_R) and right image SR_R are set up respectively to affine Transform Model:

$\left\{\begin{array}{c}{m}_l^{\′}-m_l=a_0+a_1{m}_l+a_2{n}_l\\ n_l^{\′}-n_l=b_0+b_1{m}_l+b_2{n}_l\end{array}\right.,$

$\left\{\begin{array}{c}{m}_r^{\′}-m_r=c_0+c_1{m}_r+c_2{n}_r\\ n_r^{\′}-n_r=d_0+d_1{m}_r+d_2{n}_r\end{array}\right.,$

In formula, a ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂it is respectively the coefficient of calibration again of left image SR_L and right image SR_R in super-resolution stereopsis (SR_L, SR_R).

5. a kind of three-dimensional measuring method based on super-resolution image reconstruction as claimed in claim 1 or 2, is characterized in that: in described step 7), obtain the three-dimensional coordinate of target to be measured, it specifically comprises the following steps:

1. according to the coefficient of the calibration again a of the affine Transform Model of left image SR_L and right image SR_R in super-resolution stereopsis (SR_L, SR_R) ₀, a ₁, a ₂, b ₀, b ₁, b ₂and c ₀, c ₁, c ₂, d ₀, d ₁, d ₂and the picpointed coordinate that measures under collimation mark coordinate system o'mn of left image SR_L and right image SR_R with

calculate the coordinate after correcting under measuring the collimation mark coordinate system o'mn of target at left image SR_L and right image SR_R

with be respectively:

$\left\{\begin{array}{c}{x}_{\mathrm{lrow}}^{\′\mathrm{Tar}}=x_{\mathrm{lrow}}^{\mathrm{Tar}}+a_0+a_1{x}_{\mathrm{lrow}}^{\mathrm{Tar}}+a_2{y}_{\mathrm{lcol}}^{\mathrm{Tar}}\\ y_{\mathrm{lcol}}^{\′\mathrm{Tar}}=y_{\mathrm{lcol}}^{\mathrm{Tar}}+b_0+b_1{x}_{\mathrm{lrow}}^{\mathrm{Tar}}+b_2{y}_{\mathrm{lcol}}^{\mathrm{Tar}}\end{array}\right.,$

$\left\{\begin{array}{c}{x}_{\mathrm{rrow}}^{\′\mathrm{Tar}}=x_{\mathrm{rrow}}^{\mathrm{Tar}}+c_0+c_1{x}_{\mathrm{rrow}}^{\mathrm{Tar}}+c_2{y}_{\mathrm{rcol}}^{\mathrm{Tar}}\\ y_{\mathrm{rcol}}^{\′\mathrm{Tar}}=y_{\mathrm{rcol}}^{\mathrm{Tar}}+d_0+d_1{x}_{\mathrm{rrow}}^{\mathrm{Tar}}+d_2{y}_{\mathrm{rcol}}^{\mathrm{Tar}}\end{array}\right.;$

2. use the elements of interior orientation of pinhole imaging system distortion model and the calibration of virtual three-dimensional camera, respectively by the coordinate after the correction under left image SR_L collimation mark coordinate system o'mn until gauge point

with the coordinate after correction under right image SR_R collimation mark coordinate system o'mn

be converted to the point coordinate under left image SR_L photo coordinate system oxy

with the point coordinate under right image SR_R photo coordinate system oxy

3. by the picpointed coordinate of the position, outer orientation of virtual three-dimensional camera calibration and outer orientation attitude and the left image SR_L of super-resolution stereopsis (SR_L, SR_R)

picpointed coordinate with right image SR_R

carry out forward intersection computing, obtain the three-dimensional coordinate of target to be measured under O-XYZ coordinate system.

Images