CN102447927A - Method for warping three-dimensional image with camera calibration parameter - Google Patents

Abstract

The invention discloses a method for warping a three-dimensional image with calibration parameter. In the method, a pan camera and a shift-sensor camera are used for setting to generate a destination image, and one the premise of calculating the pixel coordinates of the destination image by using a simplified three-dimensional image warping formula based on depth image rendering and a pixel depth value calculating formula, the pixel values of reference image points are copied to corresponding points of the destination image, so that the calculated amount is reduced and the hardware implementation is facilitated.

Description

A kind of 3-D view transform method that the camera calibration parameter is arranged

Technical field

The invention belongs to and draw (Depth-Image-Based Rendering is called for short DIBR) technical field based on depth image in the 3D television system, more specifically, relate to a kind of 3-D view transform method that the camera calibration parameter is arranged.

Background technology

Based on the depth image rendering technique is to generate the new virtual visual point image of a width of cloth according to reference picture (reference image) and corresponding depth image (depth image) thereof, i.e. target image (destination image).With utilizing the synthetic 3-dimensional image of left and right sides two-way planar video is that the conventional three-dimensional video format is compared; Adopt the DIBR technology only need transmit one road video and range image sequence thereof afterwards and just can synthesize 3-dimensional image; And can realize the switching of two and three dimensions very easily, the computational complexity of the three dimensions conversion of having avoided simultaneously being brought by classic view generation method.Just because of this, the DIBR technology has obtained extensive use in the synthetic 3-dimensional image of 3D TV, and it has also caused more and more keen interest of people.Usually, people adopt needs the 3D video of DIBR technology to call the 3D video (depth-image-based 3Dvideo) based on depth image.

The core procedure of DIBR technology is 3-D view conversion (3d image warping).The 3-D view conversion can be with the spot projection in the reference picture to three dimensions, again with the some re-projection in the three dimensions to the target image plane, thereby generate new viewpoint view, i.e. target image.

Yet the target image pixel calculates more complicated in traditional 3-D view transform method, and amount of calculation is bigger, can't accomplish the real-time drafting at present, and is unfavorable for the realization of hardware.

Summary of the invention

The objective of the invention is to overcome the deficiency of prior art, provide a kind of amount of calculation the little 3-D view transform method that the camera calibration parameter is arranged.

For realizing above-mentioned purpose, the present invention has the 3-D view transform method of camera calibration parameter, it is characterized in that, may further comprise the steps:

(1), initialization disparity map M, its all elements is changed to cavity point parallax value;

(2), to distinguish target image be left view or right view, if left view then is changed to 1 with Boolean variable α, from right to left, sequential scanning reference picture from top to bottom is by row traversal reference picture I _RefIn pixel; If right view, then from left to right, sequential scanning reference picture from top to bottom, and Boolean variable α is changed to 0, by row traversal reference picture I _RefIn pixel;

In when traversal, 2.1), to reference picture I _RefIn v _RefRow u _RefThe pixel u of row _Ref, calculate it at target image I according to formula (1) _DesMiddle corresponding matched pixel point u _Des

$\left\{\begin{array}{c}{u}_{\mathrm{des}}={(-1)}^{\mathrm{\α}}[2h-\frac{f\·s_x\·B}{z_w}]+u_{\mathrm{ref}}\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.---\left(1\right)$

In the formula (1), (u _Ref, v _Ref), (u _Des, v _Des) represent reference picture I respectively _RefOn pixel u _RefWith its at target image I _DesLast corresponding matched pixel point u _DesLevel, vertical coordinate, i.e. the horizontal displacement pixel number of being done when x axle, y axial coordinate, h represent by sensing conversion video camera (shift-sensor camera) parallax free plane (ZPS plane) to be set, f presentation video focal length, s _xExpression by image physical coordinates system when the image pixel coordinate system change on the x direction of principal axis number of the pixel of per unit physical length correspondence, B representes the length of base, z _wRemarked pixel point u _RefCorresponding depth value;

Depth value z _wConfirm according to formula (2):

$z_w=\frac{1}{\frac{D(u_{\mathrm{ref}},v_{\mathrm{ref}})}{g-1}\×(\frac{1}{z_{\min }}-\frac{1}{z_{\max }})+\frac{1}{z_{\max }}},---\left(2\right)$

In the formula (2), D (u _Ref, v _Ref) expression depth image in pixel (u _Ref, v _Ref) gray value, g is the depth image gray scale, z _MinBe nearest depth value, z _MaxBe depth value farthest; Usually depth image gray scale g is 8 bits, promptly 256.

2.2), judge pixel u _DesWhether drop on target image I _DesIn; If pixel u _DesHorizontal coordinate u _DesSatisfy:

0≤u _des＜W _i， (1)

Then show pixel u _DesDrop on target image I _DesIn, with pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _RefWherein, W _iFor the horizontal pixel of target image is counted;

(3), traveled through reference picture I _RefIn all pixels, export target image I then _DesAnd disparity map M.

Goal of the invention of the present invention is achieved in that

The present invention has the 3-D view transform method of camera calibration parameter; Pass through truck camera; Sensing conversion video camera (shift-sensor camera) is provided with the generation target image; The coordinate time that calculates the target image pixel in 3-D view transformation for mula and the pixel depth value computing formula based on the depth image drafting of utilizing simplification only need have certain translation in the horizontal direction; Pixel value with the reference diagram picture point copies on the target image corresponding points then, thereby has reduced amount of calculation, helps hardware and realizes and processing in real time.

Description of drawings

Fig. 1 is that reference picture is the sketch map of right view as left view generation target image among the present invention;

Fig. 2 is that reference picture is the sketch map of left view as right view generation target image among the present invention;

Fig. 3 is the 3-D view transform method one practical implementation method flow diagram that the present invention has the camera calibration parameter.

Embodiment

Describe below in conjunction with the accompanying drawing specific embodiments of the invention, so that those skilled in the art understands the present invention better.What need point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.

Fig. 1 is that reference picture is the sketch map of right view as left view generation target image among the present invention.

In this enforcement, as shown in Figure 1, according to painter's algorithm, when drawing right view, should according to from left to right, from top to bottom sequential scanning reference picture, by row traversal reference picture I _RefIn pixel u _Ref, its coordinate is (u _Ref, v _Ref); Find target image I according to the 3-D view transformation for mula then _DesLast corresponding matched pixel point u _DesLevel, vertical coordinate, with pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _Ref

Fig. 2 is that reference picture is the sketch map of left view as right view generation target image among the present invention.

In this enforcement, as shown in Figure 2, according to painter's algorithm, when drawing left view, should be according to from right to left, sequential scanning reference picture from top to bottom, by row traversal reference picture I _RefIn pixel u _Ref, its coordinate is (u _Ref, v _Ref); Find target image I according to the 3-D view transformation for mula then _DesLast corresponding matched pixel point u _DesLevel, vertical coordinate, with pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _Ref

Fig. 3 is the 3-D view transform method one practical implementation method flow diagram that the present invention has the camera calibration parameter.

In this enforcement, as shown in Figure 3, the present invention has the 3-D view transform method of camera calibration parameter to be implemented under the situation of known camera calibration parameter, is generated the function of a width of cloth target image by a width of cloth reference picture and depth image thereof;

As shown in Figure 3, concrete steps are:

1), at first imports reference picture I _RefAnd depth image D, their resolution all is W _i* H _iFocal distance f; The number s of the pixel of per unit physical length correspondence on the x direction of principal axis _xLength of base B; For the ZPS plane being set, the horizontal displacement pixel number h (h>=0) that reference picture is done; Nearest depth value z _Min, the depth value of promptly near shear plane and depth value z farthest _Max, the depth value of shear plane promptly far away; The scanning sequency sign rend_order of reference picture, the value of rend_order is obtained by painter's algorithm;

2), initiation parameter

Initialization disparity map M, all elements of putting M is-128, and this is the cavity for-128 expressions, and promptly just disparity map M all elements is changed to cavity point parallax value; With reference picture I _RefPixel directly copy target image I to _DesIn, this pixel is at target image I _DesIn position and reference picture I _RefIn the position identical, i.e. reference picture I _RefWith target image I _DesIdentical, like this can be better the fill some little cavities;

3), with vertical coordinate v _RefBe changed to 0, i.e. v _Ref=0;

4), confirm reference picture I according to scanning sequency sign rend_order _RefScanning sequency, promptly target image is left view or right view; Rend_order=0 representes from left to right, sequential scanning reference picture from top to bottom, and expression generates right view; Rend_order=1 representes from right to left, and sequential scanning reference picture from top to bottom generates left view; Concrete scanning and copy are like Fig. 1, shown in Figure 2;

5), for generating right view, with horizontal coordinate u _RefBe changed to 0, i.e. u _Ref=0; For generating left view, with horizontal coordinate u _RefBe changed to W _i-1, i.e. u _Ref=W _i-1;

6), according to formula (1) calculating pixel point u _RefCorresponding pixel points u _DesCoordinate u _Des

7), judge pixel u _DesCoordinate u _DesWhether satisfy 0≤u _Des＜W _i, i.e. pixel u _DesWhether drop on target image I _DesIn; If satisfy, carry out step 8); Do not satisfy, directly carry out step 9);

8), then with pixel u _RefPixel value copies pixel u to _Des, the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _Ref, i.e. M (u _Des, v _Des)=u _Des-u _Ref

9), for generating right view, with horizontal coordinate u _RefAdd 1, i.e. u _Ref=u _Ref+ 1; For generating left view, with horizontal coordinate u _RefSubtract 1, i.e. u _Ref=u _Ref-1;

10), for generating right view, judgement u _RefWhether less than reference picture I _RefThe horizontal pixel W that counts _i, if, then return step 6), otherwise, carry out step 11); For generating left view, judge u _RefWhether more than or equal to 0, if, then return step 6), otherwise, carry out step 11); This all is to judge whether traversal completion delegation;

11), with vertical coordinate v _RefAdd 1, i.e. v _Ref=v _Ref+ 1;

12), judge vertical coordinate v _RefWhether less than reference picture I _RefThe vertical pixel H that counts _i, if, return step 5), otherwise, carry out step 13);

13), traveled through reference picture I _RefIn all pixels, export target image I then _DesAnd disparity map M; Each element among the disparity map MM is the signed integer of 8bits, has indicated target image I _DesWhether middle corresponding points are empty point, and-128 these points of expression are empty points, and this point of all the other value representations is not empty point, and its value is parallax value.

Although above the illustrative embodiment of the present invention is described; So that the technical staff in present technique field understands the present invention, but should be clear, the invention is not restricted to the scope of embodiment; To those skilled in the art; As long as various variations appended claim limit and the spirit and scope of the present invention confirmed in, these variations are conspicuous, all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (2)

1. the 3-D view transform method that the camera calibration parameter is arranged is characterized in that, may further comprise the steps:

(1), initialization disparity map M, its all elements is changed to cavity point parallax value;

(2), to distinguish target image be left view or right view, if left view then is changed to 1 with Boolean variable α, from right to left, sequential scanning reference picture from top to bottom is by row traversal reference picture I _RefIn pixel; If right view, then from left to right, sequential scanning reference picture from top to bottom, and Boolean variable α is changed to 0, by row traversal reference picture I _RefIn pixel;

In when traversal, 2.1), to reference picture I _RefIn v _RefRow u _RefThe pixel u of row _Ref, calculate it at target image I according to formula (1) _DesMiddle corresponding matched pixel point u _Des

$\left\{\begin{array}{c}{u}_{\mathrm{des}}={(-1)}^{\mathrm{\α}}[2h-\frac{f\·s_x\·B}{z_w}]+u_{\mathrm{ref}}\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.---\left(1\right)$

In the formula (1), (u _Ref, v _Ref), (u _Des, v _Des) represent reference picture I respectively _RefOn pixel u _RefWith its at target image I _DesLast corresponding matched pixel point u _DesLevel, vertical coordinate, i.e. the horizontal displacement pixel number of being done when x axle, y axial coordinate, h represent by sensing conversion video camera (shift-sensor camera) parallax free plane (ZPS plane) to be set, f presentation video focal length, s _xExpression by image physical coordinates system when the image pixel coordinate system change on the x direction of principal axis number of the pixel of per unit physical length correspondence, B representes the length of base, z _wRemarked pixel point u _RefCorresponding depth value;

Depth value z _wConfirm according to formula (2):

$z_w=\frac{1}{\frac{D(u_{\mathrm{ref}},v_{\mathrm{ref}})}{g-1}\×(\frac{1}{z_{\min }}-\frac{1}{z_{\max }})+\frac{1}{z_{\max }}},---\left(2\right)$

In the formula (2), D (u _Ref, v _Ref) expression depth image in pixel (u _Ref, v _Ref) gray value, g is the depth image gray scale, z _MinBe nearest depth value, z _MaxBe depth value farthest;

2.2), judge pixel u _DesWhether drop on target image I _DesIn; If pixel u _DesHorizontal coordinate u _DesSatisfy:

0≤u _des＜W _i， (1)

Then show pixel u _DesDrop on target image I _DesIn, with pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _RefWherein, W _iFor the horizontal pixel of target image is counted;

(3), traveled through reference picture I _RefIn all pixels, export target image I then _DesAnd disparity map M.

2. the 3-D view transform method that the camera calibration parameter is arranged according to claim 1 is characterized in that in step (1), also will be with reference picture I _RefPixel directly copy target image I to _DesIn, and this pixel is at target image I _DesIn position and reference picture I _RefIn the position identical.

Images