CN102447927B - Method for warping three-dimensional image with camera calibration parameter - Google Patents

Abstract

The invention discloses a method for warping a three-dimensional image with calibration parameter. In the method, a pan camera and a shift-sensor camera are used for setting to generate a destination image, and one the premise of calculating the pixel coordinates of the destination image by using a simplified three-dimensional image warping formula based on depth image rendering and a pixel depth value calculating formula, the pixel values of reference image points are copied to corresponding points of the destination image, so that the calculated amount is reduced and the hardware implementation is facilitated.

Description

A kind of 3-D view transform method that the camera calibration parameter is arranged

Technical field

The invention belongs to and draw (Depth-Image-Based Rendering is called for short DIBR) technical field based on depth image in the 3D television system, more specifically say, relate to a kind of 3-D view transform method that the camera calibration parameter is arranged.

Background technology

To generate the new virtual visual point image of a width according to reference picture (reference image) and corresponding depth image (depth image) thereof based on the depth image rendering technique, i.e. target image (destination image).Be that the conventional three-dimensional video format is compared with utilizing the synthetic 3-dimensional image of left and right two-way planar video, employing DIBR technology only needs to transmit one road video afterwards and range image sequence just can synthesize 3-dimensional image, and can realize very easily the switching of two and three dimensions, avoided simultaneously the computational complexity of the three dimensions conversion that brought by classic view generation method.Just because of this, the DIBR technology is widely applied in the synthetic 3-dimensional image of 3D TV, and it has also caused more and more keen interest of people.Usually, people adopt needs the 3D video of DIBR technology to call 3D video (depth-image-based3D video) based on depth image.

The core procedure of DIBR technology is 3-D view conversion (3d image warping).The 3-D view conversion can be with reference to the spot projection in image to three dimensions, then with the some re-projection in three dimensions to the target image plane, thereby generate new viewpoint view, i.e. target image.

Yet in traditional 3-D view transform method, the target image pixel calculates more complicated, and amount of calculation is larger, can't accomplish at present the real-time drafting, and is unfavorable for the realization of hardware.

Summary of the invention

The object of the invention is to overcome the deficiencies in the prior art, provide a kind of amount of calculation the little 3-D view transform method that the camera calibration parameter is arranged.

For achieving the above object, the present invention has the 3-D view transform method of camera calibration parameter, it is characterized in that, comprises the following steps:

(1), initialization disparity map M, its all elements is set to cavity point parallax value;

(2), to distinguish target image be left view or right view, if left view is set to 1 with Boolean variable α, from right to left, sequential scanning reference picture from top to bottom is by row traversal reference picture I _refIn pixel; If right view, from left to right, sequential scanning reference picture from top to bottom, and Boolean variable α is set to 0, by row traversal reference picture I _refIn pixel;

In when traversal, 2.1), to reference picture I _refIn v _refRow u _refThe pixel p of row _ref, calculate it at target image I according to formula (1) _desThe matched pixel point p of middle correspondence _des

$\left\{\begin{array}{c}{u}_{\mathrm{des}}={(-1)}^{\mathrm{\α}}[2h-\frac{f\·s_x\·B}{z_w}]+u_{\mathrm{ref}}\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}---\left(1\right)\right.$

In formula (1), (u _ref, v _ref), (u _des, v _des) represent respectively reference picture I _refOn pixel p _refWith it at target image I _desThe matched pixel point p of upper correspondence _desLevel, vertical coordinate, i.e. x axle, y axial coordinate, the horizontal displacement pixel number that h does when representing by sensing conversion video camera (shift-sensor camera), parallax free plane (ZPS plane) to be set, f presentation video focal length, s _xWhen expression is changed to the image pixel coordinate system by image physical coordinates system on the x direction of principal axis number of pixel corresponding to per unit physical length, B represents the length of base, z _wExpression pixel p _refCorresponding depth value;

Depth value z _wDetermine according to formula (2):

$z_w=\frac{1}{\frac{D(u_{\mathrm{ref}},v_{\mathrm{ref}})}{g-1}\×(\frac{1}{z_{\min }}-\frac{1}{z_{\min }})+\frac{1}{z_{\min }}},---\left(2\right)$

In formula (2), D (u _ref, v _ref) expression depth image in pixel (u _ref, v _ref) gray value, g is the depth image gray scale, z _minBe nearest depth value, z _maxBe depth value farthest; Usually depth image gray scale g is 8 bits, namely 256.

2.2), judgement pixel p _desWhether drop on target image I _desIn; If pixel p _desHorizontal coordinate u _desSatisfy:

0≤u _des<W _i, (1)

Show pixel p _desDrop on target image I _desIn, with pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _refWherein, W _iFor the horizontal pixel of target image is counted;

(3), traveled through reference picture I _refIn all pixels, export target image I _desAnd disparity map M.

Goal of the invention of the present invention is achieved in that

The present invention has the 3-D view transform method of camera calibration parameter, pass through truck camera, sensing conversion video camera (shift-sensor camera) arranges the generation target image, the coordinate time that calculates the target image pixel in 3-D view transformation for mula and the pixel depth value computing formula based on the depth image drafting of utilizing simplification only need have certain translation in the horizontal direction, then the pixel value with reference to picture point copies on the target image corresponding points, thereby reduced amount of calculation, be conducive to hardware and realize and process in real time.

Description of drawings

Fig. 1 is that in the present invention, reference picture is the schematic diagram of right view as left view generation target image;

Fig. 2 is that in the present invention, reference picture is the schematic diagram of left view as right view generation target image;

Fig. 3 is the 3-D view transform method one specific implementation method flow chart that the present invention has the camera calibration parameter.

Embodiment

Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described, so that those skilled in the art understands the present invention better.What need to point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.

Fig. 1 is that in the present invention, reference picture is the schematic diagram of right view as left view generation target image.

In this enforcement, as shown in Figure 1, according to painter's algorithm, when drawing right view, should according to from left to right, from top to bottom sequential scanning reference picture, by row traversal reference picture I _refIn pixel p _ref, its coordinate is (u _ref, v _ref); Then find target image I according to the 3-D view transformation for mula _desThe matched pixel point p of upper correspondence _desLevel, vertical coordinate, with pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _ref

Fig. 2 is that in the present invention, reference picture is the schematic diagram of left view as right view generation target image.

In this enforcement, as shown in Figure 2, according to painter's algorithm, when drawing left view, should be according to from right to left, sequential scanning reference picture from top to bottom, by row traversal reference picture I _refIn pixel p _ref, its coordinate is (u _ref, v _ref); Then find target image I according to the 3-D view transformation for mula _desThe matched pixel point p of upper correspondence _desLevel, vertical coordinate, with pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _ref

Fig. 3 is the 3-D view transform method one specific implementation method flow chart that the present invention has the camera calibration parameter.

In this enforcement, as shown in Figure 3, the present invention has the 3-D view transform method of camera calibration parameter to realize in the situation that known camera calibration parameter, is generated the function of a width target image by a width reference picture and depth image thereof;

As shown in Figure 3, concrete steps are:

1), at first input reference picture I _refAnd depth image D, their resolution is all W _i* H _iFocal distance f; The number s of pixel corresponding to per unit physical length on the x direction of principal axis _xLength of base B; For the ZPS plane being set, the horizontal displacement pixel number h (h 〉=0) that reference picture is done; Nearest depth value z _min, i.e. the depth value of nearly shear plane and farthest depth value z _max, i.e. the depth value of shear plane far away; The scanning sequency sign rend_order of reference picture, the value of rend_order is obtained by painter's algorithm;

2), initiation parameter

Initialization disparity map M, all elements of putting M is-128, and this point of-128 expressions is the cavity, and namely just disparity map M all elements is set to cavity point parallax value; With reference to image I _refThe pixel direct copying to target image I _desIn, this pixel is at target image I _desIn position and reference picture I _refIn the position identical, i.e. reference picture I _refWith target image I _desIdentical, like this can be better the fill some little cavities;

3), with vertical coordinate v _refBe set to 0, i.e. v _ref=0;

4), determine reference picture I according to scanning sequency sign rend_order _refScanning sequency, namely target image is left view or right view; Rend_order=0 represents from left to right, sequential scanning reference picture from top to bottom, and expression generates right view; Rend_order=1 represents from right to left, and sequential scanning reference picture from top to bottom generates left view; Concrete scanning and copy are as shown in Figure 1 and Figure 2;

5), for generating right view, with horizontal coordinate u _refBe set to 0, i.e. u _ref=0; For generating left view, with horizontal coordinate u _refBe set to W _i-1, i.e. u _ref=W _i-1;

6), according to formula (1) calculating pixel point p _refCorresponding pixel points p _desCoordinate u _des

7), judgement pixel p _desCoordinate u _desWhether satisfy 0≤u _des＜W _i, i.e. pixel p _desWhether drop on target image I _desIn; If satisfy, carry out step 8); Do not satisfy, directly carry out step 9);

8), with pixel p _refPixel value copies pixel p to _des, the element (u in disparity map M _ref, v _ref) be set to u _des-u _ref, i.e. M (u _des, v _des)=u _des-u _ref

9), for generating right view, with horizontal coordinate u _ref Add 1, i.e. u _ref=u _ref+ 1; For generating left view, with horizontal coordinate u _ref Subtract 1, i.e. u _ref=u _ref-1;

10), for generating right view, judgement u _refWhether less than reference picture I _refThe horizontal pixel W that counts _i, if so, return to step 6), otherwise, carry out step 11); For generating left view, judgement u _refWhether more than or equal to 0, if so, return to step 6), otherwise, carry out step 11); This is all to judge whether to travel through to complete delegation;

11), with vertical coordinate v _ref Add 1, i.e. v _ref=v _ref+ 1;

12), judgement vertical coordinate v _refWhether less than reference picture I _refThe vertical pixel H that counts _i, if so, return to step 5), otherwise, carry out step 13);

13), traveled through reference picture I _refIn all pixels, export target image I _desAnd disparity map M; Each element in disparity map MM is the signed integer of 8bits, has indicated target image I _desWhether middle corresponding points are empty point, and-128 these points of expression are empty points, and this point of all the other value representations is not empty point, and its value is parallax value.

Although the above is described the illustrative embodiment of the present invention; so that those skilled in the art understand the present invention; but should be clear; the invention is not restricted to the scope of embodiment; to those skilled in the art; as long as various variations appended claim limit and the spirit and scope of the present invention determined in, these variations are apparent, all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (2)

1. the 3-D view transform method that the camera calibration parameter is arranged, is characterized in that, comprises the following steps:

(1), initialization disparity map M, its all elements is set to cavity point parallax value;

(2), to distinguish target image be left view or right view, if left view is set to 1 with Boolean variable α, from right to left, sequential scanning reference picture from top to bottom is by row traversal reference picture I _refIn pixel; If right view, from left to right, sequential scanning reference picture from top to bottom, and Boolean variable α is set to 0, by row traversal reference picture I _refIn pixel;

In when traversal, 2.1), to reference picture I _refIn v _refRow u _refThe pixel p of row _ref, calculate it at target image I according to formula (1) _desThe matched pixel point p of middle correspondence _des

$\left\{\begin{array}{c}{u}_{\mathrm{des}}={(-1)}^a[2h-\frac{f\&CenterDot;s_x\&CenterDot;B}{z_w}]+u_{\mathrm{ref}}\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.----\left(1\right)$

In formula (1), (u _ref, v _ref), (u _des, v _des) represent respectively reference picture I _refOn pixel p _refWith it at target image I _desThe matched pixel point p of upper correspondence _desLevel, vertical coordinate, i.e. x axle, y axial coordinate, the horizontal displacement pixel number that h does when representing by sensing conversion video camera, the parallax free plane to be set, f presentation video focal length, s _xWhen expression is changed to the image pixel coordinate system by image physical coordinates system on the x direction of principal axis number of pixel corresponding to per unit physical length, B represents the length of base, z _wExpression pixel p _refCorresponding depth value;

Depth value z _wDetermine according to formula (2):

$z_w=\frac{1}{\frac{D(u_{\mathrm{ref}},v_{\mathrm{ref}})}{g-1}\&times;(\frac{1}{z_{\min }}-\frac{1}{z_{\max }})+\frac{1}{z_{\max }}},---\left(2\right)$

In formula (2), D (u _ref, v _ref) expression depth image in pixel (u _ref, v _ref) gray value, g is the depth image gray scale, z _minBe nearest depth value, z _maxBe depth value farthest;

2.2), judgement pixel p _desWhether drop on target image I _desIn; If pixel p _desHorizontal coordinate u _desSatisfy:

0≤u _des<W _i, (1)

Show pixel p _desDrop on target image I _desIn, with pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _refWherein, W _iFor the horizontal pixel of target image is counted;

(3), traveled through reference picture I _refIn all pixels, export target image I _desAnd disparity map M.

2. the 3-D view transform method that the camera calibration parameter is arranged according to claim 1, is characterized in that in step (1), also will be with reference to image I _refThe pixel direct copying to target image I _desIn, and this pixel is at target image I _desIn position and reference picture I _refIn the position identical.

Images