CN104902253A - Three-dimensional image generating method based on improved Bayesian model - Google Patents

Abstract

The invention belongs to the field of image processing, particularly relates to a three-dimensional image generating method based on an improved Bayesian model, and aims to generate three-dimensional images more efficiently and intuitively. The method comprises the following steps: firstly, calculating the per-pixel viewpoint position of a synthesized view angle with given parallax; secondly, generating a synthesized image through the improved Bayes model; and lastly, generating a red-blue three-dimensional image with a reference image and the synthesized image. Compared with an existing method, the three-dimensional image generating method based on the improved Bayesian model provided by the invention has remarkable advantages on the aspects of time performance, quality of generated images and universality. Through adoption of the method, the parallax can be specified intuitively pixel by pixel, and post-processing of the three-dimensional image is facilitated. The method has a wide application prospect.

Description

A kind of stereoscopic image generation method based on improving Bayesian model

Technical field

The invention belongs to image processing field, being specifically related to a kind of stereoscopic image generation method based on improving Bayesian model, more efficiently, intuitively can generating stereo-picture.

Background technology

Along with 3D TV, film and game universal and popular, more and more 3D content is produced, and the stereoscopic display device of a new generation, also makes these 3D contents enter the life of ordinary consumer.But due to the complexity of stereoscopic display device and the shortage of three-dimensional material post-processed instrument, create credible, comfortable 3D content and remain a challenging job, traditional instrument and handling process can not well for the process of 3D content.Therefore, we need the handling process again thinking deeply 3D content creation and editor, and then invent more succinct, effective 3D content post-processing method.

Binocular parallax is the topmost visual cues of three-dimensional perception, therefore, for handling implement very important of parallax.The modal method of control binocular parallax is exactly the baseline between setting two different visual angles cameras, but the parallax shown by most 3D content has exceeded the receptible disparity range of people all, is very limited to the adjustment of baseline.Reduce baseline and can reduce binocular parallax, but whole scene also can be caused to flatten simultaneously, lack third dimension.It is the parallax of image of remapping that binocular parallax controls more complicated method, and then synthesize new image, the difficult point of this method is, the accurate calculating of parallax and synthetic images carry out hole repair etc.The people such as Changil Kim propose a kind of method by pixel parallax control in " Multi-Perspective Stereoscopy from Light Fields ", but the method is confined to light field condition, there is the deficiencies such as consuming time, parallax adjusting range is limited.

The people such as Sergi Pujades propose the visual angle synthetic method based on image in " Bayesian View Synthesis and Image-Based Rendering Principles ".Image error and geometric error is introduced in the modeling of the method system visual angle building-up process, construct the maximum a-posteriori estimation of a complete physics generation model and correspondence thereof, explain well based on the heuristic rule in the visual angle synthetic method of image.But the method does not consider the Generating Problems of stereo-picture, and can not by the control parallax of pixel, can not input picture be screened, therefore can not directly apply to the generation of stereo-picture.

Summary of the invention

The present invention is directed to stereo-picture Generating Problems, propose a kind of stereoscopic image generation method based on improving Bayesian model, the method has versatility, can control parallax by pixel.First, by given parallaxometer calculate synthesis visual angle by pixel viewpoint position; Then, the Bayesian model by improving generates composograph; Finally, reference picture and composograph is utilized to generate red blue stereogram.

The object of the invention is to be achieved through the following technical solutions.

A kind of stereoscopic image generation method based on improving Bayesian model of the present invention, is characterized in that: it comprises the following steps:

Step one, to obtain by reference to image composograph by pixel viewpoint position.

Described reference picture is the true picture that a width is specified, and reference picture all pixels viewpoint position is identical, and the viewpoint position of reference picture represents with symbol s.

Described composograph is the width composograph obtained by one group of input picture, and composograph specifies viewpoint position by pixel.

Step 1.1: (use symbol v by formula (1) computing reference image ₀represent) each pixel of clock standardization parallax (use symbol d ₀x () represents), standardization parallax d ₀x () (uses symbol z by the standard depth of pixel each in reference picture ₀(x) represent) and the imaging focal length (representing with symbol f) of reference picture obtain.Wherein, x is reference picture v ₀the index value of middle pixel, x ∈ [1, N], N are reference picture v ₀the quantity of middle pixel.

$d_0\left(x\right)=\frac{f}{z_0\left(x\right)}---\left(1\right)$

Step 1.2: given composograph (representing with symbol u) and reference picture v ₀by pixel image parallax (representing with symbol D (x)), because the viewpoint of composograph and the viewpoint of reference picture are positioned at same plane, and the imaging plane of correspondence is identical, by formula (2) by image parallactic D (x) calculate composograph relative to reference picture by pixel viewpoint displacement (representing with symbol Δ s (x)).

$\mathrm{\Δs}\left(x\right)=\frac{D\left(x\right)}{d_0\left(x\right)}---\left(2\right)$

Step 1.3: by formula (3) calculate composograph by pixel viewpoint position (representing with symbol s ' (x)).

s′(x)＝s+Δs(x) (3)

Wherein, s is the viewpoint position of reference picture.

Step 2, the Bayesian model passing through to improve generate composograph.

Step 2.1: for viewpoint position s ' (x) of pixel each in composograph, determines that a new imaging plane (uses symbol Γ _jrepresent).

Step 2.2: the geometry of given scenario estimates and one group of input picture (uses symbol v _irepresent), the imaging plane calculating each input picture (uses symbol Ω _irepresent) in each pixel to new imaging plane Γ _jmapping relations (use symbol φ _ijrepresent) and the imaging plane Ω of input picture _iin each pixel be mapped to new imaging plane Γ _jafter whether visible observability operator (uses symbol m _ijrepresent).Described input picture v _iin the pixel count that comprises and reference picture v ₀middle pixel count is identical.Wherein, i ∈ [1, n], n >=1; J ∈ [1, N].

Described observability operators m _ijbe a matrix, matrix m _ijline number and columns respectively with input picture v _ithe line number of middle pixel is identical with columns; Matrix m _ijelement value be that its respective pixel of 1 expression is visible, value is that 0 expression respective pixel is invisible.

The geometry of the corresponding scene of described input picture is estimated as the pixel depth information of input picture or the pixel spatial positional information of input picture.

Step 2.3: calculate input picture imaging plane Ω by formula (4) and formula (5) _iin each pixel (use symbol Γ to the imaging plane of composograph ₀represent) mapping relations (use symbol tau _irepresent) and inverse mapping relation (use symbol beta _irepresent).Because the pixel count of input picture is all identical with the pixel count of reference picture with the pixel count of composograph, therefore x also represents the index value of pixel in input picture and composograph.

τ _i(x)＝φ _ix(x) (4)

Wherein, x represents the index value of the pixel in input picture and composograph.

${\mathrm{\β}}_i={\mathrm{\τ}}_i^{-1}---\left(5\right)$

Step 2.4: the estimation calculating composograph u according to formula (6) (uses symbol represent).

Wherein, n is the quantity of input picture; ο is fonction composition operator.

Step 2.5: calculate every width input picture v according to formula (7) _i(symbol kappa is used relative to the error factor of composograph u _irepresent).

Wherein, for gradient operator; τ _iby τ _ix vector that () is formed, x ∈ [1, N]; z _ifor input picture v _ithe depth value of middle pixel.

By error factor κ _iascendingly to sort, select the minimum M width image of error factor as final input picture, and record picture number and (use symbol l _krepresent), as shown in formula (8).

l _k＝i (8)

Wherein, k ∈ [1, M], l _k∈ [1, n].

Step 2.6: the energy term (with symbol E (u) represent) of structure as shown in formula (9).

E(u)＝E _data(u)+λE _prior(u) (9)

Wherein, E _datau () obtains by formula (10); It is drawn by the reasoning of Bayes image generation model; λ is allotment coefficient, λ > 0; E _prioru () is level and smooth item, be used for filling the part lacking input image information in composograph u, E _prioru () obtains by formula (12).

Wherein, for Gauss's sensing noise variance, it is constant; for input picture _lkimage error, its value calculates by formula (11); for observability operator;

Wherein, for geometric error for known quantity.

$E_{\mathrm{prior}}\left(u\right)={\∫}_{{\mathrm{\Γ}}_0}|\▿u|---\left(12\right)$

Step 2.7: the energy term E (u) that formula (9) constructs is minimized, solves composograph u.Be specially: the estimation of the composograph adopting step 2.4 to obtain as iteration initial value, iteratively faster collapse threshold algorithm (fast iterative shrinkage-thresholding algorithm, FISTA) algorithm is utilized to carry out iterative.

Step 3, reference picture and composograph is utilized to generate the red blue stereogram meeting image parallactic D (x).Be specially: set pixel rgb value in a reference image as (R1, G1, B1), rgb value is in the composite image (R2, G2, B2), then the rgb value of this pixel in the red blue stereogram generated is (R1, G2, B2).

Beneficial effect

The stereoscopic image generation method based on improvement Bayesian model that the present invention proposes, compared with the conventional method, conveniently has clear superiority in time performance, synthetic image quality and versatility etc.; The method intuitively can also specify parallax by pixel, is convenient to the post-processed of stereo-picture, has broad application prospects.

Accompanying drawing explanation

Fig. 1 is the flow chart based on the stereoscopic image generation method of improvement Bayesian model in the specific embodiment of the invention;

Fig. 2 is the reference picture in the specific embodiment of the invention;

Fig. 3 is the input picture v in the specific embodiment of the invention ₁;

Fig. 4 is the input picture v in the specific embodiment of the invention ₂;

Fig. 5 is the input picture v in the specific embodiment of the invention ₃;

Fig. 6 is the input picture v in the specific embodiment of the invention ₄;

Fig. 7 is the input picture v in the specific embodiment of the invention ₅;

Fig. 8 is the input picture v in the specific embodiment of the invention ₆;

Fig. 9 is the input picture v in the specific embodiment of the invention ₇;

Figure 10 is the input picture v in the specific embodiment of the invention ₈;

Figure 11 is the input picture v in the specific embodiment of the invention ₉;

Figure 12 is the composograph obtained in the specific embodiment of the invention;

Figure 13 is the red blue stereogram obtained in the specific embodiment of the invention.

Embodiment

Elaborate below in conjunction with the execution mode of accompanying drawing to the inventive method.

Adopt the stereoscopic image generation method based on improving Bayesian model to generate stereo-picture in the present embodiment, as shown in Figure 1, it comprises the following steps its operating process:

Step one, to obtain by reference to image composograph by pixel viewpoint position, reference picture is as shown in Figure 2.

Step 1.1: by formula (1) computing reference image v ₀the each pixel of clock standardization parallax d ₀(x), standardization parallax d ₀x () is by the standard depth z of pixel each in reference picture ₀x the imaging focal length f of () and reference picture obtains.

Step 1.2: given composograph u and reference picture v ₀by pixel image parallax D (x), because the viewpoint of composograph and the viewpoint of reference picture are positioned at same plane, and the imaging plane of correspondence is identical, by formula (2) by image parallactic D (x) calculate composograph relative to reference picture by pixel viewpoint displacement s (x).

Step 1.3: by formula (3) calculate composograph by pixel viewpoint position s ' (x).

Step 2, the Bayesian model passing through to improve generate composograph.

Step 2.1: for viewpoint position s ' (x) of pixel each in composograph, determines a new imaging plane Γ _j.

Step 2.2: the geometry of given scenario is estimated and 9 width input picture v _i, as shown in Fig. 3 to Figure 11, calculate the imaging plane Ω of each input picture _iin each pixel to new imaging plane Γ _jmapping relations φ _ijand the imaging plane Ω of input picture _iin each pixel be mapped to new imaging plane Γ _jafter whether visible observability operators m _ij.Described input picture v _iin the pixel count that comprises and reference picture v ₀middle pixel count is identical.Wherein, i ∈ [1,9], j ∈ [1, N].

Described observability operators m _ijbe a matrix, matrix m _ijline number and columns respectively with input picture v _ithe line number of middle pixel is identical with columns; Matrix m _ijelement value be that its respective pixel of 1 expression is visible, value is that 0 expression respective pixel is invisible.

The geometry of the corresponding scene of described input picture is estimated as the pixel depth information of input picture.

Step 2.3: calculate input picture imaging plane Ω by formula (4) and formula (5) _iin each pixel to the imaging plane Γ of composograph ₀mapping relations τ _iand inverse mapping relation β _i.

Step 2.4: the estimation calculating composograph u according to formula (6) .

Step 2.5: calculate every width input picture v according to formula (7) _irelative to the error factor κ of composograph u _i.By error factor κ _iascendingly to sort, select the minimum M width image of error factor as final input picture, and record picture number l _k=6,7,8,9

Step 2.6: the energy term E (u) of structure as shown in formula (9).

Step 2.7: the energy term E (u) that formula (9) constructs is minimized, solves composograph u.Be specially: the estimation of the composograph adopting step 2.4 to obtain as iteration initial value, FISTA algorithm is utilized to carry out iterative.First FISTA algorithm external iteration number of times is set; Keep in each external iteration process value constant, after meeting internal layer stopping criterion for iteration, upgrade value, start external iteration next time.Repeat above-mentioned iterative process, until reach the external iteration number of times of setting.

The composograph obtained as shown in figure 12.

Step 3, reference picture and composograph is utilized to generate the red blue stereogram meeting image parallactic D (x).Be specially: set pixel rgb value in a reference image as (R1, G1, B1), rgb value is in the composite image (R2, G2, B2), then the rgb value of this pixel in the red blue stereogram generated is (R1, G2, B2).The red blue stereogram finally obtained as shown in figure 13.

Claims (5)

1., based on the stereoscopic image generation method improving Bayesian model, it is characterized in that: it comprises the following steps:

Step one, to obtain by reference to image composograph by pixel viewpoint position;

Described reference picture is the true picture that a width is specified, and reference picture all pixels viewpoint position is identical;

Described composograph is the width composograph obtained by one group of input picture, and composograph specifies viewpoint position by pixel;

Step 2, the Bayesian model passing through to improve generate composograph;

Step 2.1: for viewpoint position s ' (x) of pixel each in composograph, determines a new imaging plane Γ _j;

Step 2.2: the geometry of given scenario is estimated and one group of input picture v _i, calculate the imaging plane Ω of each input picture _iin each pixel to new imaging plane Γ _jmapping relations φ _ijand the imaging plane Ω of input picture _iin each pixel be mapped to new imaging plane Γ _jafter whether visible observability operators m _ij; Described input picture v _iin the pixel count that comprises and reference picture v ₀middle pixel count is identical; Wherein, i ∈ [1, n], n>=1; J ∈ [1, N];

Step 2.3: calculate input picture imaging plane Ω by formula (4) and formula (5) _iin each pixel to the imaging plane Γ of composograph ₀mapping relations τ _iand inverse mapping relation β _i; Because the pixel count of input picture is all identical with the pixel count of reference picture with the pixel count of composograph, therefore x also represents the index value of pixel in input picture and composograph;

τ _i(x)＝φ _ix(x) (4)

Wherein, x represents the index value of the pixel in input picture and composograph;

${\mathrm{\β}}_i={\mathrm{\τ}}_i^{-1}---\left(5\right)$

Step 2.4: the estimation calculating composograph u according to formula (6)

Wherein, n is the quantity of input picture; o is fonction composition operator;

Step 2.5: calculate every width input picture v according to formula (7) _irelative to the error factor κ of composograph u _i;

Wherein, for gradient operator; τ _iby τ _ix vector that () is formed, x ∈ [1, N]; z _ifor input picture v _ithe depth value of middle pixel;

By error factor κ _jascendingly to sort, select the minimum M width image of error factor as final input picture, and record picture number l _k, as shown in formula (8);

l _k＝i (8)

Wherein, k ∈ [1, M], l _k∈ [1, n];

Step 2.6: the energy term E (u) of structure as shown in formula (9);

E(u)＝E _data(u)+λE _prior(u) (9)

Wherein, E _datau () obtains by formula (10); It is drawn by the reasoning of Bayes image generation model; λ is allotment coefficient, λ > 0; E _prioru () is level and smooth item, be used for filling the part lacking input image information in composograph u, E _prioru () obtains by formula (12);

Wherein, for Gauss's sensing noise variance, it is constant; for input picture l _kimage error, its value calculates by formula (11); for observability operator;

Wherein, for geometric error for known quantity;

$E_{\mathrm{prior}}\left(u\right)={\∫}_{{\mathrm{\Γ}}_0}|\▿u|---\left(12\right)$

Step 2.7: the energy term E (u) that formula (9) constructs is minimized, solves composograph u; Be specially: the estimation of the composograph adopting step 2.4 to obtain as iteration initial value, iteratively faster collapse threshold FISTA algorithm is utilized to carry out iterative;

Step 3, reference picture and composograph is utilized to generate the red blue stereogram meeting image parallactic D (x).

2. a kind of based on improving the stereoscopic image generation method of Bayesian model as claimed in claim 1, it is characterized in that: the concrete operations by pixel viewpoint position obtaining composograph by reference to image described in step one are:

Step 1.1: by formula (1) computing reference image v ₀the each pixel of clock standardization parallax d ₀(x), standardization parallax d ₀x () is by the standard depth z of pixel each in reference picture ₀x the imaging focal length f of () and reference picture obtains; Wherein, x is reference picture v ₀the index value of middle pixel, x ∈ [1, N], N are reference picture v ₀the quantity of middle pixel;

$d_0\left(x\right)=\frac{f}{z_0\left(x\right)}---\left(1\right)$

Step 1.2: given composograph u and reference picture v ₀by pixel image parallax D (x), because the viewpoint of composograph and the viewpoint of reference picture are positioned at same plane, and the imaging plane of correspondence is identical, by formula (2) by image parallactic D (x) calculate composograph relative to reference picture by pixel viewpoint displacement s (x);

$\mathrm{\Δs}\left(x\right)=\frac{D\left(x\right)}{d_0\left(x\right)}---\left(2\right)$

Step 1.3: by formula (3) calculate composograph by pixel viewpoint position s ' (x);

s′(x)＝s+Δs(x) (3)

Wherein, s is the viewpoint position of reference picture.

3. a kind of based on improving the stereoscopic image generation method of Bayesian model as claimed in claim 1 or 2, it is characterized in that: observability operators m described in step 2 step 2.2 _ijbe a matrix, matrix m _ijline number and columns respectively with input picture v _ithe line number of middle pixel is identical with columns; Matrix m _ijelement value be that its respective pixel of 1 expression is visible, value is that 0 expression respective pixel is invisible.

4. a kind of based on improving the stereoscopic image generation method of Bayesian model as claimed in claim 1 or 2, it is characterized in that: the geometry of the corresponding scene of input picture described in step 2 step 2.2 is estimated as the pixel depth information of input picture or the pixel spatial positional information of input picture.

5. a kind of based on improving the stereoscopic image generation method of Bayesian model as claimed in claim 1 or 2, it is characterized in that: the concrete operations utilizing reference picture and composograph generation to meet the red blue stereogram of image parallactic D (x) described in step 3 are: set pixel rgb value in a reference image as (R1, G1, B1), rgb value is in the composite image (R2, G2, B2), then the rgb value of this pixel in the red blue stereogram generated is (R1, G2, B2).