CN102547338A - DIBR (Depth Image Based Rendering) system suitable for 3D (Three-Dimensional) television - Google Patents

Abstract

The invention discloses a DIBR (Depth Image Based Rendering) system suitable for a 3D (Three-Dimensional) television. A parameter setting module is used for storing parameters required for 3D image conversion and hole filling, and then, a painter's algorithm module is used for obtaining a scanning sequence during 3D image conversion, namely, the drawing sequence of a target image, so as to prevent a fold phenomenon occurring in the target image; a 3D image conversion module is use for carrying out 3D image conversion according to the parameters stored by the parameter setting module and the scanning sequence to obtain the target image and a disparity map of the target image; a median filter module is used for filtering the disparity map and then correcting a matching error of the target image according to the corrected disparity map; and finally, a hole filling module is used for filling holes for the target image according to the corrected disparity map so as to obtain the target image with reduced holes and folds.

Description

A kind of DIBR system that is applicable to the 3D TV

Technical field

The invention belongs to 3D television video post-processing technology field, more specifically, relate to a kind of DIBR system of the 3D of being applicable to TV

Background technology

Drawing (Depth-Image-Based Rendering is called for short DIBR) technology based on depth image is the key technology in the 3D television system, has caused the concern of a lot of research institutions.

The function that the DIBR system mainly accomplishes is to utilize reference picture and corresponding depth image thereof to synthesize the view of new viewpoint; Be target image (Destination Image); Thereby constitute stereo-picture to (stereo pair), its resulting target image is the postrun result of series of algorithms.

Need transmit left eye and right eye two-path video stream during traditional 3D video signal transmission, and only need transmission of one line video flowing and corresponding depth information, thereby can reduce transmission bandwidth based on the 3D television system of DIBR technology.Simultaneously, adopt the DIBR technology can realize the conversion of 2D-3D video easily, support various auto-stereoscopic displays easily.Just because of these advantages, at present a lot of scientific research institutions are all in research DIBR technology.For example, the Nerola system is the 3D television system of the present business-like DIBR of having function.

Yet present DIBR system can produce cavity (Holes) and fold (Folds) phenomenon when generating target image.Eliminate cavity and fold and be and improve the quality of synthesizing view, make it move towards the key of practicability.

Summary of the invention

The objective of the invention is to overcome the deficiency of prior art, a kind of DIBR system of the 3D of being applicable to TV is provided, this system can reduce the pseudomorphism that brings because of empty and fold preferably.

For realizing above-mentioned purpose, the present invention is applicable to the DIBR system of 3D TV, it is characterized in that, comprising:

One parameter is provided with module, is used to deposit the 3-D view conversion and fills required parameter, i.e. the sequence number n of target image with the cavity; N is an integer, and span is [4,4]; N=0 representes reference-view itself, and n＜0 expression target image is positioned at the reference picture left side, is left view; N＞0 expression target image is positioned at the reference picture right side, is right view; Proportional factor r, its span are [0,1542]; Parallax free plane corresponding gray scale value D _Zps, its span is [0,255]; The threshold value len_bighole that counts, its span is [0,7]; Expansion pixel number l, its span is [0,7];

One painter's algorithm module meets target image sequence n, when being used for confirming the 3-D view conversion, to the scanning sequency of reference picture; N＜0 expression target image is positioned at the reference picture left side, and n＞0 expression target image is positioned at the reference picture right side;

When target image was positioned at the reference picture right side, scanning sequency was for from left to right, from top to bottom; When target image was positioned at the reference picture left side, scanning sequency was for from right to left, from top to bottom;

The 3-D view conversion module is used for being provided with according to parameter parameter, scanning sequency and the depth image of module stores, to the reference picture I of input _RefCarry out three-dimension varying:

According to the scanning sequency that the painter's algorithm module is confirmed, traversal reference picture I _RefIn pixel, and once handle, for reference picture I _RefIn v _RefRow u _RefThe pixel u of row _Ref, at first calculate it at target image I according to formula (1) _DesMiddle corresponding matched pixel point u _DesWherein

$\left\{\begin{array}{c}{u}_{\mathrm{des}}-u_{\mathrm{ref}}=n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.,---\left(1\right)$

Wherein, u _Ref, v _RefBe respectively reference picture I in the image pixel coordinate system _RefThe horizontal coordinate of pixel and vertical coordinate, u _Des, v _DesBe respectively target image I in the image pixel coordinate system _DesIn the horizontal coordinate of corresponding reference picture pixel, D (u _Ref, v _Ref) expression depth image in pixel (u _Ref, v _Ref) gray value;

Then, pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _Ref

Obtain target image I at last _DesAnd disparity map M;

One medium filtering module is used to receive 3-D view conversion export target image I _DesAnd disparity map M, carry out following processing then: (1), disparity map M is carried out medium filtering; (2), according to filtered disparity map M, obtain target image I _DesPixel is at reference picture I _RefOn the match point coordinate; (3), judge whether the match point coordinate drops on reference picture I _RefIn, if, then with reference picture I _RefThe pixel value at last match point coordinate place copies target image I to _DesCorrespondence position, otherwise, do not handle;

One empty packing module is used to receive disparity map M and the target image I that the medium filtering submodule is exported _DesAnd scanning sequency and the parameter exported according to the painter's algorithm module are provided with the parameter in the module, filling target image I _DesThe middle cavity that exists:

(1), detects than macroscopic-void

With from left to right order traversal disparity map M,, then think it is here by row, note this starting point and terminal point than macroscopic-void than macroscopic-void if occur threshold value count len_bighole and above empty point continuously;

(2), expand than macroscopic-void

According to the scanning sequency of painter's algorithm module output, distinguish target image I _DesBe left view or right view, if left view, execution in step a); Like figure is right view, execution in step b);

A), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: among the disparity map M being the center than the macroscopic-void zone; The parallax value that detects from left to right than the non-cavity of macroscopic-void right hand edge point changes; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is when smaller value jumps to higher value, is the background pixel point than macroscopic-void right hand edge zone, and notes this big parallax value and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is to jump to smaller value from higher value; Perhaps parallax value is consistent when not having transition; Than macroscopic-void right hand edge zone is the foreground pixel point, and notes the parallax value of putting in first non-cavity and put parallax value as foreground pixel;

Empty edge then expands: if be the background pixel point than macroscopic-void right hand edge zone; Be the matching error zone then than the macroscopic-void left and right edges; Need expand, will deduct l pixel and terminal point abscissa than the starting point abscissa of macroscopic-void and add starting point and the terminal point of l pixel as the cavity that needs in the target image to fill; If than macroscopic-void right hand edge zone is the foreground pixel point; Then the left hand edge than macroscopic-void is the matching error zone; Only expand, will deduct l pixel and terminal point than the starting point abscissa of macroscopic-void and remain unchanged as the starting point and the terminal point in the cavity that needs in the target image to fill than the left hand edge of macroscopic-void;

B), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: in the disparity map being the center than the macroscopic-void zone; The parallax value that detects from right to left than the non-cavity of macroscopic-void left hand edge point changes; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is when higher value jumps to smaller value, is the background pixel point than macroscopic-void left hand edge zone, and notes this less parallax value and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is to jump to higher value from smaller value; Perhaps parallax value is consistent does not have transition; Than macroscopic-void left hand edge zone is the foreground pixel point, and notes the parallax value of putting in first non-cavity and put parallax value as foreground pixel;

Expand then than the macroscopic-void edge: if be the background pixel point than macroscopic-void left hand edge zone; Be the matching error zone then than the macroscopic-void left and right edges; Need expand, will deduct l pixel and terminal point abscissa than the starting point abscissa of macroscopic-void and add starting point and the terminal point of l pixel as the cavity that needs in the target image to fill; If than macroscopic-void left hand edge zone is the foreground pixel point; Then the right hand edge than macroscopic-void is the matching error zone; Only expand, will remain unchanged than the starting point of macroscopic-void and the terminal point abscissa adds that l pixel is as the starting point and the terminal point that need the cavity of filling in the target image than the right hand edge of macroscopic-void;

In this step (2), the pixel number l of expansion is the number of mistake matched pixel point, and is relevant with the accuracy that depth image is estimated, can regulate according to the target image quality that generates, and span is 0-7;

(3), fill than macroscopic-void

With target image I _DesThe middle starting point in the cavity of filling, the parallax value that the terminal point abscissa all deducts foreground pixel point of needing, obtaining need be at reference picture I _RefIn the pixel zone of duplicating, then, from reference picture I _RefIn duplicate this regional pixel and be filled into target image I _DesThe middle hole region that needs filling obtains the final objective image I _Des

Goal of the invention of the present invention is achieved in that

The present invention is applicable to the DIBR system of 3D TV; Through parameter module being set stores 3-D view conversion and the cavity required parameter of filling; Obtain the scanning of 3-D view conversion then through the painter's algorithm module; Be the drawing order of target image, with the fold phenomenon of avoiding occurring in the target image; The 3-D view conversion module carries out the 3-D view conversion according to parameter and scanning sequency that parameter is provided with module stores, obtains target image and disparity map thereof; The medium filtering module is carried out filtering to disparity map; According to the disparity map after proofreading and correct the matching error of target image is proofreaied and correct then, last, empty packing module is according to proofreading and correct disparity map; Target image is carried out the cavity fill, obtain the target image that cavity and fold reduce.

Description of drawings

Fig. 1 is the DIBR system one embodiment theory diagram that the present invention is applicable to the 3D TV.

Embodiment

Describe below in conjunction with the accompanying drawing specific embodiments of the invention, so that those skilled in the art understands the present invention better.What need point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.

Fig. 1 is the DIBR system one embodiment theory diagram that the present invention is applicable to the 3D TV;

As shown in Figure 1, in the present embodiment, be applicable to that the DIBR system of 3D TV comprises that degree of depth pretreatment module 1, parameter are provided with module 2, painter's algorithm module 3,3-D view conversion module 4, medium filtering module 5 and empty packing module 6.

Degree of depth pretreatment module 1 is used for depth image D is carried out smoothing processing, and to reduce the cavity of target image, its processing mode can have several different methods, like 4 neighborhood calculating etc.

It is the whole DIBR of completion systems that parameter is provided with module 2 functions, comprises each module parameter setting.Why DIBR of the present invention system is placed on a module with all parameters is provided with, and is because the distribution of this corresponding register address of parameter when helping hardware and realizing.The new target image of each drafting, the DIBR system all can first call parameters be provided with module.This moment, the user can be arranged to identical value last time with parameter, promptly kept parameter value constant, also can change some parameter value as required.To same width of cloth target image, its employed parameter of each pixel all is the same; And different target images, employed parameter value can be inequality.

Parameter is provided with module 2 and deposits the required parameter of 3-D view conversion: the sequence number n of target image, n are integer, and span is [4,4], and n=0 representes reference-view itself, and n＜0 expression target image is positioned at the reference picture left side, is left view; N＞0 expression target image is positioned at the reference picture right side, is right view; Proportional factor r, its span are [0,1542]; Parallax free plane corresponding gray scale value D _Zps, its span is [0,255].Required parameter is filled in the cavity: the threshold value len_bighole that counts, and its span is [0,7]; Expansion pixel number l, its span is [0,7]; Parallax value transition threshold value sharp_th, this value is used for fill judging that in the cavity higher value jumps to smaller value or smaller value jumps to higher value, surpasses this parallax value transition threshold value sharp_th and think that then transition is arranged.

In the present invention, the 3-D view transformation for mula is:

$\left\{\begin{array}{c}{u}_{\mathrm{des}}-u_{\mathrm{ref}}=n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.,---\left(1\right)$

Wherein, n is an integer, the sequence number of the target image of indicating to generate, and span is [4,4] in DIBR of the present invention system.N=0 representes reference-view itself; N＜0 expression target image is positioned at the reference picture left side, is left view; N＞0 expression target image is positioned at the reference picture right side, is right view.R is a scale factor, and its span is [0,1542].D _ZpsBe parallax free plane corresponding gray scale value, span is [0,255].u _RefBe the horizontal coordinate of reference picture pixel in the image pixel coordinate system, v _RefThe vertical coordinate of reference picture pixel in the image pixel coordinate system, u _DesBe the horizontal coordinate of corresponding reference picture pixel in the target image in the image pixel coordinate system, v _DesBe the vertical coordinate of corresponding reference picture pixel in the target image in the image pixel coordinate system, D (u _Ref, v _Ref) expression reference picture in pixel (u _Ref, v _Ref) the corresponding gray scale value.

Formula (1) is actually right corresponding pixel points level of stereo-picture and the relational expression between vertical coordinate, thus the formula target image I that can ask _DesThe coordinate figure of last corresponding pixel points.

In the present invention, parameter is provided with the depth adjustment that module 1 is used for the 3-D view conversion, through changing the right horizontal sensing parallax D of stereo-picture _c, D _c=u _Des-u _RefPerceived depth when regulating the image demonstration.

In the present invention, horizontal sensing parallax D _c=u _Ref-v _RefThe standard span does

-(W _i×5％)≤D _c≤W _i×5％， (2)

Wherein, W _iBe reference picture I _RefWidth.D _cAnd W _iUnit all be pixel.

Formula (1) substitution formula (2) is had:

$-W_i\·5\%\≤n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}})){\≤W}_i\·5\%,---\left(3\right)$

If the user uses n, r, the D of DIBR system default _ZpsParameter value, system default n, r, D _ZpsThe ginseng value is respectively 1,326,130, and inequality (3) is set up all the time.

If the user does not use n, r, the D of DIBR system default _ZpsParameter value carries out depth adjustment and changes n, r, D _ZpsAny one or more value in three parameters changes multiple parameter values branch multistep and carries out, and all allows to revise a parameter value at every turn, all need satisfy inequality (3) and could obtain euphorosia degree preferably.

When carrying out depth adjustment, be the shortcut calculation logic, once only allow to change a parameter value.When spectators regulate parameter n or D _ZpsAfter, formula (3) is when being false, and whether this module will provide a suggestion r value and supply spectators to select to adopt, and after spectators regulated parameter r, formula (3) was when being false, and this module will provide a suggestion D _ZpsWhether value supplies spectators to select to adopt.

When generating the new viewpoint view, might be with reference picture I _RefIn several pixels be mapped to target image I _DesSame point on, Here it is changes caused " fold (folds) " phenomenon by observability.The conventional method of handling the fold phenomenon is the Z-buffer algorithm, and this method need compare the depth value of each pixel, thereby render speed is slow.And painter's algorithm of the present invention is through confirming the scanning sequency of reference picture, and assurance can be drawn earlier from virtual photocentre pixel far away, behind the near point of virtual photocentre, draws, thereby sets up correct hiding relation.

Painter's algorithm module 3 receives the target image sequence n that parameter is provided with module 2 outputs, when being used for confirming the 3-D view conversion, to the scanning sequency of reference picture; N＜0 expression target image is positioned at the reference picture left side, and n＞0 expression target image is positioned at the reference picture right side;

When target image was positioned at the reference picture right side, scanning sequency was for from left to right, from top to bottom; When target image was positioned at the reference picture left side, scanning sequency was for from right to left, from top to bottom.

The scanning sequency that 3-D view conversion module 4 is confirmed according to the painter's algorithm module, traversal reference picture I _RefIn pixel u _Ref, obtain the matched pixel point according to formula (1), then with pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des～u _Ref, obtain target image I _DesAnd disparity map M.

Because a variety of causes such as inaccurate of the inaccuracy that observability changes, calculate, depth image, the target image I that generates by the DIBR technology _DesIn may comprise many mistakes matched pixel points.We are called matching error (matching error) with this mistake.Matching error will seriously reduce the right quality of stereo-picture, be mingled with some background pixels in the prospect in the object of an integral body, cause people's uncomfortable feeling.Therefore, how removing matching error is a major issue in the DIBR technology.

Though traditional filtering method can be proofreaied and correct matching error effectively, amount of calculation is big, and causes the fuzzy of target image easily.Medium filtering module 5 of the present invention is carried out medium filtering based on disparity map and is proofreaied and correct, and origin is accomplished, and specifically is: (1), disparity map M is carried out medium filtering; (2), according to filtered disparity map M, obtain target image I _DesPixel is at reference picture I _RefOn the match point coordinate; (3), judge whether the match point coordinate drops on reference picture I _RefIn, if, then with reference picture I _RefThe pixel value at last match point coordinate place copies target image I to _DesCorrespondence position, otherwise, do not handle.

Compare traditional filtering method, adopt processing of the present invention that following advantage is arranged: 1) owing to be that disparity map is carried out filtering, rather than to target image, so can not cause bluring of image; 2) amount of calculation reduces significantly.

When utilizing DIBR technology to generate new view,, can in new view, produce than macroscopic-void owing to reasons such as the depth value of depth image are discontinuous.This paper has proposed a kind of empty filling algorithm based on disparity map.This algorithm at first detects than macroscopic-void according to the disparity map of input, utilizes disparity map to remove matching error then, adopts the background pixel point filling cavity in the reference picture at last.Experiment shows that this algorithm can be filled the cavity in the new view preferably, and compares with the depth image preprocess method, can not reduce the picture quality of non-hole region.

Cavity packing module 6 receives the disparity map M and the target image I of medium filtering submodule output _DesAnd scanning sequency and the parameter exported according to the painter's algorithm module are provided with the parameter in the module, filling target image I _DesThe middle cavity that exists obtains the final objective image I _Des

Along with the development of 3D TV tech, it has been recognized that the many facilities that adopt the DIBR technology to bring.The DIBR technology has become the key technology in the 3D television system.The present invention proposes a DIBR system, it is made up of basic module such as parameter setting, painter's algorithm, 3-D view conversion, cavity filling, medium filtering and depth image preliminary treatment expansion module.The parameter of system is provided with module and supports the depth adjustment function, therefore draws the degree of depth of the stereo-picture that obtains and can regulate according to spectators' different needs.Experiment shows, it is better that the right image quality of the stereo-picture that obtains is drawn by this system, is applicable in the 3D television system of functions such as having 2D-3D conversion, free stereo demonstration.

What need to say is, the present invention is based on that present popular image resolution ratio 1024*768 describes, in the DIBR system that can expand to other resolution of the present invention.

Although above the illustrative embodiment of the present invention is described; So that the technical staff in present technique field understands the present invention, but should be clear, the invention is not restricted to the scope of embodiment; To those skilled in the art; As long as various variations appended claim limit and the spirit and scope of the present invention confirmed in, these variations are conspicuous, all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (1)

1. a DIBR system that is applicable to the 3D TV is characterized in that, comprising:

One parameter is provided with module, is used to deposit the 3-D view conversion and fills required parameter, i.e. the sequence number n of target image with the cavity; N is an integer, and span is [4,4]; N=0 representes reference-view itself, and n＜0 expression target image is positioned at the reference picture left side, is left view; N＞0 expression target image is positioned at the reference picture right side, is right view; Proportional factor r, its span are [0,1542]; Parallax free plane corresponding gray scale value D _Zps, its span is [0,255]; The threshold value len_bighole that counts, its span is [0,7]; Expansion pixel number l, its span is [0,7];

One painter's algorithm module meets target image sequence n, when being used for confirming the 3-D view conversion, to the scanning sequency of reference picture; N＜0 expression target image is positioned at the reference picture left side, and n＞0 expression target image is positioned at the reference picture right side;

When target image was positioned at the reference picture right side, scanning sequency was for from left to right, from top to bottom; When target image was positioned at the reference picture left side, scanning sequency was for from right to left, from top to bottom;

The 3-D view conversion module is used for being provided with according to parameter the depth image of parameter, scanning sequency and the input of module stores, to the reference picture I of input _RefCarry out three-dimension varying:

According to the scanning sequency that the painter's algorithm module is confirmed, traversal reference picture I _RefIn pixel, and once handle, for reference picture I _RefIn v _RefRow u _RefThe pixel u of row _Ref, at first calculate it at target image I according to formula (1) _DesMiddle corresponding matched pixel point u _DesWherein

$\left\{\begin{array}{c}{u}_{\mathrm{des}}-u_{\mathrm{ref}}=n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.,---\left(1\right)$

Wherein, u _Ref, v _RefBe respectively reference picture I in the image pixel coordinate system _RefThe horizontal coordinate of pixel and vertical coordinate, u _Des, v _DesBe respectively target image I in the image pixel coordinate system _DesIn the horizontal coordinate of corresponding reference picture pixel, D (u _Ref, v _Ref) expression depth image in pixel (u _Ref, v _Ref) gray value;

Then, pixel u _RefPixel value copy pixel u to _Des, with the element (u among the disparity map M _Ref, v _Ref) be changed to u _Des-u _Ref

Obtain target image I at last _DesAnd disparity map M;

One medium filtering module is used to receive 3-D view conversion export target image I _DesAnd disparity map M, carry out following processing then: (1), disparity map M is carried out medium filtering; (2), according to filtered disparity map M, obtain target image I _DesPixel is at reference picture I _RefOn the match point coordinate; (3), judge whether the match point coordinate drops on reference picture I _RefIn, if, then with reference picture I _RefThe pixel value at last match point coordinate place copies target image I to _DesCorrespondence position, otherwise, do not handle;

One empty packing module is used to receive disparity map M and the target image I that the medium filtering submodule is exported _DesAnd scanning sequency and the parameter exported according to the painter's algorithm module are provided with the parameter in the module, filling target image I _DesThe middle cavity that exists:

(1), detects than macroscopic-void

With from left to right order traversal disparity map M,, then think it is here by row, note this starting point and terminal point than macroscopic-void than macroscopic-void if occur threshold value count len_bighole and above empty point continuously;

(2), expand than macroscopic-void

According to the scanning sequency of painter's algorithm module output, distinguish target image I _DesBe left view or right view, if left view, execution in step a); Like figure is right view, execution in step b);

A), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: among the disparity map M being the center than the macroscopic-void zone; The parallax value that detects from left to right than the non-cavity of macroscopic-void right hand edge point changes; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is when smaller value jumps to higher value, is the background pixel point than macroscopic-void right hand edge zone, and notes this big parallax value and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is to jump to smaller value from higher value; Perhaps parallax value is consistent when not having transition; Than macroscopic-void right hand edge zone is the foreground pixel point, and notes the parallax value of putting in first non-cavity and put parallax value as foreground pixel;

Empty edge then expands: if be the background pixel point than macroscopic-void right hand edge zone; Be the matching error zone then than the macroscopic-void left and right edges; Need expand, will deduct l pixel and terminal point abscissa than the starting point abscissa of macroscopic-void and add starting point and the terminal point of l pixel as the cavity that needs in the target image to fill; If than macroscopic-void right hand edge zone is the foreground pixel point; Then the left hand edge than macroscopic-void is the matching error zone; Only expand, will deduct l pixel and terminal point than the starting point abscissa of macroscopic-void and remain unchanged as the starting point and the terminal point in the cavity that needs in the target image to fill than the left hand edge of macroscopic-void;

B), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: in the disparity map being the center than the macroscopic-void zone; The parallax value that detects from right to left than the non-cavity of macroscopic-void left hand edge point changes; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is when higher value jumps to smaller value, is the background pixel point than macroscopic-void left hand edge zone, and notes this less parallax value and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places; And transition is to jump to higher value from smaller value; Perhaps parallax value is consistent does not have transition; Than macroscopic-void left hand edge zone is the foreground pixel point, and notes the parallax value of putting in first non-cavity and put parallax value as foreground pixel;

Expand then than the macroscopic-void edge: if be the background pixel point than macroscopic-void left hand edge zone; Be the matching error zone then than the macroscopic-void left and right edges; Need expand, will deduct l pixel and terminal point abscissa than the starting point abscissa of macroscopic-void and add starting point and the terminal point of l pixel as the cavity that needs in the target image to fill; If than macroscopic-void left hand edge zone is the foreground pixel point; Then the right hand edge than macroscopic-void is the matching error zone; Only expand, will remain unchanged than the starting point of macroscopic-void and the terminal point abscissa adds that l pixel is as the starting point and the terminal point that need the cavity of filling in the target image than the right hand edge of macroscopic-void;

In this step (2), the pixel number l of expansion is the number of mistake matched pixel point, and is relevant with the accuracy that depth image is estimated, can regulate according to the target image quality that generates, and span is 0-7;

(3), fill than macroscopic-void

With target image I _DesThe middle starting point in the cavity of filling, the parallax value that the terminal point abscissa all deducts foreground pixel point of needing, obtaining need be at reference picture I _RefIn the pixel zone of duplicating, then, from reference picture I _RefIn duplicate this regional pixel and be filled into target image I _DesThe middle hole region that needs filling obtains the final objective image I _Des

Images