CN102547338B - DIBR (Depth Image Based Rendering) system suitable for 3D (Three-Dimensional) television - Google Patents

Abstract

The invention discloses a DIBR (Depth Image Based Rendering) system suitable for a 3D (Three-Dimensional) television. A parameter setting module is used for storing parameters required for 3D image conversion and hole filling, and then, a painter's algorithm module is used for obtaining a scanning sequence during 3D image conversion, namely, the drawing sequence of a target image, so as to prevent a fold phenomenon occurring in the target image; a 3D image conversion module is use for carrying out 3D image conversion according to the parameters stored by the parameter setting module and the scanning sequence to obtain the target image and a disparity map of the target image; a median filter module is used for filtering the disparity map and then correcting a matching error of the target image according to the corrected disparity map; and finally, a hole filling module is used for filling holes for the target image according to the corrected disparity map so as to obtain the target image with reduced holes and folds.

Description

A kind of DIBR system that is applicable to the 3D TV

Technical field

The invention belongs to 3D television video post-processing technology field, more specifically say, relate to a kind of DIBR system of the 3D of being applicable to TV

Background technology

Drawing (Depth-Image-Based Rendering is called for short DIBR) technology based on depth image is the key technology in the 3D television system, has caused the concern of a lot of research institutions.

The function that the DIBR system mainly completes is to utilize the view of the range image integration new viewpoint of reference picture and correspondence thereof, be target image (Destination Image), thereby consist of stereo-picture to (stereo pair), its resulting target image is the postrun result of series of algorithms.

Need to transmit left eye and right eye two-path video stream during traditional 3D video signal transmission, and only need transmission of one line video flowing and corresponding depth information based on the 3D television system of DIBR technology, thereby can reduce transmission bandwidth.Simultaneously, adopt the DIBR technology can realize easily the conversion of 2D-3D video, support easily various auto-stereoscopic displays.Just because of these advantages, at present a lot of scientific research institutions are all in research DIBR technology.For example, the Nerola system is business-like 3D television system with the DIBR function at present.

Yet present DIBR system can produce cavity (Holes) and fold (Folds) phenomenon when generating target image.Eliminate cavity and fold and be and improve the quality of synthesizing view, make it move towards practical key.

Summary of the invention

The object of the invention is to overcome the deficiencies in the prior art, a kind of DIBR system of the 3D of being applicable to TV is provided, this system can reduce the pseudomorphism that brings because of empty and fold preferably.

For achieving the above object, the present invention is applicable to the DIBR system of 3D TV, it is characterized in that, comprising:

One parameter arranges module, is used for depositing 3-D view conversion and the required parameter of cavity filling, i.e. the sequence number n of target image, n is integer, and span is [4,4], n=0 represents reference-view itself, and n＜0 expression target image is positioned on the left of reference picture, is left view; N〉0 expression target image is positioned at the reference picture right side, is right view; Proportional factor r, its span are [0,1542]; The corresponding gray value D in parallax free plane _zps, its span is [0,255]; The threshold value len_bighole that counts, its span is [0,7]; Expansion pixel number l, its span is [0,7];

One painter's algorithm module meets target image sequence n, when being used for determining the 3-D view conversion, to the scanning sequency of reference picture; N＜0 expression target image is positioned at reference picture left side, n〉0 expression target image is positioned at the reference picture right side;

When target image was positioned at the reference picture right side, scanning sequency was for from left to right, from top to bottom; When target image was positioned at the reference picture left side, scanning sequency was for from right to left, from top to bottom;

The 3-D view conversion module is for parameter, scanning sequency and the depth image of module stores are set according to parameter, to the reference picture I of input _refCarry out three-dimension varying:

According to the scanning sequency that the painter's algorithm module is determined, traversal reference picture I _refIn pixel, and once process, for reference picture I _refIn v _refRow u _refThe pixel p of row _ref, at first calculate it at target image I according to formula (1) _desThe matched pixel point p of middle correspondence _desWherein

$\left\{\begin{array}{c}{u}_{\mathrm{des}}-u_{\mathrm{ref}}=n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.,---\left(1\right)$

Wherein, u _ref, v _refBe respectively reference picture I in the image pixel coordinate system _refThe horizontal coordinate of pixel and vertical coordinate, u _des, v _desBe respectively target image I in the image pixel coordinate system _desThe horizontal coordinate of middle corresponding reference picture pixel, D (u _ref, v _ref) expression depth image in pixel (u _ref, v _ref) gray value;

Then, pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _ref

Obtain at last target image I _desAnd disparity map M;

One medium filtering module is used for receiving 3-D view conversion export target image I _desAnd disparity map M, then carry out following processing: (1), disparity map M is carried out medium filtering; (2), according to filtered disparity map M, obtain target image I _desPixel is at reference picture I _refOn the match point coordinate; (3), judge whether the match point coordinate drops on reference picture I _refIn, if so, with reference to image I _refThe pixel value at upper match point coordinate place copies target image I to _desCorrespondence position, otherwise, do not process;

One empty packing module is used for receiving disparity map M and the target image I that the medium filtering submodule is exported _desAnd according to the scanning sequency of painter's algorithm module output and parameter, parameter in module is set, fill target image I _desThe cavity of middle existence:

(1), detect than macroscopic-void

With from left to right order traversal disparity map M, if threshold value count len_bighole and above empty point occur continuously, think it is than macroscopic-void herein by row, record this than starting point and the terminal point of macroscopic-void;

(2), expand than macroscopic-void

According to the scanning sequency of painter's algorithm module output, distinguish target image I _desBe left view or right view, if left view, execution in step a); Right view as figure, execution in step b);

A), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: in disparity map M than centered by the macroscopic-void zone, the parallax value that detects from left to right than the non-empty point of macroscopic-void right hand edge changes, when parallax value transition occurs for the first time at two in succession non-empty some places, and transition is to jump to when large from little, be the background pixel point than macroscopic-void right hand edge zone, and record this larger parallax value and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places, and transition is little from jumping to greatly, when perhaps parallax value does not unanimously have transition, be the foreground pixel point than macroscopic-void right hand edge zone, and parallax value of recording first non-cavity point is put parallax value as foreground pixel;

Then empty edge expands: if be the background pixel point than macroscopic-void right hand edge zone, be the matching error zone than the macroscopic-void left and right edges, need to expand, will deduct than the starting point abscissa of macroscopic-void l pixel and terminal point abscissa and add that l pixel is as starting point and the terminal point in the cavity that needs in target image to fill; If be the foreground pixel point than macroscopic-void right hand edge zone, the left hand edge than macroscopic-void is the matching error zone, only expand than the left hand edge of macroscopic-void, will deduct l pixel than the starting point abscissa of macroscopic-void and terminal point remains unchanged as starting point and the terminal point in the cavity that needs in target image to fill;

B), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: in disparity map than centered by the macroscopic-void zone, the parallax value that detects from right to left than the non-empty point of macroscopic-void left hand edge changes, when parallax value transition occurs for the first time at two in succession non-empty some places, and transition is from jumping to greatly hour, be the background pixel point than macroscopic-void left hand edge zone, and record this and neglect difference and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places, and transition is to jump to greatly from little, perhaps parallax value does not unanimously have transition, is the foreground pixel point than macroscopic-void left hand edge zone, and parallax value of recording first non-cavity point is put parallax value as foreground pixel;

Then expand than the macroscopic-void edge: if be the background pixel point than macroscopic-void left hand edge zone, be the matching error zone than the macroscopic-void left and right edges, need to expand, will deduct than the starting point abscissa of macroscopic-void l pixel and terminal point abscissa and add that l pixel is as starting point and the terminal point in the cavity that needs in target image to fill; If be the foreground pixel point than macroscopic-void left hand edge zone, the right hand edge than macroscopic-void is the matching error zone, only expand than the right hand edge of macroscopic-void, will remain unchanged than the starting point of macroscopic-void adds that with the terminal point abscissa l pixel is as starting point and the terminal point in the cavity that needs in target image to fill;

In this step (2), the pixel number l of expansion is the number of mistake matched pixel point, and is relevant with the accuracy that depth image is estimated, can regulate according to the target image quality that generates, and span is 0-7;

(3), fill than macroscopic-void

With target image I _desThe middle starting point in the cavity of filling, the parallax value that the terminal point abscissa all deducts foreground pixel point of needing, obtaining need to be at reference picture I _refIn the pixel zone of copying, then, from reference picture I _refIn copy this regional pixel and be filled into target image I _desThe middle hole region that needs filling obtains final target image I _des

Goal of the invention of the present invention is achieved in that

The present invention is applicable to the DIBR system of 3D TV, by parameter, module is set required Parameter storage is filled in 3-D view conversion and cavity, then obtain the scanning of 3-D view conversion by the painter's algorithm module, be the drawing order of target image, with the fold phenomenon of avoiding occurring in target image; The 3-D view conversion module carries out the 3-D view conversion according to parameter and scanning sequency that parameter arranges module stores, obtains target image and disparity map thereof; The medium filtering module is carried out filtering to disparity map, then according to the disparity map after proofreading and correct, the matching error of target image is proofreaied and correct, last, empty packing module is according to proofreading and correct disparity map, target image is carried out the cavity fill, obtain the target image that cavity and fold reduce.

Description of drawings

Fig. 1 is the DIBR system one embodiment theory diagram that the present invention is applicable to the 3D TV.

Embodiment

Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described, so that those skilled in the art understands the present invention better.What need to point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.

Fig. 1 is the DIBR system one embodiment theory diagram that the present invention is applicable to the 3D TV;

As shown in Figure 1, in the present embodiment, the DIBR system that is applicable to the 3D TV comprises that degree of depth pretreatment module 1, parameter arrange module 2, painter's algorithm module 3,3-D view conversion module 4, medium filtering module 5 and empty packing module 6.

Degree of depth pretreatment module 1 is used for depth image D is carried out smoothing processing, and to reduce the cavity of target image, its processing mode can have several different methods, as 4 neighborhood calculating etc.

It is to complete whole DIBR system that parameter arranges module 2 functions, comprises the setting of modules parameter.Why DIBR of the present invention system is placed on a module with all parameters arranges, and is because the distribution of this register address corresponding to parameter when being conducive to hardware and realizing.Each draw new target image, the DIBR system all can first call parameters arrange module.This moment, the user can be arranged to parameter identical value last time, namely kept parameter value constant, also can change as required some parameter value.To same width target image, the parameter that its each pixel uses is the same; And different target images, the parameter value that uses can be not identical.

Parameter arranges module 2 and deposits the required parameter of 3-D view conversion: the sequence number n of target image, n are integer, and span is [4,4], and n=0 represents reference-view itself, and n＜0 expression target image is positioned on the left of reference picture, is left view; N〉0 expression target image is positioned at the reference picture right side, is right view; Proportional factor r, its span are [0,1542]; The corresponding gray value D in parallax free plane _zps, its span is [0,255].Required parameter is filled in the cavity: the threshold value len_bighole that counts, and its span is [0,7]; Expansion pixel number l, its span is [0,7]; Parallax value transition threshold value sharp_th, this value is used for judging that in the cavity filling higher value jumps to smaller value or smaller value jumps to higher value, surpasses this parallax value transition threshold value sharp_th and think that transition is arranged.

In the present invention, the 3-D view transformation for mula is:

$\left\{\begin{array}{c}{u}_{\mathrm{des}}-u_{\mathrm{ref}}=n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.,---\left(1\right)$

Wherein, n is integer, the sequence number of the target image of indicating to generate, and span is [4,4] in DIBR of the present invention system.N=0 represents reference-view itself; N＜0 expression target image is positioned on the left of reference picture, is left view; N〉0 expression target image is positioned at the reference picture right side, is right view.R is scale factor, and its span is [0,1542].D _zpsBe the corresponding gray value in parallax free plane, span is [0,255].u _refBe the horizontal coordinate of reference picture pixel in the image pixel coordinate system, v _refThe vertical coordinate of reference picture pixel in the image pixel coordinate system, u _desBe the horizontal coordinate of corresponding reference picture pixel in target image in the image pixel coordinate system, v _desBe the vertical coordinate of corresponding reference picture pixel in target image in the image pixel coordinate system, D (u _ref, v _ref) expression reference picture in pixel (u _ref, v _ref) corresponding gray value.

Formula (1) is actually the relational expression between the right corresponding pixel points horizontal and vertical coordinate of stereo-picture, thus the formula target image I that can ask _desThe coordinate figure of upper corresponding pixel points.

In the present invention, parameter arranges the depth adjustment that module 1 is used for the 3-D view conversion, by changing the right horizontal sensing parallax D of stereo-picture _c, D _c=u _des– u _refPerceived depth when regulating the image demonstration.

In the present invention, horizontal sensing parallax D _c=u _ref-v _refThe standard span is

-(W _i×5%)≤D _c≤W _i×5%, （2）

Wherein, W _iBe reference picture I _refWidth.D _cAnd W _iUnit be all pixel.

Formula (1) substitution formula (2) is had:

$-W_i\·5\%\≤n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\≤W_i\·5\%,---\left(3\right)$

If the user uses n, r, the D of DIBR system default _zpsParameter value, system default n, r, D _zpsThe ginseng value is respectively 1,326,130, and inequality (3) is set up all the time.

If the user does not use n, r, the D of DIBR system default _zpsParameter value carries out depth adjustment and changes n, r, D _zpsAny one or a plurality of values in three parameters change multiple parameter values minute multistep and carry out, and all allow to revise a parameter value at every turn, all need satisfy inequality (3) and could obtain euphorosia degree preferably.

When carrying out depth adjustment, be the shortcut calculation logic, once only allow to change a parameter value.When spectators regulate parameter n or D _zpsAfter, formula (3) is when being false, and whether this module will provide a suggestion r value and select to adopt for spectators, and after spectators regulated parameter r, formula (3) was when being false, and this module will provide a suggestion D _zpsWhether value is selected to adopt for spectators.

When generating the new viewpoint view, might be with reference to image I _refIn several pixels be mapped to target image I _desSame point on, Here it is changes caused " fold (folds) " phenomenon by observability.The conventional method of processing the fold phenomenon is the Z-buffer algorithm, and the method need to compare the depth value of each pixel, thereby render speed is slow.And painter's algorithm of the present invention is by determining the scanning sequency of reference picture, and assurance can first be drawn from the pixel away from virtual photocentre, draws after the point close to the virtual photocentre, thereby sets up correct hiding relation.

Painter's algorithm module 3 receives the target image sequence n that parameter arranges module 2 outputs, when being used for determining the 3-D view conversion, to the scanning sequency of reference picture; N＜0 expression target image is positioned at reference picture left side, n〉0 expression target image is positioned at the reference picture right side;

When target image was positioned at the reference picture right side, scanning sequency was for from left to right, from top to bottom; When target image was positioned at the reference picture left side, scanning sequency was for from right to left, from top to bottom.

The scanning sequency that 3-D view conversion module 4 is determined according to the painter's algorithm module, traversal reference picture I _refIn pixel p _ref, obtain the matched pixel point according to formula (1), then with pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _ref, obtain target image I _desAnd disparity map M.

The a variety of causes such as inaccurate of the inaccuracy that changes, calculates due to observability, depth image, the target image I that is generated by the DIBR technology _desIn may comprise many mistakes matched pixel points.We are called matching error (matching error) with this mistake.Matching error will seriously reduce the right quality of stereo-picture, be mingled with some background pixels in prospect in the object of an integral body, cause the uncomfortable sensation of people.Therefore, how removing matching error is a major issue in the DIBR technology.

Although traditional filtering method can be proofreaied and correct matching error effectively, amount of calculation is large, and easily causes the fuzzy of target image.Medium filtering module 5 of the present invention is carried out medium filtering based on disparity map and is proofreaied and correct, and origin is completed, and is specifically: (1), disparity map M is carried out medium filtering; (2), according to filtered disparity map M, obtain target image I _desPixel is at reference picture I _refOn the match point coordinate; (3), judge whether the match point coordinate drops on reference picture I _refIn, if so, with reference to image I _refThe pixel value at upper match point coordinate place copies target image I to _desCorrespondence position, otherwise, do not process.

Compare traditional filtering method, adopt processing of the present invention that following advantage is arranged: 1) owing to being that disparity map is carried out filtering, rather than for target image, therefore can not cause the fuzzy of image; 2) amount of calculation significantly reduces.

When utilizing the DIBR technology to generate new view, due to reasons such as the depth value of depth image are discontinuous, can produce than macroscopic-void in new view.This paper has proposed a kind of empty filling algorithm based on disparity map.At first this algorithm detects than macroscopic-void according to the disparity map of input, then utilizes disparity map to remove matching error, adopts at last the background pixel point filling cavity in reference picture.Experiment shows, this algorithm can be filled the cavity in new view preferably, and compares with the depth image preprocess method, can not reduce the picture quality of non-hole region.

Cavity packing module 6 receives disparity map M and the target image I of medium filtering submodule output _desAnd according to the scanning sequency of painter's algorithm module output and parameter, parameter in module is set, fill target image I _desThe cavity of middle existence obtains final target image I _des

Along with the development of 3D TV tech, it has been recognized that the many facilities that adopt the DIBR technology to bring.The DIBR technology has become the key technology in the 3D television system.The present invention proposes a DIBR system, it is made of the basic module such as parameter setting, painter's algorithm, 3-D view conversion, cavity filling, medium filtering and depth image preliminary treatment expansion module.The parameter of system arranges module and supports the depth adjustment function, therefore draws the degree of depth of the stereo-picture that obtains and can regulate according to spectators' different needs.Experiment shows, it is better that the right image quality of the stereo-picture that obtains is drawn by this system, is applicable to have in the 3D television system of the functions such as 2D-3D conversion, free stereo demonstration.

What need to say is, the present invention is based on that present popular image resolution ratio 1024*768 is described, in the DIBR system that can expand to other resolution of the present invention.

Although the above is described the illustrative embodiment of the present invention; so that those skilled in the art understand the present invention; but should be clear; the invention is not restricted to the scope of embodiment; to those skilled in the art; as long as various variations appended claim limit and the spirit and scope of the present invention determined in, these variations are apparent, all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (1)

1. a DIBR system that is applicable to the 3D TV, is characterized in that, comprising:

One parameter arranges module, is used for depositing 3-D view conversion and the required parameter of cavity filling, i.e. the sequence number n of target image, n is integer, and span is [4,4], n=0 represents reference-view itself, and n＜0 expression target image is positioned on the left of reference picture, is left view; N〉0 expression target image is positioned at the reference picture right side, is right view; Proportional factor r, its span are [0,1542]; The corresponding gray value D in parallax free plane _zps, its span is [0,255]; The threshold value len_bighole that counts, its span is [0,7]; Expansion pixel number l, its span is [0,7];

One painter's algorithm module meets target image sequence n, when being used for determining the 3-D view conversion, to the scanning sequency of reference picture; N＜0 expression target image is positioned at reference picture left side, n〉0 expression target image is positioned at the reference picture right side;

When target image was positioned at the reference picture right side, scanning sequency was for from left to right, from top to bottom; When target image was positioned at the reference picture left side, scanning sequency was for from right to left, from top to bottom;

The 3-D view conversion module is for the depth image of parameter, scanning sequency and the input of module stores is set according to parameter, to the reference picture I of input _refCarry out three-dimension varying:

According to the scanning sequency that the painter's algorithm module is determined, traversal reference picture I _refIn pixel, and once process, for reference picture I _refIn v _refRow u _refThe pixel p of row _ref, at first calculate it at target image I according to formula (1) _desThe matched pixel point p of middle correspondence _desWherein

$\left\{\begin{array}{c}{u}_{\mathrm{des}}-u_{\mathrm{ref}}=n\·\frac{r}{4096}\·(D_{\mathrm{zps}}-D(u_{\mathrm{ref}},v_{\mathrm{ref}}))\\ v_{\mathrm{des}}=v_{\mathrm{ref}}\end{array}\right.,---\left(1\right)$

Wherein, u _ref, v _refBe respectively reference picture I in the image pixel coordinate system _refThe horizontal coordinate of pixel and vertical coordinate, u _des, v _desBe respectively target image I in the image pixel coordinate system _desThe horizontal coordinate of middle corresponding reference picture pixel, D (u _ref, v _ref) expression depth image in pixel (u _ref, v _ref) gray value;

Then, pixel p _refPixel value copy pixel p to _des, with the element (u in disparity map M _ref, v _ref) be set to u _des-u _ref

Obtain at last target image I _desAnd disparity map M;

One medium filtering module is used for receiving 3-D view conversion export target image I _desAnd disparity map M, then carry out following processing: (1), disparity map M is carried out medium filtering; (2), according to filtered disparity map M, obtain target image I _desPixel is at reference picture I _refOn the match point coordinate; (3), judge whether the match point coordinate drops on reference picture I _refIn, if so, with reference to image I _refThe pixel value at upper match point coordinate place copies target image I to _desCorrespondence position, otherwise, do not process;

One empty packing module is used for receiving disparity map M and the target image I that the medium filtering submodule is exported _desAnd according to the scanning sequency of painter's algorithm module output and parameter, parameter in module is set, fill target image I _desThe cavity of middle existence:

(1), detect than macroscopic-void

With from left to right order traversal disparity map M, if threshold value count len_bighole and above empty point occur continuously, think it is than macroscopic-void herein by row, record this than starting point and the terminal point of macroscopic-void;

(2), expand than macroscopic-void

According to the scanning sequency of painter's algorithm module output, distinguish target image I _desBe left view or right view, if left view, execution in step a); Right view as figure, execution in step b);

A), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: in disparity map M than centered by the macroscopic-void zone, the parallax value that detects from left to right than the non-empty point of macroscopic-void right hand edge changes, when parallax value transition occurs for the first time at two in succession non-empty some places, and transition is to jump to when large from little, be the background pixel point than macroscopic-void right hand edge zone, and record this larger parallax value and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places, and transition is little from jumping to greatly, when perhaps parallax value does not unanimously have transition, be the foreground pixel point than macroscopic-void right hand edge zone, and parallax value of recording first non-cavity point is put parallax value as foreground pixel;

Then empty edge expands: if be the background pixel point than macroscopic-void right hand edge zone, be the matching error zone than the macroscopic-void left and right edges, need to expand, will deduct than the starting point abscissa of macroscopic-void l pixel and terminal point abscissa and add that l pixel is as starting point and the terminal point in the cavity that needs in target image to fill; If be the foreground pixel point than macroscopic-void right hand edge zone, the left hand edge than macroscopic-void is the matching error zone, only expand than the left hand edge of macroscopic-void, will deduct l pixel than the starting point abscissa of macroscopic-void and terminal point remains unchanged as starting point and the terminal point in the cavity that needs in target image to fill;

B), at first distinguishing than the macroscopic-void fringe region is background pixel point or foreground pixel point: in disparity map than centered by the macroscopic-void zone, the parallax value that detects from right to left than the non-empty point of macroscopic-void left hand edge changes, when parallax value transition occurs for the first time at two in succession non-empty some places, and transition is from jumping to greatly hour, be the background pixel point than macroscopic-void left hand edge zone, and record this and neglect difference and put parallax value as foreground pixel; When parallax value transition occurs for the first time at two in succession non-empty some places, and transition is to jump to greatly from little, perhaps parallax value does not unanimously have transition, is the foreground pixel point than macroscopic-void left hand edge zone, and parallax value of recording first non-cavity point is put parallax value as foreground pixel;

Then expand than the macroscopic-void edge: if be the background pixel point than macroscopic-void left hand edge zone, be the matching error zone than the macroscopic-void left and right edges, need to expand, will deduct than the starting point abscissa of macroscopic-void l pixel and terminal point abscissa and add that l pixel is as starting point and the terminal point in the cavity that needs in target image to fill; If be the foreground pixel point than macroscopic-void left hand edge zone, the right hand edge than macroscopic-void is the matching error zone, only expand than the right hand edge of macroscopic-void, will remain unchanged than the starting point of macroscopic-void adds that with the terminal point abscissa l pixel is as starting point and the terminal point in the cavity that needs in target image to fill;

In this step (2), the pixel number l of expansion is the number of mistake matched pixel point, and is relevant with the accuracy that depth image is estimated, can regulate according to the target image quality that generates, and span is 0-7;

(3), fill than macroscopic-void

With target image I _desThe middle starting point in the cavity of filling, the parallax value that the terminal point abscissa all deducts foreground pixel point of needing, obtaining need to be at reference picture I _refIn the pixel zone of copying, then, from reference picture I _refIn copy this regional pixel and be filled into target image I _desThe middle hole region that needs filling obtains final target image I _des

Images