CN102413347A - Method for correcting matching error based on depth-image-based rendering (DIBR) - Google Patents
Method for correcting matching error based on depth-image-based rendering (DIBR) Download PDFInfo
- Publication number
- CN102413347A CN102413347A CN2011103462047A CN201110346204A CN102413347A CN 102413347 A CN102413347 A CN 102413347A CN 2011103462047 A CN2011103462047 A CN 2011103462047A CN 201110346204 A CN201110346204 A CN 201110346204A CN 102413347 A CN102413347 A CN 102413347A
- Authority
- CN
- China
- Prior art keywords
- value
- matching error
- template window
- dibr
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a method for correcting a matching error based on depth-image-based rendering (DIBR). The method comprises the following steps of: firstly, filtering a parallax graph to correct a parallax value of a pixel point which causes the matching error; secondly, acquiring matching points, on a reference image, of parallax coordinate corresponding points on a target image by using a corrected parallax; and finally, if the matching points fall in the reference image, copying pixel values of the matching points on the reference image to corresponding positions of the target image, otherwise, not processing, and thus finishing correction of the matching error. Compared with the prior art, the method for correcting the matching error based on the DIBR has the advantages that: 1) by filtering the parallax graph instead of the target image, image fuzziness is avoided; and 2) by only filtering the parallax graph (a two-dimensional matrix), calculation and hardware implementation are relatively simple.
Description
Technical field
The invention belongs to and draw (Depth-Image-Based Rendering is called for short DIBR) technical field based on depth image in the 3D television system, more specifically, relate to a kind of matching error bearing calibration based on DIBR (depth image drafting).
Background technology
Draw (depth-image-based rendering based on depth image; Abbreviation DIBR) technology is to generate the new virtual visual point image of a width of cloth according to reference picture (reference image) and corresponding depth image (depth image) thereof, i.e. target image (destination image).Transmit the 3D video of right and left eyes two-path video compares with traditional needs; Adopt and only need transmit one road video and depth image thereof after the DIBR technology to generate stereo-picture right; And can realize the switching of two and three dimensions very easily;, the computational complexity of the three dimensions conversion of having avoided simultaneously being brought by classic view generation method.Just because of this, the DIBR technology has obtained extensive use in the 3D TV stereoscopic image generates (stereo pair), and it has also caused more and more keen interest of people.Usually, people adopt needs the 3D video of DIBR technology to call the 3D video (depth-image-based 3D video) based on depth image.
Yet; Because a variety of causes such as the inaccuracy of calculating, observability variation; May comprise the pixel that does not meet order matching constraint (order matching constraint) in the target image by the generation of DIBR technology, we are called matching error (matching error) with this mistake.
Matching error will seriously reduce the right quality of stereo-picture, cause people's uncomfortable feeling.Fig. 1 has shown the influence that target image caused of matching error to generating.From Fig. 1, can see, be mingled with some background pixels in the prospect in the object of an integral body.Therefore, how removing matching error is a major issue in the DIBR technology.
The bearing calibration of medium filtering matching error is a kind of nonlinear image smoothing method, compares with other linear filter with mean filter, and it can well keep the edge of target image in filtering noise.Medium filtering is that a kind of neighborhood calculates, and its principle is fairly simple, its with the target image pixel be the gray scale of all pixels in the wicket at center by ordering from small to large, get the gray value of the median of ranking results as this pixel.Be handled easily, medium filtering is got the template window that contains odd number of pixels usually.
The template window w of medium filtering can be for linear, and is square, cross etc.The median filter of standard is made up of the sliding window of an odd sized size, is generally 3 * 3 or 5 * 5 windows.As shown in Figure 2; With 3 * 3 template window is example; This window slides by pixel along the line direction of view data, during sliding each time in, all pixels in the template window are sorted according to gray value; Intermediate value 70 in these group data is as output, the gray value 210 of the center pixel of alternate template window.
Though the bearing calibration of medium filtering matching error can be proofreaied and correct matching error effectively, has following defective: owing to be that image is carried out filtering, so above-mentioned two kinds of algorithms can cause the integral body of image fuzzy, problems such as edge of image chap.
Announced on 09 14th, 2011, publication No. is that CN102186095A, name are called in the Chinese invention patent application of " a kind of matching error bearing calibration that is applicable to that depth image is drawn " and have announced a kind of matching error bearing calibration; Obtain one or more intersection regions through detection, calculate the intersection sum of intersection region; Obtain the region of search on reference picture to respectively expanding a pixel before and after the intersection region according to disparity map then; Find the pixel that except that starting point, has maximum crossing number in the intersection region; The abscissa of the match point that this pixel is new is appointed as the pixel of region of search successively; Find preliminary election match point with minimum sum of squares of deviations, after the correction parallax value, the total crossing number after confirming again to revise in this intersection region; If diminish; Then the pixel value with preliminary election match point place copies the pixel that has maximum crossing number on the target image to; If do not diminish, then other pixels that have on the reference picture of the minimum sum of squares of deviations are carried out same processing, diminish up to revised total crossing number.When total crossing number is 0 or till all pixels all can not make total crossing number diminish.Like this through confirming the intersection region and calculate minimum total crossing number that can detect and proofread and correct matching error effectively, the target image quality is improved.But this method is calculated and hardware is realized all comparatively complicated.
Summary of the invention
The objective of the invention is to overcome the deficiency of prior art, provide a kind of and calculate and all comparatively simple matching error bearing calibration of hardware realization based on DIBR.
For realizing above-mentioned purpose, the present invention is based on the matching error bearing calibration of DIBR, it is characterized in that, may further comprise the steps:
(1), disparity map is carried out medium filtering;
(2), according to filtered disparity map, obtain the match point coordinate of target image pixel on reference picture;
(3), judge whether the match point coordinate drops in the reference picture, if then the pixel value with match point coordinate place on the reference picture copies the target image correspondence position to, otherwise, do not handle.
Goal of the invention of the present invention is achieved in that
The present invention is based on the matching error bearing calibration of DIBR, earlier disparity map is carried out filtering, thereby the parallax value of the pixel that causes matching error is proofreaied and correct; Obtain the match point of parallax coordinate corresponding points on reference picture on the target image by the parallax after proofreading and correct then; If last match point drops in the reference picture, the pixel letter value of match point on the reference picture is copied on the target image correspondence position, otherwise; Do not handle, accomplish the correction of matching error like this.Compare prior art, the matching error bearing calibration that the present invention is based on DIBR has following advantage: 1) owing to be that disparity map is carried out filtering, rather than to target image, so can not cause bluring of image; 2) just disparity map (two-dimensional matrix) is carried out filtering, calculating and hardware are realized all comparatively simple.
Description of drawings
Fig. 1 is the image when having matching error in the technological target image that generates of DIBR;
Fig. 2 is that the medium filtering matching error is proofreaied and correct sketch map;
Fig. 3 is an embodiment flow chart that the present invention is based on the matching error bearing calibration of DIBR;
Fig. 4 is template window and disparity map element value corresponding relation sketch map;
Fig. 5 is reference picture, depth image and corresponding target image thereof;
Fig. 6 is traditional medium filtering matching error bearing calibration design sketch;
Fig. 7 is the matching error bearing calibration design sketch that the present invention is based on DIBR.
Embodiment
Describe below in conjunction with the accompanying drawing specific embodiments of the invention, so that those skilled in the art understands the present invention better.What need point out especially is that in the following description, when perhaps the detailed description of known function and design can desalinate main contents of the present invention, these were described in here and will be left in the basket.
Fig. 3 is an embodiment flow chart that the present invention is based on the matching error bearing calibration of DIBR.
In this enforcement, as shown in Figure 3, be input based on the matching error bearing calibration of DIBR: disparity map M, reference picture I
Ref, target image I
Des, three's resolution is W
i* H
iOutput: the disparity map M after the correction, target image I
Des
Concrete steps are:
(1), initialization template window central vertical cyclic variable v=1;
(2), initialization template window central horizontal cyclic variable u=1.
(3), the value of each element a11, a21, a31, b11, b21, b31, c11, c21, c31 is template window pairing value when covering disparity map M among the initialization template window matrix win, the centre coordinate of overlay area be (v, u), as shown in Figure 4;
(4), judge some number n in cavity among the template window matrix win; If step (5) is changeed in n＜5; Otherwise do not handle, change step (9);
(5), matrix conversion: template window matrix win is converted into 1 * 9 matrix of template window matrix win_reshape by 3 * 3 matrixes, and conversion sequence is as shown in Figure 4; Coordinate among the 1st of such 1 * 9 matrix win_reshape corresponding disparity map M of value is that (coordinate among the 2nd corresponding disparity map M is (v for v-1, the parallax value of u-1) locating; U-1) parallax value of locating; Coordinate among the 3rd corresponding disparity map M be (v+1, the parallax value of u-1) locating, and the like;
(6), medium filtering: 1 * 9 template window matrix win_reshape after the conversion is carried out the ordering of bubbling method, ask for intermediate value, and replace the numerical value of former disparity map template window center point with this intermediate value.If the intermediate value that finds is a, then M (v, u)=a;
The basic conception of bubbling method ordering (Bubble Sort) is: two more adjacent successively numbers, decimal is placed on the front, and big number is put behind.Promptly at first time: at first relatively the 1st and the 2nd number, before decimal put, after big number was put, relatively the 2nd number and the 3rd number then before decimal put, after big number is put, so continued, until more last two numbers, before decimal put, count greatly put after.So far finish for first time, the number of maximum has been put at last.At second time: still (because possible because the exchange of the 2nd number and the 3rd number since the first logarithm comparison; Make that the 1st number is no longer less than the 2nd number); Before decimal put, after big number is put, always relatively to number second from the bottom (being maximum on the position last); Finish, obtaining a new maximum number (being second largest number in fact) on the penultimate position in whole ordered series of numbers for second time.So go down, repeat above process, until final completion ordering.
In the present invention,, get final product, need all not accomplish ordering all data so when ordering only needs that a half data is accomplished ordering because only need ask for the intermediate value of 1 * 9 template window matrix win_reshape.In sequencer procedure; Value among 1 * 9 template window matrix win_reshape is done corresponding variation with aforesaid operations; Confirm that through the intermediate value of searching 1 * 9 template window matrix win_reshape after the respective change template window overlay area centre coordinate is (v, parallax value u).
In the present embodiment, as shown in Figure 3, come time number of record ordering with variable i, write down the number of times of each time comparison with variable j; As win_reshape [j]＞win_reshape [j+1], then exchange, promptly win_reshape [j] changes with the value of win_reshape [j+1], if be not more than; Then do not exchange, number of comparisons j adds 1 then, like number of comparisons j smaller or equal to 9-i, i.e. first time be no more than 8 times; Then return the comparison of carrying out next group number, if number of comparisons j does not satisfy smaller or equal to 9-i, promptly 8, explain and more finish for first time; Time number i adds 1, repeats from the 1st logarithm, to the comparison of 9-i logarithm, judges transposing or does not change; Up to judge finishing, promptly number of comparisons j does not satisfy smaller or equal to 9-i, and promptly 7, continue like this to compare; After time number i does not satisfy i≤5, promptly compared 5 times, obtained intermediate value, i.e. win_reshape (5)=a;
(7), try to achieve the match point coordinate of target image corresponding points on reference picture after the parallax correction.Parallax after the correction is a, and then the abscissa of its match point on reference picture is u
Ref=u-a;
(8), judge whether the abscissa of this match point drops in the reference picture, if promptly:
0≤u
ref＜W
i， (1)
Then with this match point (v, u
Ref) pixel value copy to the target image corresponding points (v, u) on; Otherwise, change (9);
(9), u=u+1, be about to the template window level and move down a point;
(10) if u＜W
i-1, i.e. delegation's filtering does not finish, and then changes the medium filtering of the next point of (3) beginning; Otherwise, change (11);
(11), v=v+1, the soon horizontal line down of template window;
(12) if v＜H
i-1, promptly entire image does not have filtering to finish, and changes (3); Otherwise, finish.
Below comparison is tested in the medium filtering matching error bearing calibration of the present invention and prior art.
Test purpose: correction target image I
DesIn matching error.This target image is generated according to reference picture and depth image thereof by the DIBR technology;
Test-purpose: verify and assess the matching error bearing calibration based on DIBR of the present invention;
Method of testing: in this test, adopt " Ballet " sequence and corresponding calibration parameter thereof to carry out the error correction experiment.The production method of " Ballet " sequence is referring to list of references; Zitnick; C.L.; Et al.High-quality video view interpolation using a layered representation.in ACM SIGGRAPH and ACM Trans.on Graphics.2004.Los Angeles, CA, USA.(c) figure, i.e. target image I among Fig. 5
DesThe target image that is generated by the DIBR technology when being length of base B=30cm is by finding out target image I among Fig. 5 (c)
DesIn contain many matching errors, seriously reduced the quality of image, for example people's " spot " on the face.And black region is represented the cavity in the image, can eliminate through empty filling algorithm, here for the ease of observing algorithm effect, in this test and do not fill.
Test one
Adopt the bearing calibration of traditional medium filtering matching error to target image I
DesHandle, the result is as shown in Figure 6.
Test two
Employing the present invention is based on the matching error bearing calibration of DIBR to target image I
DesHandle, the result is as shown in Figure 7.
Test result analysis:
Can find out that by Fig. 6, Fig. 7 above-mentioned two kinds of methods all can effectively be removed matching error.But traditional medium filtering matching error bearing calibration has caused the fuzzy of image to a certain extent, and adopts the matching error bearing calibration that the present invention is based on DIBR simple, under the prerequisite that does not influence image definition, can effectively proofread and correct matching error.
Although above the illustrative embodiment of the present invention is described; So that the technical staff in present technique field understands the present invention, but should be clear, the invention is not restricted to the scope of embodiment; To those skilled in the art; As long as various variations appended claim limit and the spirit and scope of the present invention confirmed in, these variations are conspicuous, all utilize innovation and creation that the present invention conceives all at the row of protection.
Claims (3)
1. the matching error bearing calibration based on DIBR is characterized in that, may further comprise the steps:
(1), disparity map is carried out medium filtering;
(2), according to filtered disparity map, obtain the match point coordinate of target image pixel on reference picture;
(3), judge whether the match point coordinate drops in the reference picture, if then the pixel value with match point coordinate place on the reference picture copies the target image correspondence position to, otherwise, do not handle.
2. the matching error bearing calibration based on DIBR according to claim 1 is characterized in that, in the described medium filtering, if cavity point number does not satisfy n＜5 in the template window matrix, then the parallax value of template window center is not handled.
3. the matching error bearing calibration based on DIBR according to claim 2; It is characterized in that, in the described medium filtering, pairing value when the value of each element is template window covering disparity map in the first initialization template window matrix; The centre coordinate of overlay area be (v, u);
Then, judge some number n in cavity in the template window matrix; If n＜5 continue to handle; Otherwise do not handle;
The template window matrix is converted into 1 * 9 matrix of template window matrix by 3 * 3 matrixes; Coordinate in the 1st of 1 * 9 matrix corresponding disparity map of value is that (coordinate among the 2nd corresponding disparity map M is (v for v-1, the parallax value of u-1) locating; U-1) parallax value of locating; Coordinate in the 3rd corresponding disparity map be (v+1, the parallax value of u-1) locating, and the like;
1 * 9 template window matrix after the conversion is carried out the ordering of bubbling method, ask for intermediate value, and replace the numerical value of former disparity map template window center point with this intermediate value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110346204.7A CN102413347B (en) | 2011-11-04 | 2011-11-04 | Method for correcting matching error based on depth-image-based rendering (DIBR) |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110346204.7A CN102413347B (en) | 2011-11-04 | 2011-11-04 | Method for correcting matching error based on depth-image-based rendering (DIBR) |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102413347A true CN102413347A (en) | 2012-04-11 |
| CN102413347B CN102413347B (en) | 2014-03-19 |
Family
ID=45915148
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110346204.7A Expired - Fee Related CN102413347B (en) | 2011-11-04 | 2011-11-04 | Method for correcting matching error based on depth-image-based rendering (DIBR) |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102413347B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932661A (en) * | 2012-11-29 | 2013-02-13 | 重庆大学 | Median filtering matching error correction method for disparity map, and circuit for implementing method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6519358B1 (en) * | 1998-10-07 | 2003-02-11 | Sony Corporation | Parallax calculating apparatus, distance calculating apparatus, methods of the same, and information providing media |
| CN101325725A (en) * | 2008-07-24 | 2008-12-17 | 四川大学 | Method for extracting parallax of stereoscopic image based on sub-pixel |
| CN101771893A (en) * | 2010-01-05 | 2010-07-07 | 浙江大学 | Video frequency sequence background modeling based virtual viewpoint rendering method |
| CN102186095A (en) * | 2011-05-03 | 2011-09-14 | 四川虹微技术有限公司 | Matching error correction method applicable for depth-image-based rendering |
-
2011
- 2011-11-04 CN CN201110346204.7A patent/CN102413347B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6519358B1 (en) * | 1998-10-07 | 2003-02-11 | Sony Corporation | Parallax calculating apparatus, distance calculating apparatus, methods of the same, and information providing media |
| CN101325725A (en) * | 2008-07-24 | 2008-12-17 | 四川大学 | Method for extracting parallax of stereoscopic image based on sub-pixel |
| CN101771893A (en) * | 2010-01-05 | 2010-07-07 | 浙江大学 | Video frequency sequence background modeling based virtual viewpoint rendering method |
| CN102186095A (en) * | 2011-05-03 | 2011-09-14 | 四川虹微技术有限公司 | Matching error correction method applicable for depth-image-based rendering |
Non-Patent Citations (3)
| Title |
|---|
| 《Circuits and Systems (MWSCAS), 2011 IEEE 54th International Midwest Symposium on》 20110810 Hyun-Woong Cho等 Depth-Image-Based 3D Rendering with Edge 1-4页 1-2 , * |
| HYUN-WOONG CHO等: "Depth-Image-Based 3D Rendering with Edge", 《CIRCUITS AND SYSTEMS (MWSCAS), 2011 IEEE 54TH INTERNATIONAL MIDWEST SYMPOSIUM ON》, 10 August 2011 (2011-08-10), pages 1 - 4, XP031941313, DOI: doi:10.1109/MWSCAS.2011.6026369 * |
| 张倩等: "采用图像修复的基于深度图像复制", 《光电子.激光》, no. 10, 15 October 2009 (2009-10-15) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932661A (en) * | 2012-11-29 | 2013-02-13 | 重庆大学 | Median filtering matching error correction method for disparity map, and circuit for implementing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102413347B (en) | 2014-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liang et al. | Learning for disparity estimation through feature constancy | |
| CN107578430B (en) | Stereo matching method based on self-adaptive weight and local entropy | |
| CN109544447A (en) | A kind of image split-joint method, device and storage medium | |
| CN106023230B (en) | A kind of dense matching method of suitable deformation pattern | |
| CN103384343B (en) | A kind of method and device thereof filling up image cavity | |
| CN102972038A (en) | Image processing apparatus, image processing method, program, and integrated circuit | |
| WO2017156905A1 (en) | Display method and system for converting two-dimensional image into multi-viewpoint image | |
| CN107369131A (en) | Conspicuousness detection method, device, storage medium and the processor of image | |
| CN111899295B (en) | Monocular scene depth prediction method based on deep learning | |
| CN108921942A (en) | The method and device of 2D transformation of ownership 3D is carried out to image | |
| CN103024421A (en) | Method for synthesizing virtual viewpoints in free viewpoint television | |
| CN107945222A (en) | A kind of new Stereo matching cost calculates and parallax post-processing approach | |
| CN104869386A (en) | Virtual viewpoint synthesizing method based on layered processing | |
| CN104680487A (en) | Non-local image inpainting method based on low-rank matrix recovery | |
| CN102547338A (en) | DIBR (Depth Image Based Rendering) system suitable for 3D (Three-Dimensional) television | |
| CN111489383B (en) | Depth image up-sampling method and system based on depth marginal point and color image | |
| CN110033483A (en) | Based on DCNN depth drawing generating method and system | |
| CN109272539A (en) | The decomposition method of image texture and structure based on guidance figure Total Variation | |
| CN115601406A (en) | Local stereo matching method based on fusion cost calculation and weighted guide filtering | |
| CN111179173B (en) | Image splicing method based on discrete wavelet transform and gradient fusion algorithm | |
| CN114862926A (en) | Stereo matching method and system fusing AD cost and multi-mode local feature cost | |
| CN104270624A (en) | Region-partitioning 3D video mapping method | |
| CN108924434B (en) | Three-dimensional high dynamic range image synthesis method based on exposure transformation | |
| CN105898279B (en) | A kind of objective evaluation method for quality of stereo images | |
| CN107169498A (en) | It is a kind of to merge local and global sparse image significance detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140319 Termination date: 20161104 |