CN101873508B - Intermediate view synthesis method based on improved BP (Belief Propagation) algorithm - Google Patents
Intermediate view synthesis method based on improved BP (Belief Propagation) algorithm Download PDFInfo
- Publication number
- CN101873508B CN101873508B CN2010101826345A CN201010182634A CN101873508B CN 101873508 B CN101873508 B CN 101873508B CN 2010101826345 A CN2010101826345 A CN 2010101826345A CN 201010182634 A CN201010182634 A CN 201010182634A CN 101873508 B CN101873508 B CN 101873508B
- Authority
- CN
- China
- Prior art keywords
- formula
- disparity map
- image
- zone
- algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses an intermediate view synthesis method based on an improved BP (Belief Propagation) algorithm. The method comprises the following steps of: (1) obtaining the optical parallax vector information of each pixel by adopting an iterative interactive method based on the improved BP; and (2) detecting a shielding area, compensating the optical parallax of the shielding area, and then carrying out calculationus of interpolation on of an intermedate middle view: firstly, carrying out area dividing on an image by utilizing a consistency constraint condition to divide the image into a shielding area, a consistency area and a blur area, meanwhile, compensating the shielding area, and acquiring the middle intermediate view by adopting different interpolations according to the characteristics of the area per se. The invention improves the BP algorithm, removes the poor effect of excessive with smoothness transition and further improves the accuracy of the parallax calculation. In addition, the a novel interpolation algorithm is designed, the characteristics of each part of a three-dimensional image are better considered, and the visual effect and quality of a virtual view is improved.
Description
Technical field
The present invention relates to the multimedia signal processing technique field, especially in the 3D video, carry out the field of virtual vision application and development.
Background technology
From the eighties of eighties of last century, abroad just begin to have the report of view synthetic technology, domestic relevant scholar is in recent years also for the synthetic a few thing of having done of image.The more than ten years in past, synthetic this direction of view has received widely to be paid close attention to, because it has related to the technology of computer vision, computer graphics and Digital Image Processing all directions, this makes it not only be difficult to solve, and method also is tending towards variation.
The synthetic of virtual view is one of difficult point of three-dimensional video-frequency for many years always.The view that the mode of traditional composograph is based on 3-D geometric model synthesizes.The method based on model for traditional need be paid huge workload owing to generate complicated geometrical model, and needs a lot of skills; In addition, in order to draw out lighting effect preferably, often need very expensive hardware spending, amount of calculation is huge, be difficult on common computing platform, handle in real time, and the sense of reality of image does not ensure.So another kind of view synthetic technology based on image obtains increasing concern under this situation.Based on the view synthetic technology of the image product that to be computer vision cooperate with figure educational circles.Just proposed the imagination that combines with graphics as far back as vision worker in 1984, the trial of cooperating had several times also been arranged, but never find an appropriate direction, disputed on very big for ultimate aim especially.Free discussion by initiations such as R C Jain in 1991 afterwards all once promoted both cooperations, such as having proposed image modeling based on CAD etc.
It is the view distortion that Seitz and Dyer have proposed a kind of image deformation method; The image of the new viewpoint that this method generated can keep the appearance profile characteristic of original reference image; This method has solved original image distortion method in the process that two width of cloth images to the same scene different points of view are out of shape, and intermediate image is difficult to guarantee the problem of the inner shape characteristic in the original image.Synthetic method is looked in the virtual centre that people such as Ott propose, and it is synthetic that first utilizes stereo-picture to carry out view, and they have considered the problem of the virtual contact ac of participant's eyes in teleconference.Given two video cameras that are positioned over teleconference display two ends; Utilize two known width of cloth images to synthesize virtual medial view; But do not consider occlusion area in the literary composition, ignored in the disparity map, used gray-level interpolation in the hollow sectors of synthetic view owing to block the cavity of causing.People such as McMillan provide in hypothesis under the situation of disparity map, have obtained the method for in real time synthetic new viewpoint image in the helmet-type display system, and have provided a kind of straightforward procedure that solves occlusion issue based on forward direction mapping (Forwarding Mapping).When Chen and Williams effectively carry out problem that image draws in solving computer graphics, introduced image interpolation method, only pointed out that under the baseline condition parallel with the plane of delineation, the image interpolation method of linearity could synthesize correct view.People such as Lawrence at first utilize belief propagation algorithm (BP), over-segmentation technology under Hidden Markov (MRF) framework, obtain dense disparity map, utilize interpolation technique to synthesize medial view then.
It is external to compare, domestic less to the synthetic research of view, after obtaining reliable dense disparity map, generates disparity map through the correspondence position of pixel in the image of the left and right sides of seeking in the medial view like people such as Zhang Zhaoyang, Anping, Lv Chaohui, Luo Yan; People such as Hu Zhiping propose the method for the synthetic new viewpoint of a kind of image based on the horizontal and vertical mobile collection of video camera, and this method preferential profile that guarantees in composograph is accurately corresponding, thereby makes the composograph clear-cut; People such as Wu Qiongyu have then proposed a kind of view synthesizing method as if the image sequence of learning based on image under the demarcation condition, and this method has been avoided coupling and three-dimensional reconstruction, directly generates medial view; People such as Chai Deng-feng have proposed a kind of two minutes figure and have cut the estimation that algorithm carries out parallax, and foreground area and background area are then distinguished in the detection of blocking then, find out the match point of medial view in input picture at last and obtain the medial view picture.
Summary of the invention
Synthetic technology is looked in the centre based on disparity estimation that the purpose of this invention is to provide under a kind of parallel double lens camera system, comprises the application of disparity estimation and two aspects of view interpolation.
The present invention adopts following technical scheme for realizing above-mentioned purpose:
The present invention is based on the centre of improving the BP algorithm looks synthetic method and comprises the steps:
Step 1) adopts the method based on the iteration of improving BP, obtains the difference vector information of each pixel;
Step 2); Detect occlusion area; And the parallax of occlusion area compensated, and then carry out the interpolation calculation of medial view: at first utilize the consistency constraint condition that image is carried out dividing region, be divided into occlusion area, consistent zone and fuzzy region to image; Meanwhile carry out the compensation of occlusion area, take different interpolation to obtain medial view according to the characteristics in each zone itself then.
Preferably, the described method based on the iteration of improving BP of step 1) is following:
The first step, the initial matching cost when calculating the 0th grade, the data item during then to each rank is carried out initialization; The color distance of the node when calculating each rank and its neighbours' domain node, and the color distance value between the adjacent node during to each rank is carried out initialization;
Second step, total L level, l=0,1 ..., L, the iterations of each grade are t=0,1 ..., T, L and T are the natural number greater than 1;
In the 3rd step, utilize formula
To iterations is that 0 o'clock message is carried out initialization; Then according to formula
Calculate the message m of each node
L, t(d
q), and be the minimum value of information minimum of each node selection;
In the 4th step, the message of calculating gained is handled as follows:
for?d
q?from?1?tok-1：
m
l，t(d
q)←min(m
l，t(d
q)，m
l，t(d
q-1)+c)
for?d
q?from?k-2to?0：
m
l，t(d
q)←min(m
l，t(dq)，m
l，t(d
q+1)+c)
Add minimum the ρ (I) of
formula, and save as MIN;
The 5th step, compare value in the message vector and MIN, if less than MIN, then the value of correspondence position substitutes with MIN in the message vector, at last the message vector is carried out normalization, gets message values to the end;
In the 6th step, iterations adds 1, repeats for the 3rd to the 5th step, is T up to iterations;
In the 7th step, progression subtracts 1, repeats for the 3rd to the 6th step, knows that progression becomes 0;
In the 8th step, obtaining each node according to formula
at last is that the optimum label of pixel is a parallax.
Preferably, step 2) the described consistency constraint condition of utilizing is carried out dividing region to image, is divided into occlusion area, consistent zone and fuzzy region to image and is meant: at first, and for left disparity map d
LRUtilize formula d
LR(x
L, y)-d
RL(x
L-d
LR(x
L, y), y)＞T detects right half occlusion area, then for right disparity map d
RLUtilize formula d
RL(x
R, y)-d
LR(x
R+ d
RL(x
R, y), y)＞T detects left half occlusion area; And respectively a left side half occlusion area in half occlusion area of the right side in the left disparity map and the right disparity map is carried out mark; In addition, the parallax value of the rightmost partial pixel point of the Far Left partial pixel of left image and right image all composes 0; Then, to satisfying formula d in the left disparity map
LR(x
L, y)=d
RL(x
L-d
LR(x
L, y), pixel region y) is labeled as right consistent zone, for satisfying formula d in the right disparity map
RL(x
R, y)=d
LR(x
R+ d
RL(x
R, y), pixel region y) is labeled as the first from left and causes the zone; At last, the zone that is not labeled in the left disparity map, being labeled as right fuzzy region, is the zone marker that is not labeled in the right disparity map left fuzzy region.
Preferably, step 2) described interpolation method step is following:
At first, be reference with left image, utilize formula:
Then, be reference with right image, utilize formula:
At last, left and right sides transition medial view is looked like to utilize formula:
Preferably, the medial view that obtains synthesizing is as I
MIn still have the zone do not shone upon, take the method for reverse mapping, in the image of the left and right sides, seek its immediate point respectively,
Choose respectively and satisfy formula
And formula
Left and right sides image in optimal match point, wherein the hunting zone of the independent variable of two formulas is respectively formula x
M≤x
L≤x
M+ (1-a) d
MaxWith formula x
M-ad
Max≤x
R≤x
MUtilize formula aI then
LM(x
L ', y)+(1-a) I
RM(x
R ', y) interpolation is carried out in the zone of not shone upon, obtained final medial view picture.
Advantage of the present invention and effect are:
1. improve the BP algorithm, removed excessive level and smooth ill effect, further improved the accuracy that parallax calculates.
2. design new interpolation algorithm, considered the characteristics of stereo-picture various piece better, improved the visual effect and the quality of virtual view.
Other advantages of the present invention and effect will continue to describe below.
Description of drawings
Fig. 1 is the virtual view synthetic schemes.
Fig. 2 is that the parallel camera chain of binocular solid obtains fixed scene figure.
Fig. 3 be the disparity map that obtained when getting different value of the cutoff value of linear model performance relatively.(a) comparison of unshielding regional disparity error rate; (b) comparison of discontinuity zone parallax error rate; (c) comparison of All Ranges parallax error rate.
The disparity map that Fig. 4 obtains when being level and smooth different setting the: the disparity map that (a) obtains with conventional linear model (cutoff value was got 5.0 o'clock); (b) with disparity map that algorithm of the present invention obtained; (c) disparity map after the secondary series disparity map is optimized.
Fig. 5 is the composite diagram on the Venus image sequence: (a-c) be true picture; (d-f) for using the composograph of document algorithm; (g-i) for using the composograph of algorithm of the present invention.
Fig. 6 is the residual plot that the Venus image sequence uses algorithms of different to produce: (a-c) for using the composograph of document algorithm; (d-f) for using the composograph of algorithm of the present invention.
Embodiment
Below in conjunction with accompanying drawing and embodiment, technical scheme of the present invention is done further to set forth.
The present invention is based on the centre of improving the BP algorithm and look synthetic method, may further comprise the steps:
From disparity estimation and two big aspects of view interpolation middle view synthetic technology is analyzed and studied.The disparity estimation purpose here also is to obtain dense reliable disparity map; But we will adopt a kind of method based on the iteration of improving BP here; Obtain the difference vector information of each pixel; All be that difference between the simple parallax label that is endowed according to neighbor is provided with level and smooth cutoff value when utilizing linear model to calculate level and smooth, cause the excessively level and smooth of disparity map possibly in view of former BP algorithm.We have proposed a kind of color distance that utilizes and come self adaptation that the method for level and smooth cutoff value is set in this part; After having obtained disparity map, we will carry out view interpolation, but, because the existence of occlusion area has had dense disparity map, might not obtain good medial view picture, because not there is the matched pixel point in occlusion area itself.Therefore, before view interpolation, need detect occlusion area; And the parallax of occlusion area compensated; And then carry out the interpolation calculation of medial view, and considering that here each zone of image possibly take on a different character, we have proposed a kind of new interpolation strategies; At first utilize the consistency constraint condition that image is carried out dividing region; Be divided into occlusion area, consistent zone and fuzzy region to image, meanwhile carry out the compensation of occlusion area, take different interpolation to obtain medial view according to the characteristics in each zone itself then.
Look in the synthetic technology based on the centre of improving the BP algorithm described, described following based on the parallax calculation procedure of improving the BP algorithm:
The first step, the initial matching cost when calculating the 0th grade, the data item during then to each rank is carried out initialization; The color distance of the node when calculating each rank and its neighbours' domain node, and the color distance value between the adjacent node during to each rank is carried out initialization.
Second step, suppose that we have the L level, l=0,1 ..., L, the iterations of each grade are t=0,1 ..., T.
In the 3rd step, utilizing formula
is that 0 o'clock message is carried out initialization to iterations.Then according to formula
Calculate the message m of each node
L, t(d
q), and be the minimum value of information minimum of each node selection.
In the 4th step, the message of calculating gained is handled as follows:
for?d
q?from?1?to?k-1：
m
l，t(d
q)←min(m
l，t(d
q)，m
l，t(d
q-1)+c)
for?d
q?fromk-2to?0：
m
l，t(d
q)←min(m
l，t(d
q)，m
l，t(d
q+1)+c)
Add minimum the ρ (I) of
formula, and save as MIN.
The 5th step, compare value in the message vector and MIN, if less than MIN, then the value of correspondence position substitutes with MIN in the message vector, at last the message vector is carried out normalization, gets message values to the end.
In the 6th step, iterations adds 1, repeats for the 3rd to the 5th step, is T up to iterations.
In the 7th step, progression subtracts 1, repeats for the 3rd to the 6th step, knows that progression becomes 0;
The 8th goes on foot, and obtains the optimum label (parallax) of each node (pixel) at last according to formula
.
Look in the synthetic technology based on the centre of improving the BP algorithm described, the described consistency constraint condition of utilizing is carried out dividing region to image, is divided into occlusion area, consistent zone and fuzzy region to image and is meant at first, for left disparity map d
LRUtilize formula d
LR(x
L, y)-d
RL(x
L-d
LR(x
L, y), y)＞T detects right half occlusion area, then for right disparity map d
RLUtilize formula d
RL(x
R, y)-d
LR(x
R+ d
RL(x
R, y), y)＞T detects left half occlusion area.And respectively a left side half occlusion area in half occlusion area of the right side in the left disparity map and the right disparity map is carried out mark.In addition; The rightmost partial pixel of Far Left partial pixel and the right image of left side image is because the restriction of left and right cameras coverage, though in disparity estimation, also obtained parallax value; In fact there is not corresponding matched pixel point in it; Therefore according to the definition of occlusion area, also should to its block mark (for compensate make things convenient for mark the time, the parallax value of these pixels all composes 0).
Then, to satisfying formula d in the left disparity map
LR(x
L, y)=d
RL(x
L-d
LR(x
L, y), pixel region y), we are labeled as right consistent zone, for satisfying formula d in the right disparity map
RL(x
R, y)=d
LR(x
R+ d
RL(x
R, y), pixel region y), we are labeled as the first from left and cause the zone.At last, we are labeled as right fuzzy region to the zone that is not labeled in the left disparity map, are the zone marker that is not labeled in the right disparity map left fuzzy region.Left and right sides image has been divided into half occlusion area, consistent zone and fuzzy region three parts respectively like this.
Look in the synthetic technology based on the centre of improving the BP algorithm described, described interpolation strategies step is following:
The process of view interpolation was divided into for three steps, and at first, we are reference with left image, utilize formula
Look (I in the middle of generating left transition
LM), the pixel value of its right-of-center in political views's half occlusion area directly replaces with the pixel value of the correspondence position of left image, and right uniformity zone and right fuzzy region are utilized interpolation technique.Then, we are reference with right image, utilize formula
Generate right transition medial view picture (I
RM); When generating right transition medial view picture, we have only handled left fuzzy region and left half occlusion area, the first from left is caused the zone do not handle; This be because the first from left cause the zone with right consistent zone in pixel be one to one; Therefore the abscissa value of middle view image therefrom also equates, so after interpolation is carried out in consistent zone to the right side, there is no need again the first from left to be caused the zone and carry out interpolation.So just, obtained left and right sides transition medial view picture.At last, we look like to utilize formula
to carry out interpolation again one time to left and right sides transition medial view.So just, obtained medial view.
The medial view that the above step of process is synthesized is as I
MIn still have the zone do not shone upon, therefore, need further handle it.In order to address this problem, take the method for reverse mapping, in the image of the left and right sides, seek its immediate point respectively,
We choose respectively and satisfy formula for this reason
And formula
Left and right sides image in optimal match point, wherein the hunting zone of the independent variable of two formulas is respectively formula x
M≤x
L≤x
M+ (1-a) d
MaxWith formula x
M-ad
Max≤x
R≤x
MUtilize formula aI then
LM(x
L ', y)+(1-a) I
RM(x
R ', y) interpolation is carried out in the zone of not shone upon, obtained final medial view picture.
In the 3 D video display system, the number of video camera is limited, can only describe three-dimensional scenic from specific visual angle.If seek out the picture of free-viewing angle, realize the effect of " looking around ", the method for solution is exactly the synthetic of medial view.The synthetic of medial view is exactly according to known source images, synthesizes the image of the virtual view between the source images viewpoint, and the simplest situation is a parallel double purpose camera chain; At this moment, virtually look synthetic, generate the medial view that is positioned on the straight line that left and right sides reference view determined promptly according to the image of the Same Scene of left and right cameras picked-up; As shown in Figure 1; It mainly was divided into for two steps: at first, to about two width of cloth reference pictures carry out solid coupling, obtain the difference vector of each pixel; Then according to the match point in the reference picture of the left and right sides to carry out interpolation obtain in the middle of the image of new viewpoint.
As shown in Figure 2, we obtain with the parallel camera chain of binocular and comprise the scene of A to the F object, obtain I
LAnd I
RLeft and right sides stereo-picture is right.As can be seen from the figure, right image I
RIn can only see the projection of A, B, C and F object, and can only see the projection of A, C, E and F object among the left image I L.We see being merely able to as B project objects zone at right image, and are called left half occlusion area at the pixel region that left image can't see; As E project objects zone, seeing and call right half occlusion area at left image at the pixel region that right image can't see; And be defined as full occlusion area to the pixel region of as D, in the image of the left and right sides, all can't see.
When medial view synthetic; Full occlusion area as D might be half occlusion area for certain the virtual viewpoint on the baseline, yet considers that from left and right sides image pair can not get any information of object D; Simultaneously also for the detection of simplifying occlusion area and the difficulty of compensation; We do not consider the occlusion area as D, and the arbitrary objects on the image that hypothesis intermediate-view place video camera is obtained, and that sees in all at least can the arbitrary image in the image of the left and right sides arrives.
Experiment selects for use classical stereo-picture that Tsukuba, Venus, Teddy, Cones are tested.Experiment is to move gained on the PC of Pentium43.4GHz at CPU, and operating system is WindowsXP SP2, and programming tool is VC++6.0.When the realization of algorithm, we are provided with 5 grades BP algorithm, the iterations during each rank be respectively 10,10,5,5,5}.The parameter value of algorithm is as shown in the table.
Fig. 3 has compared level and smooth and has adopted linear model to calculate; The performance of the disparity map that linear model is obtained when getting different cutoff value is when level and smooth item utilizes formula V (x)=min (c|x|, ρ (I)) to calculate; We get 1.0,3.0,5.0,7.0,9.0 respectively by cutoff value, obtain the first five point.Adopt algorithm of the present invention to obtain last point according to the adaptive level and smooth item of cutoff value calculating that is provided with of color distance.
From figure, can see; Calculate the disparity map that level and smooth item is obtained according to the adaptive method that the linear model cutoff value is set of color distance; No matter be whole mistake matching rate (all); Still at the mistake matching rate in parallax discontinuity zone (disc.) and unshielding zone (nocc.), basically all be in minimum position, especially to Venus, Teddy and Cones image.This shows; What the present invention proposed is provided with the algorithm of linear model cutoff value according to the color distance self adaptation; Can a dynamic scope be set for the maximin of level and smooth cost, not only can avoid the excessively level and smooth of disparity map, and help the raising of three-dimensional coupling accuracy.
Level and smooth disparity map that is obtained of method calculating that the linear model cutoff value is set according to this self adaptation is as shown in Figure 4.
The algorithm (representing with Proposed in the experimental result) that we here propose the present invention and a kind of current current paper by Changming Jin, the virtual view composition algorithm (representing with Other in the experimental result) of Hong Jeong proposition compares.Respectively from subjectivity watch, residual plot and three aspects of Y-PSNR come the evaluation algorithms result.
During experiment; Use Venus, Teddy, Cones image sequence; Left and right sides reference picture is respectively second in image sequence and the 6th two field picture; Because the distance of obtaining between the viewpoint of every two field picture equates, we make a be respectively 0.25,0.5,0.75 synthesizing the 3rd, the 4th, the 5th two field picture here.
We at first adopt improvement BP algorithm parallax estimation method to obtain the right horizontal parallax figure of stereo-picture.As shown in Figure 5, we have listed the image that the Venus image sequence adopted invention to propose the virtual view that algorithm synthesizes in algorithm and the document at 0.25,0.5,0.75 o'clock in a difference value.
The earlier simple virtual synthetic view that obtains, the virtual visual point image that composition algorithm can obtain having photo feel is looked in centre of the present invention, more approaches true picture than the algorithm in the document.
In order more to be clear that the difference of algorithms of different performance, Fig. 6 has listed corresponding residual image.White dotted line in the residual plot is exactly the discrepancy of virtual composograph and true picture.Can find out that result's white dotted line that algorithm draws in the document is more clear, that is to say that error rate is higher.In addition, the erroneous point major part occurs in the fringe region of object, so the problem that the processing at edge still will further be optimized in the future.
Algorithm of the present invention is at unshielding, and the error rate of three aspects of error rate of discontinuous and Zone Full all is lower than the algorithm in the document.Than complex image such as Venus, error rate can be reduced to 30% of document algorithm for background; Than complex image such as Teddy and Cones, error rate can be reduced to 80% of document algorithm for background.
At last; We have compared the Y-PSNR that different virtual is looked composition algorithm; We can draw, and for the fairly simple image of background (Venus), algorithm of the present invention has improved 0.1 to 0.3dB than algorithm in the document; And for the image (Teddy and Cones) of more complicated, algorithm of the present invention has improved 0.4 to 1dB than algorithm in the document.
Claims (4)
1. the medial view synthetic method based on improvement BP algorithm is characterized in that comprising the steps:
Step 1) adopts the method based on the iteration of improving BP, obtains the difference vector information of each pixel;
Wherein, described method based on the iteration of improving BP is following:
The first step, the initial matching cost when calculating the 0th grade, the data item during then to each rank is carried out initialization; The color distance of the node when calculating each rank and its neighbours' domain node, and the color distance value between the adjacent node during to each rank is carried out initialization;
Second step, total L level, l=0,1 ..., L, the iterations of each grade are t=0,1 ..., T, L and T are the natural number greater than 1;
In the 3rd step, utilize formula
M (d) expression message wherein, the maximum iteration time of T representative when the l-1 level, (x y) is the position of node p when the l level, and u is the node that comprises node p in the l level in the l-1 level, is that 0 o'clock message is carried out initialization to iterations; Then according to formula
Wherein c is a constant value, | d
p-d
q| the expression disparity difference,
Calculate the message m of each node
L, t(d
q), and be the minimum value of information minimum of each node selection;
In the 4th step, the message of calculating gained is handled as follows:
Add minimum
The ρ of formula (I), wherein, τ is the cutoff value of color distance, T
1And T
2The difference that is respectively the label between the neighborhood node was greater than 1 o'clock minimum and maximum level and smooth cost, and I is the color distance value, and saves as MIN;
The 5th step, compare value in the message vector and MIN, if less than MIN, then the value of correspondence position substitutes with MIN in the message vector, at last the message vector is carried out normalization, gets message values to the end;
In the 6th step, iterations adds 1, repeats for the 3rd to the 5th step, is T up to iterations;
In the 7th step, progression subtracts 1, repeats for the 3rd to the 6th step, becomes 0 up to progression;
The 8th step is at last according to formula
D wherein
q(d
q) the expression data item, the optimum label that obtains each node and be pixel is a parallax;
Step 2); Detect occlusion area; And the parallax of occlusion area compensated, and then carry out the interpolation calculation of medial view: at first utilize the consistency constraint condition that image is carried out dividing region, be divided into occlusion area, consistent zone and fuzzy region to image; Meanwhile carry out the compensation of occlusion area, take different interpolation to obtain medial view according to occlusion area, consistent zone and the characteristics of fuzzy region itself then.
2. according to claim 1 based on the medial view synthetic method of improving the BP algorithm; It is characterized in that step 2) the described consistency constraint condition of utilizing carries out dividing region to image; Being divided into occlusion area, consistent zone and fuzzy region to image is meant: at first, and for left disparity map d
LRUtilize formula d
LR(x
L, y)-d
RL(x
L-d
LR(x
L, y), y)＞T, wherein T is a threshold value, detects right half occlusion area, then for right disparity map d
RLUtilize formula d
RL(x
R, y)-d
LR(x
R+ d
RL(x
R, y), y)＞T detects left half occlusion area; And respectively a left side half occlusion area in half occlusion area of the right side in the left disparity map and the right disparity map is carried out mark; In addition, the parallax value of the rightmost partial pixel point of the Far Left partial pixel of left disparity map and right disparity map all composes 0; Then, to satisfying formula d in the left disparity map
LR(x
L, y)=d
RL(x
L-d
LR(x
L, y), pixel region y) is labeled as right consistent zone, for satisfying formula d in the right disparity map
RL(x
R, y)=d
LR(x
R+ d
RL(x
R, y), pixel region y) is labeled as the first from left and causes the zone; At last, the zone that is not labeled in the left disparity map, being labeled as right fuzzy region, is the zone marker that is not labeled in the right disparity map left fuzzy region.
3. according to claim 1 based on the medial view synthetic method of improving the BP algorithm, it is characterized in that step 2) described interpolation method step is following:
At first, be reference with left disparity map, utilize formula:
Then, be reference with right disparity map, utilize formula:
At last, left and right sides transition medial view is looked like to utilize formula:
4. according to claim 3 based on the medial view synthetic method of improving the BP algorithm, it is characterized in that the medial view I that obtains synthesizing
MIn still have the zone do not shone upon, take the method for reverse mapping, in horizontal parallax figure, seek its immediate point respectively,
Choose respectively and satisfy formula
And formula
Horizontal parallax figure in optimal match point, wherein the hunting zone of the independent variable of two formulas is respectively formula x
M≤x
L≤x
M+ (1-a) d
MaxWith formula x
M-ad
Max≤x
R≤x
M, the x of correspondence when formula was got minimum value after arg represented; Utilize formula aI then
LM(x
L ', y)+(1-a) I
RM(x
R ', y) interpolation is carried out in the zone of not shone upon, obtained final medial view picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101826345A CN101873508B (en) | 2010-05-25 | 2010-05-25 | Intermediate view synthesis method based on improved BP (Belief Propagation) algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101826345A CN101873508B (en) | 2010-05-25 | 2010-05-25 | Intermediate view synthesis method based on improved BP (Belief Propagation) algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101873508A CN101873508A (en) | 2010-10-27 |
| CN101873508B true CN101873508B (en) | 2012-07-04 |
Family
ID=42998124
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101826345A Active CN101873508B (en) | 2010-05-25 | 2010-05-25 | Intermediate view synthesis method based on improved BP (Belief Propagation) algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101873508B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102802004A (en) * | 2012-08-15 | 2012-11-28 | 上海易维视科技有限公司 | 3-dimensional module for naked eyes |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102075779B (en) * | 2011-02-21 | 2013-05-08 | 北京航空航天大学 | Intermediate view synthesizing method based on block matching disparity estimation |
| CN102254348B (en) * | 2011-07-25 | 2013-09-18 | 北京航空航天大学 | Virtual viewpoint mapping method based o adaptive disparity estimation |
| CN102447934B (en) * | 2011-11-02 | 2013-09-04 | 吉林大学 | Synthetic method of stereoscopic elements in combined stereoscopic image system collected by sparse lens |
| CN103593641B (en) * | 2012-08-16 | 2017-08-11 | 株式会社理光 | Object detecting method and device based on stereo camera |
| CN104766275B (en) * | 2014-01-02 | 2017-09-08 | 株式会社理光 | Sparse disparities figure denseization method and apparatus |
| CN105007478A (en) * | 2015-07-08 | 2015-10-28 | 南京信息工程大学 | View synthesis method based on mean shift stereo matching |
| CN106530409B (en) * | 2016-11-03 | 2019-08-27 | 浙江大学 | Regional area consistency corresponding method in Stereo matching |
| CN106648109A (en) * | 2016-12-30 | 2017-05-10 | 南京大学 | Real scene real-time virtual wandering system based on three-perspective transformation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1687961A (en) * | 2005-04-19 | 2005-10-26 | 浙江大学 | Computerized truth identifying method for traditional Chinese painting |
| CN101256625A (en) * | 2007-12-19 | 2008-09-03 | 重庆大学 | Method for extracting human ear image edge combining multiple methods |
| CN101312540A (en) * | 2008-07-03 | 2008-11-26 | 浙江大学 | Virtual visual point synthesizing method based on depth and block information |
| CN101605270A (en) * | 2009-07-16 | 2009-12-16 | 清华大学 | Generate the method and apparatus of depth map |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7599547B2 (en) * | 2005-11-30 | 2009-10-06 | Microsoft Corporation | Symmetric stereo model for handling occlusion |
-
2010
- 2010-05-25 CN CN2010101826345A patent/CN101873508B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1687961A (en) * | 2005-04-19 | 2005-10-26 | 浙江大学 | Computerized truth identifying method for traditional Chinese painting |
| CN101256625A (en) * | 2007-12-19 | 2008-09-03 | 重庆大学 | Method for extracting human ear image edge combining multiple methods |
| CN101312540A (en) * | 2008-07-03 | 2008-11-26 | 浙江大学 | Virtual visual point synthesizing method based on depth and block information |
| CN101605270A (en) * | 2009-07-16 | 2009-12-16 | 清华大学 | Generate the method and apparatus of depth map |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102802004A (en) * | 2012-08-15 | 2012-11-28 | 上海易维视科技有限公司 | 3-dimensional module for naked eyes |
| CN102802004B (en) * | 2012-08-15 | 2016-06-01 | 上海易维视科技有限公司 | Bore hole 3D module |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101873508A (en) | 2010-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101873508B (en) | Intermediate view synthesis method based on improved BP (Belief Propagation) algorithm | |
| Chang et al. | Content-aware display adaptation and interactive editing for stereoscopic images | |
| US8711204B2 (en) | Stereoscopic editing for video production, post-production and display adaptation | |
| US9445072B2 (en) | Synthesizing views based on image domain warping | |
| US9414048B2 (en) | Automatic 2D-to-stereoscopic video conversion | |
| Feng et al. | Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications | |
| Wu et al. | Repetition-based dense single-view reconstruction | |
| US8928654B2 (en) | Methods, systems, devices and associated processing logic for generating stereoscopic images and video | |
| Jeong et al. | Motion-compensated frame interpolation based on multihypothesis motion estimation and texture optimization | |
| EP3735677A1 (en) | Fusing, texturing, and rendering views of dynamic three-dimensional models | |
| US8666146B1 (en) | Discontinuous warping for 2D-to-3D conversions | |
| US20150022631A1 (en) | Content-aware display adaptation methods and editing interfaces and methods for stereoscopic images | |
| CN101287142A (en) | Method for converting flat video to tridimensional video based on bidirectional tracing and characteristic points correction | |
| JP2021535466A (en) | Methods and systems for reconstructing scene color and depth information | |
| JP2014078095A (en) | Image processing device, image processing method, and program | |
| CN113538667B (en) | Dynamic scene light field reconstruction method and device | |
| KR20090008808A (en) | Apparatus and method for converting 2d image signals into 3d image signals | |
| CN103581650A (en) | Method for converting binocular 3D video into multicast 3D video | |
| Devernay et al. | Adapting stereoscopic movies to the viewing conditions using depth-preserving and artifact-free novel view synthesis | |
| CN105657401A (en) | Naked eye 3D display method and system and naked eye 3D display device | |
| Hsu et al. | Spatio-temporally consistent view synthesis from video-plus-depth data with global optimization | |
| CN103634584B (en) | A kind of multiple views 3D image synthesizing method | |
| CN101945299B (en) | Camera-equipment-array based dynamic scene depth restoring method | |
| US20200077071A1 (en) | Apparatus and method for processing a depth map | |
| Sharma | Uncalibrated camera based content generation for 3D multi-view displays |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: STATE GRID CORPORATION OF CHINA JIANGSU ELECTRIC P Effective date: 20140609 Owner name: INFORMATION COMMUNICATION BRANCH JIANGSU ELECTRIC Free format text: FORMER OWNER: SOWTHEAST UNIV. Effective date: 20140609 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 215123 SUZHOU, JIANGSU PROVINCE TO: 210024 NANJING, JIANGSU PROVINCE |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20140609 Address after: 210024 No. 20 West Beijing Road, Jiangsu, Nanjing Patentee after: Information & Telecommunication Branch of State Grid Jiangsu Electric Power Company Patentee after: State Grid Corporation of China Patentee after: Jiangsu Electric Power Company Address before: 215123 Jiangsu city of Suzhou province Dushu Lake Industrial Park High Linquan parish Street No. 399 Patentee before: Southeast University |