CN105007495A - Difference estimation method based on multiple layers of 3DRS and device thereof - Google Patents
Difference estimation method based on multiple layers of 3DRS and device thereof Download PDFInfo
- Publication number
- CN105007495A CN105007495A CN201510514518.1A CN201510514518A CN105007495A CN 105007495 A CN105007495 A CN 105007495A CN 201510514518 A CN201510514518 A CN 201510514518A CN 105007495 A CN105007495 A CN 105007495A
- Authority
- CN
- China
- Prior art keywords
- block
- pyramid
- iteration
- difference
- searching route
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention provides a difference estimation method based on multi-layer 3DRS and a device thereof. The method comprises a step of building an image pyramid with preset layers according to the left image and a right image of a 3D frame image and generating a block grid difference field pyramid with a preset block size according to the image pyramid, a step of setting the search path of iteration times, wherein the search in each time of iteration is search row by row or column by column from one end of each row or column to the other end, and the search paths of two adjacent rows or columns are opposite, a step of completing the multi-layer 3DRS block estimation iteration calculation with preset iteration times on each layer of block grid difference field pyramid, and obtaining the each block difference estimation value of a most bottom layer in the block grid difference field pyramid. According to the method and the device, the search paths of two adjacent rows or columns are opposite, and thus the difference estimation result of a 3D frame image is more balanced and reliable so as to obtain an accurate and uniform depth estimation result.
Description
Technical field
The present invention relates to the process field of 3D rendering, particularly relate to a kind of difference estimation method based on 3DRS and device.
Background technology
3DRS (3-Dimension Recursive Search, multi-layer three-dimension recursive search) algorithm is a kind of motion estimation algorithm, proposed in " True-Motion Estimation with 3-DRecursive Search Block Matching " literary composition in 1993 by Gerard de Haan.3DRS algorithm in original text first with block (block) for the difference between unit computed image and reference picture, then by block corrosion (BlockErosion) method the sports ground (Motion Field) under block grid (block grid) or difference field (Displacement Field) expanded to the final result under pixel grid (Pixel grid).The recurrence of initial 3DRS indication is on image space, there is no the way of iteration at first, and its convergence direction is determined by its estimator (Estimator) and the direction of search.
Improvement after 3DRS proposes remains its basic thought.By improving the ACS (asynchronous cyclic search) table " Sub-pixel motion estimation with 3D Recursiveblock matching " in former algorithm, express the methods such as the combination " Optimizationof Hierarchical 3DRS Motion Estimators for Picture Rate Conversion " of (Pyramidal Representation) with pyramid, define the multilayer 3DRS algorithm (Hierarchical 3DRS) with sub-pixel precision.
3D changes automatically, refers to traditional about 3D bitmap-format, converts 2D+Z (degree of depth) form that can be used for multi-angle (Multi-view) 3D rendering and generate to.At present, multilayer 3DRS algorithm is widely used in the estimation of Depth of 3D two field picture in 3D video, the depth estimation algorithm of multilayer 3DRS with pixel-recursive algorithm or 3DRS for core.But 3DRS algorithm and multilayer 3DRS algorithm realization all have employed the mode of comparatively simple searching route, as left-to-right line by line, by the upper left corner to the lower right corner.Under the prerequisite of every layer of successive ignition, simple searching route fails to make full use of the advantage of successive ignition, and the difference estimation obtained after making search iteration may exist uneven situation, thus has influence on the difference estimation result that in 3D video, 3D frame pixel is final.How making difference estimation result more balanced, reliable, is the problem that industry needs solution badly.
Summary of the invention
The invention provides a kind of difference estimation method based on multilayer 3DRS and device, for solving more balanced, the reliable problem of the difference estimation result how making 3D two field picture.
The embodiment of the present invention is by the following technical solutions:
On the one hand, the invention provides a kind of difference estimation method based on multilayer 3DRS, the method comprises:
Build the image pyramid of the default number of plies according to the left figure of 3D two field picture and right figure respectively, generate the block grid difference field pyramid presetting block size according to described image pyramid;
Arrange the searching route of default iterations, wherein the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column;
Successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
On the other hand, present invention also offers a kind of difference estimation device based on multilayer 3DRS, this device comprises:
Pyramid generation unit, for building the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, generates the block grid difference field pyramid presetting block size according to described image pyramid;
Searching route setting unit, for the searching route of default iterations, wherein the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column;
Iterative computation unit, be connected with searching route setting unit with pyramid generation unit, for successively using described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Compared with prior art, a kind of difference estimation method based on multilayer 3DRS provided by the invention and device, have following beneficial effect:
The present invention is by adopting the searching route that adjacent rows/row searching route is contrary, in the pyramid of the present embodiment gained block grid difference field, each piece of difference estimation value of the bottom is comparatively even generally, effectively prevent the generation causing " abnormity point " because searching route is single; Employing makes each iteration compared to last iteration by image and searching route 90-degree rotation, Iterative path is made to become four-way iteration, Iterative path is made to become the four-way iteration of each iteration 90-degree rotation, and adopt the tilting Y estimator matched, make the difference estimation result of 3D two field picture more balanced, reliable, the uniformity that further reinforcement multilayer 3DRS algorithm is restrained in different directions, to obtain more accurate, uniform depth estimation result, the present invention effectively inhibits 3DRS algorithm to estimate the generation of singular point in difference field.Adopt pixel value difference method to supplement when building image pyramid, image information is preserved more complete, pyramidal generation is more accurate.
Accompanying drawing explanation
Fig. 1 is the method flow diagram of first embodiment of a kind of difference estimation method based on multilayer 3DRS provided by the invention.
Fig. 2 is the method flow diagram of second embodiment of a kind of difference estimation method based on multilayer 3DRS provided by the invention.
Fig. 3 is the structured flowchart of first embodiment of a kind of difference estimation device based on multilayer 3DRS provided by the invention.
Fig. 4 is the structured flowchart of second embodiment of a kind of difference estimation device based on multilayer 3DRS provided by the invention.
Fig. 5 a is a kind of difference estimation method based on multilayer 3DRS provided by the invention and the searching route of device, the schematic diagram one of estimator.
Fig. 5 b is a kind of difference estimation method based on multilayer 3DRS provided by the invention and the searching route of device, the schematic diagram two of estimator.
Fig. 5 c is a kind of difference estimation method based on multilayer 3DRS provided by the invention and the searching route of device, the schematic diagram three of estimator.
Fig. 5 d is a kind of difference estimation method based on multilayer 3DRS provided by the invention and the searching route of device, the schematic diagram four of estimator.
Embodiment
The technical problem solved for making the present invention, the technical scheme of employing and the technique effect that reaches are clearly, be described in further detail below in conjunction with the technical scheme of accompanying drawing to the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those skilled in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
Fig. 1 illustrates the method flow diagram of first embodiment according to a kind of difference estimation method based on multilayer 3DRS provided by the invention.A kind of estimation of Depth side based on multilayer 3DRS of the present embodiment performs primarily of intelligent terminal.Wherein, intelligent terminal includes but not limited to: PC, notebook computer, mobile phone, panel computer, video playback apparatus etc., intelligent terminal can be the intelligent terminal playing or resolve 3D video.The method comprises the following steps:
11: the image pyramid building the default number of plies according to the left figure of 3D two field picture and right figure respectively, the block grid difference field pyramid presetting block size is generated according to described image pyramid.
Concrete, adopt average weighted method to build the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, described image pyramid is initialized as the block grid difference field pyramid of default block size.Image pyramid comprises left figure pyramid and right figure pyramid.The left figure of 3D two field picture builds the left figure pyramid of the default number of plies, and the right figure of 3D two field picture also builds the right figure pyramid of the default number of plies, and wherein, the length and width of high one deck image pyramid is 1/2 times of the length and width of end one deck image pyramid respectively.Last layer generates based on its lower one deck.For the arbitrary pixel (x, y) in last layer, its value is obtained by the M*N region in layer 1 centered by pixel (2x, 2y) and weighted average filtering core phase multiply accumulating, for level and smooth consideration in reality, can use larger region.Generate the block grid difference field pyramid presetting block size according to image pyramid, difference field pyramidal generation method is that the phasor difference of coordinate corresponding compared to identical layer image slices vegetarian refreshments in left layer pyramid for identical layer image pixel coordinates in right figure pyramid is formed difference field pyramid.Block size is generally square pixel block, as 8*8, then difference field pyramid is divided into the block of several 8*8 by the block grid that is 8*8 with grid block size, obtains block grid difference field pyramid.
In the present embodiment, during pyramidal every layer of synthetic image, first obtain the size that can generate this layer according to the size of 3D two field picture, divided by the block length of side, if cannot divide exactly, be then the size that can be divided exactly by the block length of side by 3D two field picture sectional drawing, namely lost a not enough block length of side pixel row or column, generate the block grid difference field pyramid presetting block size.
Such as: for the 3D two field picture of 1080*960 (long * is wide) pixel, pyramid is 3 layers, by generation be successively from bottom to up: the image pyramid of 1080*960,540*480,270*240 pixel; Directly can judge that the picture size of 3D two field picture is that block size adopts 8*8 pixel, each layer block grid difference field pyramid size is respectively 135*120 block, 67*60 block and 33*30 block.Difference field pyramid has abandoned the part less than 8 pixels when generating, the length as the second layer is 540 pixels/8=67.5, loses the part less than 8 pixels, becomes 67.
12: the searching route arranging default iterations, wherein the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column.
Concrete, search for line by line if arrange, then search the other end by from the one end of often going, adjacent rows searching route is contrary, and even a behavior is searched for from left to right, then next behavior is searched for from right to left; Search for by column if arrange, search the other end from the one end often arranged, adjacent two row searching routes are contrary, and even one be classified as from top to bottom, then next is classified as from bottom to up.
For the searching route of searching for line by line, if iterations is 4, searches for line by line, from top to bottom search for line by line if arrange, odd-numbered line search is for search for from left to right, and even number line search is for search for from right to left.
Further, adjacent twice searching route is contrary, and the terminal of last search is the searching route of the starting point of searching for next time, as above example, if the search starting point of odd-times iteration is the upper left corner, the search terminal of iteration is the lower right corner, and the search starting point of even-times iteration is the lower right corner, the search terminal upper left corner of iteration.
13: successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Concrete, from top, successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Wherein, multilayer 3DRS block estimates that iterative computation is the block estimation iterative computation step of each piece of difference estimation value obtaining the difference field pyramid bottom in multilayer 3DRS algorithm, but this step changes structure and the searching route of estimator in multilayer 3DRS algorithm.
As shown in accompanying drawing 5a and 5c, estimator is that the spatial prediction block of this non-iterative computation in the tilting Y estimator (ObliqueY estimator) mated with searching route is placed on the block position of the iteration of tilting Y estimator center line symmetry with this.
Based on tilting Y estimator, do following estimator design.To arrange the searching route of searching for line by line, for odd-times iteration (Fig. 5 a), when searching for from left to right, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), time prediction block (X+2, Y+2) structure; When searching for from right to left, estimator structure is spatial prediction block (X-1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2).For even-times iteration (Fig. 5 c), when searching for from right to left, estimator is spatial prediction block (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2) structure; When searching for from left to right, estimator structure is spatial prediction block (X+1, Y), (X, Y-1), hierarchical prediction block (X, Y), time prediction block (X+2, Y+2).
It should be noted that, when the interlaced alternative direction of search, according to tilting Y estimator, due to block (X+1, Y) (odd iteration), (X-1, Y) (even iteration) cannot as spatial prediction block (not yet calculating), therefore, select (X+1, Y) (odd iteration), (X+1, Y) (even iteration) replaces.
According to each candidate vector of the current computing block that estimated value obtains, candidate vector comprises: spatial candidate vector (Spatial Candidate), time candidate vector (Temporal Candidate), layering candidate vector (Hierarchy Candidate), renewal vector (Update Candidate).Wherein, to be multiplied by by the difference estimation value of last layer corresponding blocks by spatial prediction block 2 obtain spatial candidate vector (Spatial Candidate), obtain time candidate vector (TemporalCandidate) by time prediction block by the difference estimation result of previous frame or last iteration, the difference estimation value that completed block by current iteration by hierarchical prediction block obtains layering candidate vector (Hierarchy Candidate), upgrade vector (Update Candidate) shows to generate random vector by spatial candidate vector and ASC and superpose and generate.
Difference (Distortion) between the comparison window relatively being pointed to region centered by the comparison window in region centered by the band of position and current computing block by candidate vector each in the estimator centered by current computing block, the corresponding vector of selection differential reckling is as the result of calculation of current block.Point to the band of position respectively, difference calculates in certain comparison window (Search window), and is retrained by comparison range (Search Area).A kind of way of routine is set to consistent with block size by the size of comparison window, as 8*8 pixel in example.Difference completes calculating (namely calculating in units of pixel) under pixel grid, the normal form adopting absolute difference and (SAD:sumof absolute difference) or absolute difference average (MAD:mean of absolute difference).
Upgrade vector shown with ASC by spatial candidate vector in random vector superpose and form.Wherein, because ASC table has sub-pixel precision (as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is pointed to non-integer center, interpolation algorithm need be adopted to obtain the value of non-integer coordinates position.
Each multilayer 3DRS block is estimated to include in iterative process: judge that whether the sensing band of position of each candidate vector of the current computing block that estimated value obtains is integer (the sensing band of position corresponding center position coordinates whether be integer), if so, the difference between comparison window that this candidate vector points to region centered by the comparison window in region centered by the band of position and current computing block is then directly calculated; If not, then go out by dull segmentation cube interpolation method interpolation the comparison window of regional center that this candidate vector points to, then centered by the comparison window of regional center pointed to of this candidate vector of obtaining of calculated difference and current computing block region comparison window between difference; Select with current computing block centered by the minimum candidate vector of the difference of comparison window in region as the difference estimation result of current computing block current iteration.
Such as, when computing block grid (10,10) during position, if spatial candidate vector is (1,1), ASC vector is (0.25,0) band of position that center is (11.25,11) will, then be pointed to, the value of 8*8 point in the region needing interpolation to go out position centered by (11.25,11).
The method of the present embodiment, compared with the searching route of searching in the same way line by line, in the pyramid of the present embodiment gained block grid difference field, each piece of difference estimation value of the bottom is comparatively even generally, effectively prevent the generation causing " abnormity point " because searching route is single.
Fig. 2 illustrates the method flow diagram according to a kind of difference estimation method second embodiment based on multilayer 3DRS provided by the invention.A kind of difference estimation method based on multilayer 3DRS of the present embodiment performs primarily of intelligent terminal.Wherein, intelligent terminal includes but not limited to: PC, notebook computer, mobile phone, panel computer, video playback apparatus etc., intelligent terminal can be the intelligent terminal playing or resolve 3D video.The method comprises the following steps:
21: the image pyramid building the default number of plies according to the left figure of 3D two field picture and right figure respectively, the block grid difference field pyramid presetting block size is generated according to described image pyramid.
Concrete, adopt average weighted method to build the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, described image pyramid is initialized as the block grid difference field pyramid of default block size.Image pyramid comprises left figure pyramid and right figure pyramid.The left figure of 3D two field picture builds the left figure pyramid of the default number of plies, and the right figure of 3D two field picture also builds the right figure pyramid of the default number of plies, and wherein, the length and width of high one deck image pyramid is 1/2 times of the length and width of end one deck image pyramid respectively.Generate the block grid difference field pyramid presetting block size according to image pyramid, difference field pyramidal generation method is that the phasor difference of coordinate corresponding compared to identical layer image slices vegetarian refreshments in left layer pyramid for identical layer image pixel coordinates in right figure pyramid is formed difference field pyramid.Block size is generally square pixel block, as 8*8, then difference field pyramid is divided into the block of several 8*8 by the block grid that is 8*8 with grid block size, obtains block grid difference field pyramid.
In the present embodiment, this step 21 comprises step 211 and step 212.
211: judge the left figure of 3D two field picture and the length of right figure and wide whether respectively can by 2 of the block length of side
ndoubly divide exactly, n, for presetting the number of plies, if so, then builds the multilayer 3DRS pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, if not, then adopt nearest interpolation in the left figure of 3D two field picture and the surrounding benefit value of right figure to can by 2 of the block length of side
nthe size doubly divided exactly, then the pyramid building the default number of plies according to the left figure of the 3D two field picture after benefit value and right figure.
Whether such as, be preset as 3 layers, block is of a size of 8*8 pixel, then judge the left figure of 3D two field picture and the length of right figure and widely to be divided exactly by 64, if can not, then adopt nearest interpolation to make the surrounding benefit value of the left figure of 3D two field picture and right figure to the size that can be divided exactly by 64.
212: the block grid difference field pyramid described multilayer pyramid being initialized as default block size.
22: the searching route that default iterations is set, each iteration compared to last iteration by image relevant relative to current iteration for searching route and each difference field pattern 90-degree rotation, the searching route of each iteration is searches for by row/column, the other end is searched, the searching route that adjacent rows/row searching route is contrary from one end of every row/column.
Concrete, each iteration compared to last iteration by image relevant relative to current iteration for searching route and the equal 90-degree rotation of each difference field pattern, can for turning clockwise or being rotated counterclockwise, wherein, the image can be correlated with by rotation current iteration and each difference field are realized, to be rotated counterclockwise, comprise: during each iteration, after relevant image and each difference field pattern are rotated counterclockwise 90 degree, left-right reversed is carried out to each vector in each difference field after upset, and negate process realization is done to each vector x durection component after exchanging.Specific implementation is shown in two kinds of implementations in step 23.
In each iteration, search for line by line if arrange, then search the other end by from the one end of often going, adjacent rows searching route is contrary, and even a behavior is searched for from left to right, then next behavior is searched for from right to left; Search for by column if arrange, search the other end from the one end often arranged, adjacent two row searching routes are contrary, and even one be classified as from top to bottom, then next is classified as from bottom to up.
23: successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Concrete, from top, successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Wherein, multilayer 3DRS block estimates that iterative computation is the block estimation iterative computation step of each piece of difference estimation value obtaining the difference field pyramid bottom in multilayer 3DRS algorithm, but this step changes structure and the searching route of estimator in multilayer 3DRS algorithm.
Further, the multilayer 3DRS block each layer block grid difference field pyramid being completed to default iterations estimates iterative computation, comprising:
After the iterative computation of multilayer 3DRS block estimation is each time completed to each layer block grid difference field pyramid, judge whether described time iterations of working as reaches default iterations, if so, then stops this layer of block grid difference field pyramid iterative computation; If not, then adopt described searching route to estimate iterative computation to multilayer 3DRS block next time, then repeat current iteration process and calculate as next iteration, until reach default iterations.
The searching route that step 23 adopts above-mentioned steps 22 to preset, can adopt and realize in two ways:
Example one, estimator are the block position of the iteration spatial prediction block of this non-iterative computation in tilting Y estimator being placed on tilting Y estimator center line symmetry with this, then estimator need mate with searching route.For searching route for search for line by line, as shown in accompanying drawing 5a, 5b, 5c and 5d, such as, adopt and search for line by line, if presetting iterations is 4, iteration is 5a for the first time, has made following estimator design with the tilting Y estimator mated with searching route.For 4 iteration, (a), estimator design is identical with the estimator of the example odd-times iteration in the step 23 of the first embodiment for Fig. 5 for the 1st iteration.During the 2nd iteration (Fig. 5 b), search for from top to bottom, from the upper left corner to the lower right corner, be equal to after the image of being correlated with to second time multilayer 3DRS block estimation iterative computation and each difference field pattern being rotated counterclockwise 90 degree in this step, left-right reversed is carried out to each vector in each difference field after upset, negate process is done to each vector x durection component after exchanging, repeat current iteration process again as second time iterative computation, and this kind does not need the implementation of image rotating and each difference field pattern, more easily by intuitivism apprehension, the estimator of this kind of implementation is spatial prediction block is (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y+2) structure.In like manner can obtain, in this implementation, accompanying drawing (Fig. 5 c) is seen in the iterative search path of third time, and estimator design is identical with the estimator of the even-times iteration of the example in the step 23 of the first embodiment.In like manner can obtain, in this implementation, accompanying drawing (Fig. 5 d) is seen in the iterative search path of third time, during upper extremely lower search, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2).With search for identical in interlaced alternative direction, estimator design time for cannot as spatial prediction block (not yet calculating) block use symmetry blocks replace.
Example two, the search pattern of the iteration of first time in accompanying drawing 5a and estimator is adopted to estimate.After completing the 1st iteration, second time multilayer 3DRS block is estimated the image that iterative computation is relevant and each difference field pattern turn over counterclockwise and turn 90 degrees.Left and right is carried out for each vector in each difference field after upset exchange, and negate process is done to vector x durection component, that is:
V(x,y)＝(V
x(x,y),V
y(x,y))
T
V′(x,y)＝(-V
y(y,x),V
x(y,x))
T
Wherein, V (x, y) is first time iterative search result vector, and V ' (x, y) is vector after upset negate.Repeat iterative process for the first time and, as second time iteration, namely carry out iterative computation as second time iteration according to accompanying drawing 5a searching route and estimator.In like manner third time multilayer 3DRS block is estimated that the image that iterative computation is relevant and each difference field pattern turn over turn 90 degrees according to above-mentioned, each vector carries out left and right and exchanges, and negate process is done to vector x durection component, repeat the 1st iterative process again and carry out third time iteration as second time iteration, in like manner complete 4 iteration or more high reps iteration more successively.
Wherein, the above-mentioned block of multilayer 3DRS next time estimate the image that iterative computation is correlated with and each difference field pattern for for obtain next iteration calculate needed for spatial candidate vector, time candidate vector, layering candidate vector, upgrade vectorial image and each difference field pattern, also comprise the result when time search iteration obtains, namely when each piece of difference estimation value in this layer of block grid difference field pyramid after secondary iteration.
According to each candidate vector of the current computing block that estimated value obtains, candidate vector comprises: spatial candidate vector, time candidate vector, layering candidate vector, renewal vector.Wherein, to be multiplied by by the difference estimation value of last layer corresponding blocks by spatial prediction block 2 obtain spatial candidate vector, obtain time candidate vector by time prediction block by the difference estimation result of previous frame or last iteration, the difference estimation value that completed block by current iteration by hierarchical prediction block obtains layering candidate vector, upgrade vector shows to generate random vector by spatial candidate vector and ASC and superpose and generate.
Difference between the comparison window relatively being pointed to region centered by the comparison window in region centered by the band of position and current computing block by candidate vector each in the estimator centered by current computing block, the corresponding vector of selection differential reckling is as the result of calculation of current block.Point to the band of position respectively, difference calculates in certain comparison window, and is retrained by comparison range.A kind of way of routine is set to consistent with block size by the size of comparison window, as 8*8 pixel in example.Difference completes calculating (namely calculating in units of pixel) under pixel grid, normal adopts absolute difference and or the average form of absolute difference.
Upgrade vector shown with ASC by spatial candidate vector in random vector superpose and form.Wherein, because ASC table has sub-pixel precision (as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is pointed to non-integer center, interpolation algorithm need be adopted to obtain the value of non-integer coordinates position.
Each multilayer 3DRS block is estimated to include in iterative process: judge that whether the sensing band of position of each candidate vector of the current computing block that estimated value obtains is integer (the sensing band of position corresponding center position coordinates whether be integer), if so, the difference between comparison window that this candidate vector points to region centered by the comparison window in region centered by the band of position and current computing block is then directly calculated; If not, then go out by dull segmentation cube interpolation method interpolation the comparison window of regional center that this candidate vector points to, then centered by the comparison window of regional center pointed to of this candidate vector of obtaining of calculated difference and current computing block region comparison window between difference; Select with current computing block centered by the minimum candidate vector of the difference of comparison window in region as the difference estimation result of current computing block current iteration.
24: according to each piece of difference estimation value generating depth map of the bottom in the pyramid of block grid difference field.
This step has a lot of mode to realize, usually by block corrosion, the expansion of block grid difference field can be obtained pixel difference field pattern, again by pixel difference field pattern being carried out difference deep conversion in the mode such as to table look-up or linear, obtain depth map in the interval that pixel difference field pattern is quantized to 0 to 255 (unit 1) the most at last, after generating depth map, just 3D two field picture can be converted to 2D+Z (degree of depth) form that can be used for various visual angles (Multi-view) 3D rendering and generate.
For estimation of Depth, more compact, scope is less difference field is conducive to less error in final 0 ~ 255 scope quantification.The different picture of two width is adopted to carry out contrast experiment, shown by experimental data: the four-way searching route multilayer 3DRS in the present embodiment and searching route multilayer 3DRS in the same way line by line, or first searching route in embodiment compare, minimum (the first width figure of disparity range, search for line by line: 72.25 pixels, interlaced alternative direction is searched for: 76 pixels, improves searching route: 63.5 pixels; Second width figure, searches for: 89.5 pixels line by line, and interlaced alternative direction is searched for: 100 pixels, improves searching route: 84.5 pixels), and in the present embodiment, effectively inhibit 3DRS algorithm to estimate the generation of singular point in difference field.
Fig. 3 illustrates the structured flowchart according to a kind of difference estimation device first embodiment based on multilayer 3DRS provided by the invention.A kind of difference estimation device based on multilayer 3DRS of the present embodiment performs primarily of intelligent terminal.Wherein, intelligent terminal includes but not limited to: PC, notebook computer, mobile phone, panel computer, video playback apparatus etc., intelligent terminal can be the intelligent terminal playing or resolve 3D video.This device comprises pyramid generation unit 31, searching route setting unit 32 and iterative computation unit 33.
Pyramid generation unit 31, for building the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, generates the block grid difference field pyramid presetting block size according to described image pyramid.
Concrete, adopt average weighted method to build the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, described image pyramid is initialized as the block grid difference field pyramid of default block size.Image pyramid comprises left figure pyramid and right figure pyramid.The left figure of 3D two field picture builds the left figure pyramid of the default number of plies, and the right figure of 3D two field picture also builds the right figure pyramid of the default number of plies, and wherein, the length and width of high one deck image pyramid is 1/2 times of the length and width of end one deck image pyramid respectively.Generate the block grid difference field pyramid presetting block size according to image pyramid, difference field pyramidal generation method is that the phasor difference of coordinate corresponding compared to identical layer image slices vegetarian refreshments in left layer pyramid for identical layer image pixel coordinates in right figure pyramid is formed difference field pyramid.Block size is generally square pixel block, as 8*8, then difference field pyramid is divided into the block of several 8*8 by the block grid that is 8*8 with grid block size, obtains block grid difference field pyramid.
In the present embodiment, during pyramidal every layer of synthetic image, first obtain the size that can generate this layer according to the size of 3D two field picture, divided by the block length of side, if cannot divide exactly, be then the size that can be divided exactly by the block length of side by 3D two field picture sectional drawing, namely lost a not enough block length of side pixel row or column, generate the block grid difference field pyramid presetting block size.
Such as: for the 3D two field picture of 1080*960 (long * is wide) pixel, pyramid is 3 layers, by generation be successively from bottom to up: the image pyramid of 1080*960,540*480,270*240 pixel; When directly can judge that the picture size of 3D two field picture is block size employing 8*8 pixel, each layer block grid difference field pyramid size is respectively 135*120 block, 67*60 block and 33*30 block.Difference field pyramid has abandoned the part less than 8 pixels when generating, the length as the second layer is 540 pixels/8=67.5, loses the part less than 8 pixels, becomes 67.
Searching route setting unit 32, for arranging the searching route of default iterations, wherein the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column.
Concrete, search for line by line if arrange, then search the other end by from the one end of often going, adjacent rows searching route is contrary, and even a behavior is searched for from left to right, then next behavior is searched for from right to left; Search for by column if arrange, search the other end from the one end often arranged, adjacent two row searching routes are contrary, and even one be classified as from top to bottom, then next is classified as from bottom to up.
For the searching route of searching for line by line, if iterations is 4, searches for line by line, from top to bottom search for line by line if arrange, odd-numbered line search is for search for from left to right, and even number line search is for search for from right to left.
Further, adjacent twice searching route is contrary, and the terminal of last search is the searching route of the starting point of searching for next time, as above example, if the search starting point of odd-times iteration is the upper left corner, the search terminal of iteration is the lower right corner, and the search starting point of even-times iteration is the lower right corner, the search terminal upper left corner of iteration.
Iterative computation unit 33, be connected with searching route setting unit with pyramid generation unit, for successively using described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Concrete, from top, successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Wherein, multilayer 3DRS block estimates that iterative computation is the block estimation iterative computation step of each piece of difference estimation value obtaining the difference field pyramid bottom in multilayer 3DRS algorithm, but this step changes structure and the searching route of estimator in multilayer 3DRS algorithm.
As shown in accompanying drawing 5a and 5c, estimator is that the spatial prediction block of this non-iterative computation in the tilting Y estimator (ObliqueY estimator) mated with searching route is placed on the block position of the iteration of tilting Y estimator center line symmetry with this.
Based on tilting Y estimator, do following estimator design.To arrange the searching route of searching for line by line, for odd-times iteration (Fig. 5 a), during left-to-right search, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), time prediction block (X+2, Y+2) structure; During right-to-left search, estimator structure is spatial prediction block (X-1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2).For even-times iteration (Fig. 5 c), during right-to-left search, estimator is spatial prediction block (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2) structure; During left-to-right search, estimator structure is spatial prediction block (X+1, Y), (X, Y-1), hierarchical prediction block (X, Y), time prediction block (X+2, Y+2).
It should be noted that, when the interlaced alternative direction of search, according to tilting Y estimator, due to block (X+1, Y) (odd iteration), (X-1, Y) (even iteration) cannot as spatial prediction block (not yet calculating), therefore, select (X+1, Y) (odd iteration), (X+1, Y) (even iteration) replaces.
According to each candidate vector of the current computing block that estimated value obtains, candidate vector comprises: spatial candidate vector, time candidate vector, layering candidate vector, renewal vector.
Difference between the comparison window relatively being pointed to region centered by the comparison window in region centered by the band of position and current computing block by candidate vector each in the estimator centered by current computing block, the corresponding vector of selection differential reckling is as the result of calculation of current block.Point to the band of position respectively, difference calculates in certain comparison window, and is retrained by comparison range.A kind of way of routine is set to consistent with block size by the size of comparison window, as 8*8 pixel in example.Difference completes calculating (namely calculating in units of pixel) under pixel grid, the normal form adopting absolute difference and (SAD:sum of absolute difference) or absolute difference average (MAD:mean of absolute difference).
Upgrade vector shown with ASC by spatial candidate vector in random vector superpose and form.Wherein, because ASC table has sub-pixel precision (as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is pointed to non-integer center, interpolation algorithm need be adopted to obtain the value of non-integer coordinates position.
Each multilayer 3DRS block is estimated to include in iterative process: judge that whether the sensing band of position of each candidate vector of the current computing block that estimated value obtains is integer (the sensing band of position corresponding center position coordinates whether be integer), if so, the difference between comparison window that this candidate vector points to region centered by the comparison window in region centered by the band of position and current computing block is then directly calculated; If not, then go out by dull segmentation cube interpolation method interpolation the comparison window of regional center that this candidate vector points to, then centered by the comparison window of regional center pointed to of this candidate vector of obtaining of calculated difference and current computing block region comparison window between difference; Select with current computing block centered by the minimum candidate vector of the difference of comparison window in region as the difference estimation result of current computing block current iteration.
The device of the present embodiment, compared with the searching route of searching in the same way line by line, in the pyramid of the present embodiment gained block grid difference field, each piece of difference estimation value of the bottom is comparatively even generally, effectively prevent the generation causing " abnormity point " because searching route is single.
Fig. 4 illustrates the structured flowchart according to a kind of difference estimation device second embodiment based on multilayer 3DRS provided by the invention.A kind of difference estimation device based on multilayer 3DRS of the present embodiment performs primarily of intelligent terminal.Wherein, intelligent terminal includes but not limited to: PC, notebook computer, mobile phone, panel computer, video playback apparatus etc., intelligent terminal can be the intelligent terminal playing or resolve 3D video.This device comprises pyramid generation unit 41, searching route setting unit 42, iterative computation unit 43 and depth map generating unit 44.
Pyramid generation unit 41, for building the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, generates the block grid difference field pyramid presetting block size according to described image pyramid.
Concrete, adopt average weighted method to build the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, described image pyramid is initialized as the block grid difference field pyramid of default block size.Image pyramid comprises left figure pyramid and right figure pyramid.The left figure of 3D two field picture builds the left figure pyramid of the default number of plies, and the right figure of 3D two field picture also builds the right figure pyramid of the default number of plies, and wherein, the length and width of high one deck image pyramid is 1/2 times of the length and width of end one deck image pyramid respectively.Generate the block grid difference field pyramid presetting block size according to image pyramid, difference field pyramidal generation method is that the phasor difference of coordinate corresponding compared to identical layer image slices vegetarian refreshments in left layer pyramid for identical layer image pixel coordinates in right figure pyramid is formed difference field pyramid.
In the present embodiment, pyramid generation unit 41 comprises to be built image pyramid module 411 and generates difference field pyramid module 412.
Build image pyramid module 411, for the left figure and right figure that judge 3D two field picture length and wide whether respectively can by 2 of the block length of side
ndoubly divide exactly, n, for presetting the number of plies, if so, then builds the multilayer 3DRS pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, if not, then adopt nearest interpolation in the left figure of 3D two field picture and the surrounding benefit value of right figure to can by 2 of the block length of side
nthe size doubly divided exactly, then the pyramid building the default number of plies according to the left figure of the 3D two field picture after benefit value and right figure.
Generate difference field pyramid module 412, for described multilayer pyramid being initialized as the block grid difference field pyramid of default block size.
Searching route setting unit 42, for arranging the searching route of default iterations, each iteration compared to last iteration by image relevant relative to current iteration for searching route and each difference field pattern 90-degree rotation, the searching route of each iteration is searches for by row/column, the other end is searched, the searching route that adjacent rows/row searching route is contrary from one end of every row/column.
Concrete, each iteration compared to last iteration by image relevant relative to current iteration for searching route and the equal 90-degree rotation of each difference field pattern, can for turning clockwise or being rotated counterclockwise, wherein, the image can be correlated with by rotation current iteration and each difference field are realized, to be rotated counterclockwise, comprise: during each iteration, after relevant image and each difference field pattern are rotated counterclockwise 90 degree, left-right reversed is carried out to each vector in each difference field after upset, and negate process realization is done to each vector x durection component after exchanging.During each iteration, after relevant image and each difference field pattern are rotated counterclockwise 90 degree, left-right reversed is carried out to each vector in each difference field after upset, and negate process realization is done to each vector x durection component after exchanging.Specific implementation is shown in two kinds of implementations in iterative computation unit 43.
Iterative computation unit 43, for successively using described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Concrete, from top, successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Wherein, multilayer 3DRS block estimates that iterative computation is the block estimation iterative computation step of each piece of difference estimation value obtaining the difference field pyramid bottom in multilayer 3DRS algorithm, but this step changes structure and the searching route of estimator in multilayer 3DRS algorithm.
Further, in the present embodiment, iterative computation unit 43 comprises iteration module 431 and estimation of Depth value acquisition module 432.
Iteration module 431, if for each layer block grid difference field pyramid is completed to default iterations multilayer 3DRS block estimate iterative computation time searching route for search for line by line, after then the iterative computation of multilayer 3DRS block estimation each time being completed to each layer block grid difference field pyramid, judge whether described time iterations of working as reaches default iterations, if so, then stop this layer of block grid difference field pyramid iterative computation; If not, then adopt described searching route to estimate iterative computation to multilayer 3DRS block next time, then repeat current iteration process and calculate as next iteration, until reach default iterations.
Iteration module 431 can be adopted and be realized in two ways:
Example one, estimator are the block position of the iteration spatial prediction block of this non-iterative computation in tilting Y estimator being placed on tilting Y estimator center line symmetry with this, then estimator need mate with searching route.For searching route for search for line by line, as shown in accompanying drawing 5a, 5b, 5c and 5d, such as, adopt and search for line by line, if presetting iterations is 4, iteration is 5a for the first time, has made following estimator design with the tilting Y estimator mated with searching route.For 4 iteration, (a), estimator design is identical with the estimator of the example odd-times iteration in the step 23 of the first embodiment for Fig. 5 for the 1st iteration.During the 2nd iteration (Fig. 5 b), search for from top to bottom, from the upper left corner to the lower right corner, be equal to after the image of being correlated with to second time multilayer 3DRS block estimation iterative computation and each difference field pattern being rotated counterclockwise 90 degree in this step, left-right reversed is carried out to each vector in each difference field after upset, negate process is done to each vector x durection component after exchanging, repeat current iteration process again as second time iterative computation, and this kind does not need the implementation of image rotating and each difference field pattern, more easily by intuitivism apprehension, the estimator of this kind of implementation is spatial prediction block is (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y+2) structure.In like manner can obtain, in this implementation, accompanying drawing (Fig. 5 c) is seen in the iterative search path of third time, and estimator design is identical with the estimator of the even-times iteration of the example in the step 23 of the first embodiment.In like manner can obtain, in this implementation, accompanying drawing (Fig. 5 d) is seen in the iterative search path of third time, during upper extremely lower search, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2).With search for identical in interlaced alternative direction, estimator design time for cannot as spatial prediction block (not yet calculating) block use symmetry blocks replace.
Example two, the search pattern of the iteration of first time in accompanying drawing 5a and estimator is adopted to estimate.After completing the 1st iteration, second time multilayer 3DRS block is estimated the image that iterative computation is relevant and each difference field pattern turn over counterclockwise and turn 90 degrees.Left and right is carried out for each vector in each difference field after upset exchange, and negate process is done to vector x durection component, that is:
V(x,y)＝(V
x(x,y),V
y(x,y))
T
V′(x,y)＝(-V
y(y,x),V
x(y,x))
T
Wherein, V (x, y) is first time iterative search result vector, and V ' (x, y) is vector after upset negate.Repeat iterative process for the first time and, as second time iteration, namely carry out iterative computation as second time iteration according to accompanying drawing 5a searching route and estimator.In like manner third time multilayer 3DRS block is estimated that the image that iterative computation is relevant and each difference field pattern turn over turn 90 degrees according to above-mentioned, each vector carries out left and right and exchanges, and negate process is done to vector x durection component, repeat the 1st iterative process again and carry out third time iteration as second time iteration, in like manner complete 4 iteration or more high reps iteration more successively.
Wherein, the above-mentioned block of multilayer 3DRS next time estimate the image that iterative computation is correlated with and each difference field pattern for for obtain next iteration calculate needed for spatial candidate vector, time candidate vector, layering candidate vector, upgrade vectorial image and each difference field pattern, also comprise the result when time search iteration obtains, namely when each piece of difference estimation value in this layer of block grid difference field pyramid after secondary iteration.
Estimator is for be placed on the block position of the iteration of tilting Y estimator center line symmetry with this by the spatial prediction block of this non-iterative computation in tilting Y estimator (Oblique Y estimator), then estimator need mate with searching route.For searching route for search for line by line, as shown in accompanying drawing 5a, 5b, 5c and 5d, do following estimator design with tilting Y estimator.For 4 iteration, (a), estimator design is identical with the estimator of the example odd-times iteration in the step 23 of the first embodiment for Fig. 5 for the 1st iteration.Time 2nd time iteration (Fig. 5 b), during upper extremely lower search, estimator is spatial prediction block is (X-1, Y), (X, Y+1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2); During lower supreme search, estimator is spatial prediction block is (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), the structure of time prediction block (X-2, Y+2).3rd iteration (Fig. 5 c), estimator design is identical with the estimator of the even-times iteration of the example in the step 23 of the first embodiment.Time the 4th iteration (Fig. 5 d), during lower supreme search, estimator is spatial prediction block is (X+1, Y), (X, Y-1), hierarchical prediction block (X, Y), the structure of time prediction block (X-2, Y+2); During upper extremely lower search, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2).With search for identical in interlaced alternative direction, estimator design time for cannot as spatial prediction block (not yet calculating) block use symmetry blocks replace.
Example two, the search pattern of the iteration of first time in accompanying drawing 5a and estimator is adopted to estimate.After completing the 1st iteration, second time multilayer 3DRS block is estimated the image that iterative computation is relevant and each difference field pattern turn over counterclockwise and turn 90 degrees.Left and right is carried out for each vector in each difference field after upset exchange, and negate process is done to vector x durection component, that is:
V(x,y)＝(V
x(x,y),V
y(x,y))
T
V′(x,y)＝(-V
y(y,x),V
x(y,x))
T
Wherein, V (x, y) is first time iterative search result vector, and V ' (x, y) is vector after upset negate.Repeat iterative process for the first time and, as second time iteration, namely carry out iterative computation as second time iteration according to accompanying drawing 5a searching route and estimator.In like manner third time multilayer 3DRS block is estimated that the image that iterative computation is relevant and each difference field pattern turn over turn 90 degrees according to above-mentioned, each vector carries out left and right and exchanges, and negate process is done to vector x durection component, repeat the 1st iterative process again and carry out third time iteration as second time iteration, in like manner complete 4 iteration or more high reps iteration more successively.
Wherein, the above-mentioned block of multilayer 3DRS next time estimate the image that iterative computation is correlated with and each difference field pattern for for obtain next iteration calculate needed for spatial candidate vector, time candidate vector, layering candidate vector, upgrade vectorial image and each difference field pattern, also comprise the result when time search iteration obtains, namely when each piece of difference estimation value in this layer of block grid difference field pyramid after secondary iteration.
According to each candidate vector of the current computing block that estimated value obtains, candidate vector comprises: spatial candidate vector (Spatial Candidate), time candidate vector (Temporal Candidate), layering candidate vector (Hierarchy Candidate), renewal vector (Update Candidate).Wherein, to be multiplied by by the difference estimation value of last layer corresponding blocks by spatial prediction block 2 obtain spatial candidate vector (Spatial Candidate), obtain time candidate vector (TemporalCandidate) by time prediction block by the difference estimation result of previous frame or last iteration, the difference estimation value that completed block by current iteration by hierarchical prediction block obtains layering candidate vector (Hierarchy Candidate), upgrade vector (Update Candidate) shows to generate random vector by spatial candidate vector and ASC and superpose and generate.
Difference (Distortion) between the comparison window relatively being pointed to region centered by the comparison window in region centered by the band of position and current computing block by candidate vector each in the estimator centered by current computing block, the corresponding vector of selection differential reckling is as the result of calculation of current block.Point to the band of position respectively, difference calculates in certain comparison window (Search window), and is retrained by comparison range (Search Area).A kind of way of routine is set to consistent with block size by the size of comparison window, as 8*8 pixel in example.Difference completes calculating (namely calculating in units of pixel) under pixel grid, the normal form adopting absolute difference and (SAD:sumof absolute difference) or absolute difference average (MAD:mean of absolute difference).
Upgrade vector shown with ASC by spatial candidate vector in random vector superpose and form.Wherein, because ASC table has sub-pixel precision (as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is pointed to non-integer center, interpolation algorithm need be adopted to obtain the value of non-integer coordinates position.
Each multilayer 3DRS block is estimated to include in iterative process: judge that whether the sensing band of position of each candidate vector of the current computing block that estimated value obtains is integer (the sensing band of position corresponding center position coordinates whether be integer), if so, the difference between comparison window that this candidate vector points to region centered by the comparison window in region centered by the band of position and current computing block is then directly calculated; If not, then go out by dull segmentation cube interpolation method interpolation the comparison window of regional center that this candidate vector points to, then centered by the comparison window of regional center pointed to of this candidate vector of obtaining of calculated difference and current computing block region comparison window between difference; Select with current computing block centered by the minimum candidate vector of the difference of comparison window in region as the difference estimation result of current computing block current iteration.
Estimation of Depth value acquisition module 432, is connected with iteration module 431, for determining that the pyramidal multilayer 3DRS block in block grid difference field estimates that iterative computation stops, obtaining each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Depth map generating unit 44, for each piece of difference estimation value generating depth map according to the bottom in the pyramid of block grid difference field.
Because each piece of difference estimation value of the bottom in the pyramid of the present embodiment gained block grid difference field is comparatively balanced, reliable generally, 3DRS algorithm after improving searching route, is effectively inhibit to estimate the generation of singular point in difference field.
This step has a lot of mode to realize, usually by block corrosion, the expansion of block grid difference field can be obtained pixel difference field pattern, again by pixel difference field pattern being carried out difference deep conversion in the mode such as to table look-up or linear, obtain depth map in the interval that pixel difference field pattern is quantized to 0 to 255 (unit 1) the most at last, after generating depth map, just 3D two field picture can be converted to 2D+Z (degree of depth) form that can be used for various visual angles (Multi-view) 3D rendering and generate.
For estimation of Depth, more compact, scope is less difference field is conducive to less error in final 0 ~ 255 scope quantification.The different picture of two width is adopted to carry out contrast experiment, shown by experimental data: the four-way searching route multilayer 3DRS in the present embodiment and searching route multilayer 3DRS in the same way line by line, or first searching route in embodiment compare, minimum (the first width figure of disparity range, search for line by line: 72.25 pixels, interlaced alternative direction is searched for: 76 pixels, improves searching route: 63.5 pixels; Second width figure, searches for: 89.5 pixels line by line, and interlaced alternative direction is searched for: 100 pixels, improves searching route: 84.5 pixels), and in the present embodiment, effectively inhibit 3DRS algorithm to estimate the generation of singular point in difference field.
In sum, a kind of difference estimation method based on multilayer 3DRS of the present invention and device, by adopting the searching route that adjacent rows/row searching route is contrary, and make each iteration compared to last iteration by image and searching route 90-degree rotation, Iterative path is made to become four-way iteration, make the difference estimation result of 3D two field picture more balanced, reliable, the uniformity that further reinforcement multilayer 3DRS algorithm is restrained in different directions strengthens the uniformity that multilayer 3DRS algorithm is restrained in different directions further, to obtain more accurate, uniform depth estimation result.
Above-described embodiment is illustrative principle of the present invention and effect thereof only, but not for limiting the present invention.Any person skilled in the art scholar all without prejudice under spirit of the present invention and category, can modify above-described embodiment or changes.Therefore, such as have in art usually know the knowledgeable do not depart from complete under disclosed spirit and technological thought all equivalence modify or change, must be contained by claim of the present invention.
Claims (12)
1., based on a difference estimation method of multilayer 3DRS, it is characterized in that, comprising:
Build the image pyramid of the default number of plies according to the left figure of 3D two field picture and right figure respectively, generate the block grid difference field pyramid presetting block size according to described image pyramid;
Arrange the searching route of default iterations, wherein the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column;
Successively use described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
2. the method for claim 1, is characterized in that, described in obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field after, also comprise, according to each piece of difference estimation value generating depth map of the bottom in described piece of grid difference field pyramid.
3. the method for claim 1, is characterized in that, the described multilayer 3DRS block to the pyramidal default iterations in each layer block grid difference field estimates iterative computation, comprising:
After the iterative computation of multilayer 3DRS block estimation is each time completed to each layer block grid difference field pyramid, judge whether described time iterations of working as reaches default iterations, if so, then stops this layer of block grid difference field pyramid iterative computation; If not, then adopt described searching route to estimate iterative computation to multilayer 3DRS block next time, then repeat current iteration process and calculate as next iteration, until reach default iterations.
4. the method for claim 1, it is characterized in that, the described searching route arranging default iterations, comprising: the searching route arranging default iterations, each iteration compared to last iteration by image relevant relative to current iteration for searching route and each difference field pattern 90-degree rotation.
5. the method for claim 1, is characterized in that, described estimator is the block position of the iteration spatial prediction block when secondary non-iterative computation in the tilting Y estimator mated with searching route being placed on tilting Y estimator center line symmetry with this.
6. the method for claim 1, is characterized in that, the described left figure according to 3D two field picture and right figure builds the pyramid of the default number of plies respectively, comprising:
Judge the left figure of 3D two field picture and the length of right figure and wide whether respectively can by 2 of the block length of side
ndoubly divide exactly, n, for presetting the number of plies, if so, then builds the multilayer 3DRS pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure; If not, then nearest interpolation is adopted in the surrounding benefit value of the left figure of 3D two field picture and right figure to can by 2 of the block length of side
nthe size doubly divided exactly, then the pyramid building the default number of plies according to the left figure of the 3D two field picture after benefit value and right figure.
7., based on a difference estimation device of multilayer 3DRS, it is characterized in that, comprising:
Pyramid generation unit, for building the image pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure, generates the block grid difference field pyramid presetting block size according to described image pyramid;
Searching route setting unit, for arranging the searching route of default iterations, wherein the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column;
Iterative computation unit, be connected with searching route setting unit with pyramid generation unit, for successively using described searching route to estimate iterative computation to the multilayer 3DRS block that each layer block grid difference field pyramid completes default iterations according to the estimator preset, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
8. device as claimed in claim 7, is characterized in that, also comprise depth map generating unit, for each piece of difference estimation value generating depth map according to the bottom in described piece of grid difference field pyramid.
9. device as claimed in claim 7, it is characterized in that, described iterative computation unit, comprising:
Iteration module, estimate iterative computation for successively using the multilayer 3DRS block of described searching route to the default iterations that each layer block grid difference field gold word carries out according to the estimator preset:
After the iterative computation of multilayer 3DRS block estimation is each time completed to each layer block grid difference field pyramid, judge whether described time iterations of working as reaches default iterations, if so, then stops this layer of block grid difference field pyramid iterative computation; If not, then adopt described searching route to estimate iterative computation to multilayer 3DRS block next time, then repeat current iteration process and calculate as next iteration, until reach default iterations.
Estimation of Depth value acquisition module, is connected with iteration module, for determining that the pyramidal multilayer 3DRS block in block grid difference field estimates that iterative computation stops, obtaining each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
10. device as claimed in claim 7, it is characterized in that, described searching route setting unit, specifically for: the searching route that default iterations is set, each iteration compared to last iteration by image relevant relative to current iteration for searching route and each difference field pattern 90-degree rotation, the searching route of each iteration is by row/column search, searches the other end, the searching route that adjacent rows/row searching route is contrary from one end of every row/column.
11. devices as claimed in claim 7, it is characterized in that, described pyramid generation unit, comprising:
Build image pyramid module, for the left figure and right figure that judge 3D two field picture length and wide whether respectively can by 2 of the block length of side
ndoubly divide exactly, n, for presetting the number of plies, if so, then builds the multilayer 3DRS pyramid of the default number of plies respectively according to the left figure of 3D two field picture and right figure; If not, then nearest interpolation is adopted in the surrounding benefit value of the left figure of 3D two field picture and right figure to can by 2 of the block length of side
nthe size doubly divided exactly, then the pyramid building the default number of plies according to the left figure of the 3D two field picture after benefit value and right figure;
Generate difference field pyramid module, for generating the block grid difference field pyramid presetting block size according to described image pyramid.
12. devices as claimed in claim 7, is characterized in that, described estimator is the block position of the iteration spatial prediction block when secondary non-iterative computation in the tilting Y estimator mated with searching route being placed on tilting Y estimator center line symmetry with this.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510514518.1A CN105007495B (en) | 2015-08-20 | 2015-08-20 | A kind of difference estimation method and device based on multilayer 3DRS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510514518.1A CN105007495B (en) | 2015-08-20 | 2015-08-20 | A kind of difference estimation method and device based on multilayer 3DRS |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105007495A true CN105007495A (en) | 2015-10-28 |
| CN105007495B CN105007495B (en) | 2018-07-20 |
Family
ID=54379983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510514518.1A Active CN105007495B (en) | 2015-08-20 | 2015-08-20 | A kind of difference estimation method and device based on multilayer 3DRS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105007495B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105809717A (en) * | 2016-03-10 | 2016-07-27 | 上海玮舟微电子科技有限公司 | Depth estimation method, system and electronic equipment |
| CN105952233A (en) * | 2016-05-10 | 2016-09-21 | 中国电力科学研究院 | Transmission tower adjacent-tower body mouth width determination method and device |
| CN106993108A (en) * | 2017-04-07 | 2017-07-28 | 上海顺久电子科技有限公司 | A kind of method and apparatus of random quantity of the determination video image in estimation |
| WO2021003807A1 (en) * | 2019-07-10 | 2021-01-14 | 浙江商汤科技开发有限公司 | Image depth estimation method and device, electronic apparatus, and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090304293A1 (en) * | 2008-06-08 | 2009-12-10 | Te-Hao Chang | Motion estimation method and related apparatus for efficiently selecting motion vector |
| WO2011001364A1 (en) * | 2009-06-29 | 2011-01-06 | Nxp B.V. | Parallel three-dimensional recursive search (3drs) meandering algorithm |
| CN102447892A (en) * | 2010-09-30 | 2012-05-09 | 宝利微电子系统控股公司 | Motion estimation method and device |
| CN102881014A (en) * | 2012-09-07 | 2013-01-16 | 航天恒星科技有限公司 | Quick stereo matching method based on graph cut |
| CN103679680A (en) * | 2012-08-31 | 2014-03-26 | 富士通株式会社 | Stereo matching method and system |
| US20140285623A1 (en) * | 2011-10-10 | 2014-09-25 | Koninklijke Philips N.V. | Depth map processing |
-
2015
- 2015-08-20 CN CN201510514518.1A patent/CN105007495B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090304293A1 (en) * | 2008-06-08 | 2009-12-10 | Te-Hao Chang | Motion estimation method and related apparatus for efficiently selecting motion vector |
| WO2011001364A1 (en) * | 2009-06-29 | 2011-01-06 | Nxp B.V. | Parallel three-dimensional recursive search (3drs) meandering algorithm |
| CN102447892A (en) * | 2010-09-30 | 2012-05-09 | 宝利微电子系统控股公司 | Motion estimation method and device |
| US20140285623A1 (en) * | 2011-10-10 | 2014-09-25 | Koninklijke Philips N.V. | Depth map processing |
| CN103679680A (en) * | 2012-08-31 | 2014-03-26 | 富士通株式会社 | Stereo matching method and system |
| CN102881014A (en) * | 2012-09-07 | 2013-01-16 | 航天恒星科技有限公司 | Quick stereo matching method based on graph cut |
Non-Patent Citations (3)
| Title |
|---|
| ADERENNE HEINRICH ET.AL.: "Optimization of Hierarchical 3DRS Motion Estimators for Picture Rate Conversion", 《IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING》 * |
| 王建国，周源华: "应用于格式转换的三维递归搜索块匹配算法", 《计算机工程与应用》 * |
| 贾茜等: "基于三维递归搜索的多级运动估计视频帧率上转换算法", 《电子与信息学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105809717A (en) * | 2016-03-10 | 2016-07-27 | 上海玮舟微电子科技有限公司 | Depth estimation method, system and electronic equipment |
| CN105809717B (en) * | 2016-03-10 | 2019-01-11 | 上海玮舟微电子科技有限公司 | A kind of depth estimation method, system and electronic equipment |
| CN105952233A (en) * | 2016-05-10 | 2016-09-21 | 中国电力科学研究院 | Transmission tower adjacent-tower body mouth width determination method and device |
| CN106993108A (en) * | 2017-04-07 | 2017-07-28 | 上海顺久电子科技有限公司 | A kind of method and apparatus of random quantity of the determination video image in estimation |
| WO2021003807A1 (en) * | 2019-07-10 | 2021-01-14 | 浙江商汤科技开发有限公司 | Image depth estimation method and device, electronic apparatus, and storage medium |
| JP2022515517A (en) * | 2019-07-10 | 2022-02-18 | 浙江商▲湯▼科技▲開▼▲発▼有限公司 | Image depth estimation methods and devices, electronic devices, and storage media |
| JP7116262B2 (en) | 2019-07-10 | 2022-08-09 | 浙江商▲湯▼科技▲開▼▲発▼有限公司 | Image depth estimation method and apparatus, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105007495B (en) | 2018-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102771110B (en) | The Object tracking of the vector using graphics engine to obtain in movement estimation system | |
| CN103858148B (en) | Planar Mapping and tracking, device and equipment for mobile device | |
| CN107135397B (en) | A kind of panorama video code method and apparatus | |
| CN105007495A (en) | Difference estimation method based on multiple layers of 3DRS and device thereof | |
| CN106920259A (en) | A kind of localization method and system | |
| CN102710934B (en) | Motion predicting or compensating method | |
| CN106257497B (en) | Matching method and device for image homonymy points | |
| CN104517317A (en) | Three-dimensional reconstruction method of vehicle-borne infrared images | |
| CN104754338A (en) | Selection method and device for intra-frame predication mode | |
| CN109191511A (en) | A kind of binocular solid matching process based on convolutional neural networks | |
| CN106845338A (en) | Pedestrian detection method and system in video flowing | |
| CN109801323A (en) | Pyramid binocular depth with self-promotion ability estimates model | |
| CN105957003B (en) | The super dimension method for reconstructing of porous media based on study | |
| JPH07170496A (en) | Method and apparatus for estimating movement based on block matching | |
| JP2011141710A (en) | Device, method and program for estimating depth | |
| JP2013073598A (en) | Image processing device, image processing method, and program | |
| CN103400393A (en) | Image matching method and system | |
| CN105809717B (en) | A kind of depth estimation method, system and electronic equipment | |
| US20130235274A1 (en) | Motion vector detection device, motion vector detection method, frame interpolation device, and frame interpolation method | |
| CN103020964A (en) | Multi-view stereo matching method based on self-adaptive watershed image segmentation | |
| CN104333758A (en) | Depth map prediction method, pixel detection method and related devices | |
| Kolesnikov | Efficient online algorithms for the polygonal approximation of trajectory data | |
| Luo et al. | Stereo matching and occlusion detection with integrity and illusion sensitivity | |
| CN102867327A (en) | Textile flexible movement reestablishing method based on neural network system | |
| Hou et al. | Depth estimation and object detection for monocular semantic SLAM using deep convolutional network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200331 Address after: 215634 north side of Chengang road and west side of Ganghua Road, Jiangsu environmental protection new material industrial park, Zhangjiagang City, Suzhou City, Jiangsu Province Patentee after: ZHANGJIAGANG KANGDE XIN OPTRONICS MATERIAL Co.,Ltd. Address before: 201203, room 5, building 690, No. 202 blue wave road, Zhangjiang hi tech park, Shanghai, Pudong New Area Patentee before: WZ TECHNOLOGY Inc. |
|
| TR01 | Transfer of patent right |