CN105007495B - A kind of difference estimation method and device based on multilayer 3DRS - Google Patents

A kind of difference estimation method and device based on multilayer 3DRS Download PDF

Info

Publication number
CN105007495B
CN105007495B CN201510514518.1A CN201510514518A CN105007495B CN 105007495 B CN105007495 B CN 105007495B CN 201510514518 A CN201510514518 A CN 201510514518A CN 105007495 B CN105007495 B CN 105007495B
Authority
CN
China
Prior art keywords
pyramid
block
iteration
difference
3drs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510514518.1A
Other languages
Chinese (zh)
Other versions
CN105007495A (en
Inventor
于炀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhangjiagang Kangdexin Optronics Material Co Ltd
Original Assignee
SHANGHAI WEI ZHOU MICROELECTRONICS TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SHANGHAI WEI ZHOU MICROELECTRONICS TECHNOLOGY Co Ltd filed Critical SHANGHAI WEI ZHOU MICROELECTRONICS TECHNOLOGY Co Ltd
Priority to CN201510514518.1A priority Critical patent/CN105007495B/en
Publication of CN105007495A publication Critical patent/CN105007495A/en
Application granted granted Critical
Publication of CN105007495B publication Critical patent/CN105007495B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)

Abstract

The present invention provides a kind of difference estimation method and devices based on multilayer 3DRS.A kind of difference estimation method based on multilayer 3DRS includes：The image pyramid for building the default number of plies respectively according to the left figure of 3D frame images and right figure generates the block grid difference field pyramid for presetting block size according to described image pyramid；The searching route of default iterations is set, and search is to be searched for by row/column in each iteration, and the other end, the adjacent rows/opposite searching route of row searching route are searched from one end of every row/column；The multilayer 3DRS blocks for completing to preset iterations to each layer block grid difference field pyramid estimate iterative calculation, obtain each piece of difference estimation value of the bottom in the pyramid of block grid difference field.The present invention uses adjacent rows/opposite searching route of row searching route, keeps the difference estimation result of 3D frame images more balanced, reliable, to obtain more accurate, uniform depth estimation result.

Description

A kind of difference estimation method and device based on multilayer 3DRS
Technical field
The present invention relates to the process field of 3D rendering more particularly to a kind of difference estimation method and devices based on 3DRS.
Background technology
3DRS (3-Dimension Recursive Search, multi-layer three-dimension recursive search) algorithm is a kind of estimation Algorithm, by Gerard de Haan 1993 in《True-Motion Estimation with 3-D Recursive Search Block Matching》It is proposed in one text.3DRS algorithms in original text are first that unit calculates figure with block (block) Then picture and the difference between reference picture corrode (Block Erosion) method by block grid (block grid) by block Under sports ground (Motion Field) or difference field (Displacement Field) be extended to pixel grid (Pixel Grid the final result under).The recurrence of initial 3DRS meanings is initially to have no the way of iteration, convergence side on image space It is determined to by its estimator (Estimator) and the direction of search.
Improvement after 3DRS is proposed remains its basic thought.By improving the ACS (asynchronous in former algorithm Cyclic search) table《Sub-pixel motion estimation with 3D Recursive block matching》, and pyramid expression (Pyramidal Representation) combination《Optimization of Hierarchical 3DRS Motion Estimators for Picture Rate Conversion》The methods of, it forms Multilayer 3DRS algorithms (Hierarchical 3DRS) with sub-pixel precision.
3D automatic conversions refer to traditional 3D or so bitmap-format, are converted into can be used for multi-angle (Multi-view) 3D rendering 2D+Z (depth) format of generation.Currently, multilayer 3DRS algorithms are widely used in 3D videos in the estimation of Depth of 3D frames image, The depth estimation algorithm of multilayer 3DRS is using pixel-recursive algorithm or 3DRS as core.However, 3DRS algorithms and multilayer 3DRS algorithms The mode for being all made of relatively simple searching route is realized, line by line such as left-to-right, by the upper left corner to the lower right corner.Repeatedly at every layer Under the premise of iteration, simple searching route fails to make full use of the advantage of successive ignition so that the difference obtained after search iteration There may be non-uniform situations for different estimation, to influence the difference estimation result that 3D frames pixel is final in 3D videos.How The problem of keeping difference estimation result more balanced, reliable, being industry urgent need to resolve.
Invention content
The present invention provides a kind of difference estimation method and devices based on multilayer 3DRS, for solving how to make 3D frame figures More balanced, the reliable problem of the difference estimation result of picture.
The embodiment of the present invention uses following technical scheme：
On the one hand, the present invention provides a kind of difference estimation method based on multilayer 3DRS, this method includes：
The image pyramid for building the default number of plies respectively according to the left figure of 3D frame images and right figure, according to described image gold word Tower generates the block grid difference field pyramid for presetting block size；
The searching route of default iterations is set, wherein the searching route of each iteration is to be searched for by row/column, from every One end of row/column searches the other end, the adjacent rows/opposite searching route of row searching route；
Described search path is successively used to complete each layer block grid difference field pyramid according to preset estimator default The multilayer 3DRS blocks estimation iterative calculation of iterations, obtains each piece of difference estimation of the bottom in the pyramid of block grid difference field Value.
On the other hand, the present invention also provides a kind of difference estimation device based on multilayer 3DRS, which includes：
Pyramid generation unit, the image gold word for building the default number of plies respectively according to the left figure and right figure of 3D frame images Tower generates the block grid difference field pyramid for presetting block size according to described image pyramid；
Searching route setting unit, the searching route for presetting iterations, wherein the searching route of each iteration is equal To be searched for by row/column, the other end, the adjacent rows/opposite searching route of row searching route are searched from one end of every row/column；
Unit is iterated to calculate, is connected with pyramid generation unit and searching route setting unit, for being estimated according to preset Gauge successively uses described search path to complete to preset the multilayer 3DRS blocks of iterations to each layer block grid difference field pyramid Estimation iterative calculation, obtains each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Compared with prior art, a kind of difference estimation method and device based on multilayer 3DRS provided by the invention, has Following advantageous effect：
The present invention is by using adjacent rows/opposite searching route of row searching route, the present embodiment gained block map declination In different field pyramid each piece of difference estimation value of the bottom generally more uniformly, effectively prevent since searching route is single and Caused by " abnormal point " generation；Using making each iteration be rotated by 90 ° image and searching route compared to last iteration, So that Iterative path is become four-way iteration, Iterative path is made to become the four-way iteration that each iteration is rotated by 90 °, and using therewith The tilting Y estimators matched keep the difference estimation result of 3D frame images more balanced, reliable, further strengthen multilayer 3DRS algorithms Uniformity convergent in different directions, to obtain more accurate, uniform depth estimation result, the present invention effectively inhibits 3DRS algorithms estimate the generation of singular point in difference field.It is supplemented using pixel value difference method when building image pyramid, image is made to believe Breath preservation is more complete, and pyramidal generation is more accurate.
Description of the drawings
Fig. 1 is a kind of method stream of one embodiment of difference estimation method based on multilayer 3DRS provided by the invention Cheng Tu.
Fig. 2 is a kind of method stream of second embodiment of difference estimation method based on multilayer 3DRS provided by the invention Cheng Tu.
Fig. 3 is a kind of structural frames of one embodiment of difference estimation device based on multilayer 3DRS provided by the invention Figure.
Fig. 4 is a kind of structural frames of second embodiment of difference estimation device based on multilayer 3DRS provided by the invention Figure.
Fig. 5 a are a kind of searching route of difference estimation method and device based on multilayer 3DRS provided by the invention, estimation The schematic diagram one of device.
Fig. 5 b are a kind of searching route of difference estimation method and device based on multilayer 3DRS provided by the invention, estimation The schematic diagram two of device.
Fig. 5 c are a kind of searching route of difference estimation method and device based on multilayer 3DRS provided by the invention, estimation The schematic diagram three of device.
Fig. 5 d are a kind of searching route of difference estimation method and device based on multilayer 3DRS provided by the invention, estimation The schematic diagram four of device.
Specific implementation mode
For make present invention solves the technical problem that, the technical solution that uses and the technique effect that reaches it is clearer, below The technical solution of the embodiment of the present invention will be described in further detail in conjunction with attached drawing, it is clear that described embodiment is only It is a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those skilled in the art exist The every other embodiment obtained under the premise of creative work is not made, shall fall within the protection scope of the present invention.
Fig. 1 is shown according to a kind of one embodiment of the difference estimation method based on multilayer 3DRS provided by the invention Method flow diagram.The a kind of of the present embodiment is mainly executed by intelligent terminal based on the estimation of Depth side of multilayer 3DRS.Wherein, intelligence Can terminal include but not limited to：PC, laptop, mobile phone, tablet computer, video playback apparatus etc., intelligent terminal It can be the intelligent terminal that can play or parse 3D videos.This approach includes the following steps：
11：The image pyramid for building the default number of plies respectively according to the left figure of 3D frame images and right figure, according to described image Pyramid generates the block grid difference field pyramid for presetting block size.
Specifically, building the figure of the default number of plies respectively using average weighted method according to the left figure of 3D frame images and right figure As pyramid, described image pyramid is initialized as to the block grid difference field pyramid of default block size.In image pyramid Including left figure pyramid and right figure pyramid.The left figure of 3D frame images builds the left figure pyramid of the default number of plies, 3D frame images Right figure also builds the right figure pyramid of the default number of plies, wherein the high pyramidal length and width of a tomographic image is bottom one tomographic image gold respectively 1/2 times of the length and width of word tower.Last layer is based on its next layer generation.For any pixel point (x, y) in last layer, value Mutually multiply accumulating to obtain with weighted average filtering core by the regions M*N in layer 1 centered on pixel (2x, 2y), in practice for It is smooth to consider, the region of bigger can be used.The block grid difference field pyramid for presetting block size is generated according to image pyramid, it is poor The different pyramidal generation method in field is that identical layer image pixel coordinates in right figure pyramid are identical compared in left layer pyramid The phasor difference of the corresponding coordinate of tomographic image pixel constitutes difference field pyramid.The generally square block of pixels of block size, such as 8* 8, then difference field pyramid is obtained into block map declination by being divided into the block of several 8*8 for the block grid of 8*8 with grid block size Different field pyramid.
In the present embodiment, when generating every layer of image pyramid, this first can be generated according to the acquisition of the size of 3D frame images 3D frame image sectional drawings are the size that can be divided exactly by the block length of side, i.e., if cannot divide exactly by the size divided by the block length of side of layer The insufficient block length of side pixel row or column is had lost, the block grid difference field pyramid for presetting block size is generated.
Such as：For the 3D frame images of 1080*960 (long * wide) pixel, pyramid is 3 layers, will be generated from bottom to up successively For：The image pyramid of 1080*960,540*480,270*240 pixel；It can directly judge that the picture size of 3D frame images is Block size uses 8*8 pixels, and each layer block grid difference field pyramid size is respectively 135*120 blocks, 67*60 blocks and 33*30 blocks. Difference field pyramid has abandoned the part less than 8 pixels when generating, and such as a length of 540 pixels/8=67.5 of the second layer, loses not The part of 8 pixels of foot, becomes 67.
12：The searching route of default iterations is set, wherein the searching route of each iteration is to be searched for by row/column, The other end, the adjacent rows/opposite searching route of row searching route are searched from one end of every row/column.
Specifically, if setting is searched for line by line, the other end, adjacent rows searching route phase will be searched from one end of every row Instead, even a behavior is searched for from left to right, then next behavior is searched for from right to left；If setting is searched for by column, from one end of each column The other end is searched, adjacent two row searching route is on the contrary, even one is classified as from top to bottom, then next to be classified as from bottom to up.
By taking the searching route searched for line by line as an example, if iterations are 4, if setting is searched for line by line, from top to bottom search line by line Rope, to search for from left to right, even number line search is to search for from right to left for odd-numbered line search.
Further, adjacent searching route twice is on the contrary, and the last terminal searched for is the starting point searched for next time Searching route, example as above, if the search starting point of odd-times iteration is the upper left corner, the search terminal of iteration is the lower right corner, even-times The search starting point of iteration is the lower right corner, the search terminal upper left corner of iteration.
13：Described search path is successively used to complete each layer block grid difference field pyramid according to preset estimator pre- If the multilayer 3DRS blocks estimation iterative calculation of iterations, each piece of difference for obtaining the bottom in the pyramid of block grid difference field are estimated Evaluation.
Specifically, successively using described search path to each layer block map declination since top according to preset estimator Different field pyramid is completed to preset the multilayer 3DRS blocks estimation iterative calculation of iterations, obtains in the pyramid of block grid difference field most Each piece of difference estimation value of bottom.
Wherein, multilayer 3DRS blocks estimation iterative calculation is each of the acquisition difference field pyramid bottom in multilayer 3DRS algorithms The block estimation iterative calculation step of block difference estimation value, but the step change in multilayer 3DRS algorithms the structure of estimator and Searching route.
As shown in attached drawing 5a and 5c, estimator is will be with matched tilting Y estimators (the Oblique Y of searching route Estimator the spatial prediction block that this in) does not iterate to calculate is placed on and symmetrical iteration in the tilting Y estimators Block position.
Based on tilting Y estimators, the design of estimated below device has been done.For the searching route searched for line by line is set, For odd-times iteration (Fig. 5 a), when searching for from left to right, estimator is spatial prediction block (X-1, Y), (X, Y-1), and layering is pre- Survey block (X, Y), time prediction block (X+2, Y+2) structure；When searching for from right to left, estimator structure be spatial prediction block (X-1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2).For even-times iteration (Fig. 5 c), from right to left When search, estimator is spatial prediction block (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2) Structure；When searching for from left to right, estimator structure be spatial prediction block (X+1, Y), (X, Y-1), hierarchical prediction block (X, Y), when Between prediction block (X+2, Y+2).
It should be noted that in the interlaced alternative direction of search, according to tilting Y estimators, since block (X+1, Y) is (strange Secondary iteration), (X-1, Y) (even iteration) can not be used as spatial prediction block (not yet calculate and arrive), therefore, selected (X+1, Y) (strange Secondary iteration), (X+1, Y) (even iteration) replaces.
According to each candidate vector for the current calculation block that estimated value obtains, candidate vector includes：Spatial candidate vector (Spatial Candidate), time candidate vector (Temporal Candidate), layering candidate vector (Hierarchy Candidate), renewal vector (Update Candidate).Wherein, estimated by the difference of last layer corresponding blocks by spatial prediction block Evaluation is multiplied by and 2 obtains spatial candidate vectorial (Spatial Candidate), by time prediction block by previous frame or last time iteration Difference estimation result obtain time candidate vector (Temporal Candidate), complete by current iteration by hierarchical prediction block Blocking difference estimation is worth to layering candidate vector (Hierarchy Candidate), renewal vector (Update Candidate) random vector is generated with ASC tables be superimposed generation by spatial candidate vector.
Compare by region centered on each candidate vector direction band of position in the estimator centered on current calculation block Compare the difference (Distortion) between the comparison window in region centered on window and current calculation block, selection differential reckling institute is right Answer result of calculation of the vector as current block.It is respectively directed to the band of position, difference is centainly comparing in window (Search window) It calculates, and is constrained by comparing range (Search Area).A kind of conventional way is will to compare being dimensioned to for window It is consistent with block size, such as 8*8 pixels in example.Difference is completed to calculate (i.e. to calculate as unit of pixel) under pixel grid, Chang Cai With absolute difference and (SAD：Sum of absolute difference) or absolute difference be averaged (MAD：mean of absolute Difference form).
Renewal vector is superimposed with random vector in ASC tables by spatial candidate vector and is formed.Wherein, since ASC tables have Asia Pixel precision (such as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is directed toward with non-integer center When position, need to the value of non-integer coordinates position be obtained using interpolation algorithm.
Include in each multilayer 3DRS blocks estimation iterative process：Judge the every of the current calculation block that estimated value obtains Whether the direction band of position of a candidate vector is integer (being directed toward whether the corresponding center position coordinates in the band of position are integer), If so, directly calculating the comparison window in region and region centered on current calculation block centered on the candidate vector direction band of position Comparison window between difference；If it is not, then passing through the dull area for being segmented cube interpolation method interpolation and going out candidate vector direction The comparison window at domain center, then the comparison window of regional center that the obtained candidate vector of calculating difference is directed toward is with current calculation block Difference between the comparison window of central area；The candidate of selection and the difference minimum of the comparison window in region centered on current calculation block Difference estimation result of the vector as current calculation block current iteration.
For example, when calculation block grid (10,10) position, if spatial candidate vector is (1,1), ASC vectors be (0.25, 0) it, then will be directed toward the band of position that center is (11.25,11), the area for the position that needs interpolation to go out centered on (11.25,11) The value of 8*8 points in domain.
The method of the present embodiment, compared with the searching route searched in the same direction line by line, block grid difference field obtained by the present embodiment Each piece of difference estimation value of the bottom generally more uniformly, effectively prevents causing since searching route is single in pyramid " abnormal point " generation.
Fig. 2 shows according to a kind of side of second embodiment of difference estimation method based on multilayer 3DRS provided by the invention Method flow chart.A kind of difference estimation method based on multilayer 3DRS of the present embodiment is mainly executed by intelligent terminal.Wherein, intelligence Can terminal include but not limited to：PC, laptop, mobile phone, tablet computer, video playback apparatus etc., intelligent terminal It can be the intelligent terminal that can play or parse 3D videos.This approach includes the following steps：
21：The image pyramid for building the default number of plies respectively according to the left figure of 3D frame images and right figure, according to described image Pyramid generates the block grid difference field pyramid for presetting block size.
Specifically, building the figure of the default number of plies respectively using average weighted method according to the left figure of 3D frame images and right figure As pyramid, described image pyramid is initialized as to the block grid difference field pyramid of default block size.In image pyramid Including left figure pyramid and right figure pyramid.The left figure of 3D frame images builds the left figure pyramid of the default number of plies, 3D frame images Right figure also builds the right figure pyramid of the default number of plies, wherein the high pyramidal length and width of a tomographic image is bottom one tomographic image gold respectively 1/2 times of the length and width of word tower.The block grid difference field pyramid for presetting block size, difference field gold are generated according to image pyramid The generation method of word tower is by identical layer image pixel coordinates in right figure pyramid compared to identical tomographic image in left layer pyramid The phasor difference of the corresponding coordinate of pixel constitutes difference field pyramid.The generally square block of pixels of block size, such as 8*8 then will Difference field pyramid obtains block grid difference field gold by being divided into the block of several 8*8 for the block grid of 8*8 with grid block size Word tower.
In the present embodiment, this step 21 includes step 211 and step 212.
211：Judge the left figure of 3D frame images and the length of right figure and it is wide whether respectively can be by the 2 of the block length of sidenDivide exactly again, n is The default number of plies, if so, the multilayer 3DRS pyramids of the default number of plies are built respectively according to the left figure of 3D frame images and right figure, if not It is then to be worth to can be by the 2 of the block length of side in the left figure of 3D frame images and the surrounding benefit of right figure using nearest interpolationnWhat is divided exactly again is big Small, left figure and right figure further according to the 3D frame images after benefit value build the pyramid of the default number of plies.
For example, being preset as 3 layers, the size of block is 8*8 pixels, then the length and width of the left figure and right figure that judge 3D frame images are No is divided exactly by 64, if it is not, making the left figure of 3D frame images and the surrounding benefit of right figure be worth to energy quilt using nearest interpolation 64 sizes divided exactly.
212：The multilayer pyramid is initialized as to the block grid difference field pyramid of default block size.
22：The searching route of default iterations is set, and each iteration is opposite by searching route compared to last iteration It being rotated by 90 ° in the relevant image of current iteration and each difference field figure, the searching route of each iteration is to be searched for by row/column, from One end per row/column searches the other end, the adjacent rows/opposite searching route of row searching route.
Specifically, every time iteration compared to last iteration by searching route relative to the relevant image of current iteration and each Difference field figure is rotated by 90 °, and can be to rotate clockwise or rotate counterclockwise, wherein can be related by rotating current iteration Image and each difference field realize, for rotating counterclockwise, including：When each iteration, relevant image and each difference field After figure is rotated by 90 ° counterclockwise, left-right reversed is carried out to each vector in each difference field after overturning, and to each vector x after exchange Durection component negates processing and realizes.Specific implementation is shown in two kinds of realization methods in step 23.
In each iteration, if setting is searched for line by line, the other end will be searched from one end of every row, adjacent rows search for road Diameter on the contrary, an even behavior is searched for from left to right, then search for from right to left by next behavior；If setting is searched for by column, from each column One end searches the other end, and adjacent two row searching route is on the contrary, even one is classified as from top to bottom, then next to be classified as from bottom to up.
23：Described search path is successively used to complete each layer block grid difference field pyramid according to preset estimator pre- If the multilayer 3DRS blocks estimation iterative calculation of iterations, each piece of difference for obtaining the bottom in the pyramid of block grid difference field are estimated Evaluation.
Specifically, successively using described search path to each layer block map declination since top according to preset estimator Different field pyramid is completed to preset the multilayer 3DRS blocks estimation iterative calculation of iterations, obtains in the pyramid of block grid difference field most Each piece of difference estimation value of bottom.
Wherein, multilayer 3DRS blocks estimation iterative calculation is each of the acquisition difference field pyramid bottom in multilayer 3DRS algorithms The block estimation iterative calculation step of block difference estimation value, but the step change in multilayer 3DRS algorithms the structure of estimator and Searching route.
Further, the multilayer 3DRS blocks for completing to preset iterations to each layer block grid difference field pyramid estimate iteration It calculates, including：
After completing the iterative calculation of multilayer 3DRS blocks estimation each time to each layer block grid difference field pyramid, work as described in judgement Whether secondary iterations reach default iterations, if so, stopping to this layer of block grid difference field pyramid iterative calculation；If It is no, then it uses described search path to estimate to iterate to calculate to multilayer 3DRS blocks next time, repeats under the conduct of current iteration process An iteration calculates, until reaching default iterations.
Step 23 uses 22 preset searching route of above-mentioned steps, and two ways may be used to realize：
Example one, estimator are to be placed on the spatial prediction block that this in tilting Y estimators does not iterate to calculate and the tilting Y The block position of symmetrical iteration in estimator, then estimator need to be matched with searching route.It is to search for line by line with searching route For, as shown in attached drawing 5a, 5b, 5c and 5d, for example, using searching for line by line, if default iterations are 4, first time iteration is 5a, to have done the design of estimated below device with the matched tilting Y estimators of searching route.By taking 4 iteration as an example, the 1st iteration (figure 5a), estimator design is identical as the estimator of example odd-times iteration in the step 23 of first embodiment.When the 2nd iteration (Fig. 5 b) is searched for from top to bottom, and from the upper left corner to the lower right corner, being equal in this step will change to second of multilayer 3DRS blocks estimation After the generation relevant image of calculating and each difference field figure are rotated by 90 ° counterclockwise, each vector in each difference field after overturning is carried out left Right exchange negates processing to each vector x durection component after exchange, repeats current iteration process as second of iteration meter It calculates, and such realization method that need not rotate image and each difference field figure, it is more easy to by intuitivism apprehension, such realization method is estimated Gauge is that spatial prediction block is (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), the structure of time prediction block (X-2, Y+2). It can similarly obtain, attached drawing (Fig. 5 c), estimator design and first embodiment are seen in the iterative search path of third time in this realization method Step 23 in example even-times iteration estimator it is identical.It can similarly obtain, the iteration of third time is searched in this realization method Attached drawing (Fig. 5 d) is seen in rope path, and when up to lower search, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2).It is identical as in the search of interlaced alternative direction, for can not when estimator designs Block as spatial prediction block (not yet calculate) uses symmetry blocks to replace.
Example two is estimated using the search pattern and estimator of first time iteration in attached drawing 5a.Complete the 1st iteration Afterwards, second of multilayer 3DRS blocks estimation is iterated to calculate into relevant image and each difference field figure turns over turn 90 degrees counterclockwise.For turning over Each vector carries out left and right exchange in each difference field after turning, and negates processing to vector x durection component, i.e.,：
V (x, y)=(Vx(x,y),Vy(x,y))T
V ' (x, y)=(- Vy(y,x),Vx(y,x))T
Wherein, V (x, y) is first time iterative search result vector, and V ' (x, y) is vector after overturning negates.Repeat first Secondary iterative process is iterated calculating according to attached drawing 5a searching routes and estimator and changes as second as second of iteration Generation.Iterating to calculate relevant image and each difference field figure, which turns over and turn 90 degrees, similarly is estimated to third time multilayer 3DRS blocks according to above-mentioned, Each vector carries out left and right exchange, and negates processing to vector x durection component, repeats the 1st iterative process as second Iteration carries out third time iteration, similarly carries out being sequentially completed 4 iteration or higher number of iterations again.
Wherein, it is for obtaining that the above-mentioned 3DRS blocks of multilayer next time estimation, which iterates to calculate relevant image and each difference field figure, Next iteration calculate needed for spatial candidate vector, time candidate vector, layering candidate vector, renewal vector image and respectively Difference field figure, further include when time search iteration obtain as a result, i.e. when secondary iteration after this layer of block grid difference field pyramid in respectively Block difference estimation value.
According to each candidate vector for the current calculation block that estimated value obtains, candidate vector includes：Spatial candidate vector, when Between candidate vector, layering candidate vector, renewal vector.Wherein, the difference estimation value of last layer corresponding blocks is pressed by spatial prediction block 2 are multiplied by obtain spatial candidate vector, obtain the time by the difference estimation result of previous frame or last iteration by time prediction block Candidate vector, the difference estimation that block is completed by current iteration by hierarchical prediction block are worth to layering candidate vector, renewal vector Random vector is generated with ASC tables be superimposed generation by spatial candidate vector.
Compare by region centered on each candidate vector direction band of position in the estimator centered on current calculation block Compare the difference between the comparison window in region centered on window and current calculation block, vector corresponding to selection differential reckling is as working as Preceding piece of result of calculation.It is respectively directed to the band of position, difference calculates in certain relatively window, and is carried out about by comparing range Beam.A kind of conventional way be will compare window be dimensioned to it is consistent with block size, such as 8*8 pixels in example.Difference is in pixel It completes to calculate (calculating as unit of pixel) under grid, frequently with the average form of absolute difference and/or absolute difference.
Renewal vector is superimposed with random vector in ASC tables by spatial candidate vector and is formed.Wherein, since ASC tables have Asia Pixel precision (such as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is directed toward with non-integer center When position, need to the value of non-integer coordinates position be obtained using interpolation algorithm.
Include in each multilayer 3DRS blocks estimation iterative process：Judge the every of the current calculation block that estimated value obtains Whether the direction band of position of a candidate vector is integer (being directed toward whether the corresponding center position coordinates in the band of position are integer), If so, directly calculating the comparison window in region and region centered on current calculation block centered on the candidate vector direction band of position Comparison window between difference；If it is not, then passing through the dull area for being segmented cube interpolation method interpolation and going out candidate vector direction The comparison window at domain center, then the comparison window of regional center that the obtained candidate vector of calculating difference is directed toward is with current calculation block Difference between the comparison window of central area；The candidate of selection and the difference minimum of the comparison window in region centered on current calculation block Difference estimation result of the vector as current calculation block current iteration.
24：Depth map is generated according to each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
This step has many modes may be implemented, and can usually be corroded by block and extend block grid difference field to obtain pixel Difference field figure, then by the way that pixel difference field figure is carried out difference deep conversion in a manner of tabling look-up or is linear etc., finally by pixel difference Different field figure, which is quantized in the section of 0 to 255 (unit 1), obtains depth map, can convert 3D frame images after generating depth map 2D+Z (depth) format generated at can be used for various visual angles (Multi-view) 3D rendering.
For estimation of Depth, one more compact, that range is smaller difference field is conducive in final 0~255 range The quantization of smaller error.Contrast experiment has been carried out using two width difference pictures, has been shown by experimental data：Four-way in the present embodiment Searching route multilayer 3DRS is compared with the searching route in searching route multilayer 3DRS in the same direction line by line or first embodiment, difference Range is minimum, and (the first width figure, is searched for line by line：72.25 pixels, the search of interlaced alternative direction：76 pixels improve searching route： 63.5 pixels；Second width figure, is searched for line by line：89.5 pixels, the search of interlaced alternative direction：100 pixels improve searching route： 84.5 pixels), and the generation of singular point in 3DRS algorithms estimation difference field is effectively inhibited in the present embodiment.
Fig. 3 is shown according to a kind of knot of difference estimation device one embodiment based on multilayer 3DRS provided by the invention Structure block diagram.A kind of difference estimation device based on multilayer 3DRS of the present embodiment is mainly executed by intelligent terminal.Wherein, intelligence Terminal includes but not limited to：PC, laptop, mobile phone, tablet computer, video playback apparatus etc., intelligent terminal can The intelligent terminal of 3D videos can be played or parse by thinking.The device includes pyramid generation unit 31, searching route setting unit 32 and iterative calculation unit 33.
Pyramid generation unit 31, the image gold for building the default number of plies respectively according to the left figure and right figure of 3D frame images Word tower generates the block grid difference field pyramid for presetting block size according to described image pyramid.
Specifically, building the figure of the default number of plies respectively using average weighted method according to the left figure of 3D frame images and right figure As pyramid, described image pyramid is initialized as to the block grid difference field pyramid of default block size.In image pyramid Including left figure pyramid and right figure pyramid.The left figure of 3D frame images builds the left figure pyramid of the default number of plies, 3D frame images Right figure also builds the right figure pyramid of the default number of plies, wherein the high pyramidal length and width of a tomographic image is bottom one tomographic image gold respectively 1/2 times of the length and width of word tower.The block grid difference field pyramid for presetting block size, difference field gold are generated according to image pyramid The generation method of word tower is by identical layer image pixel coordinates in right figure pyramid compared to identical tomographic image in left layer pyramid The phasor difference of the corresponding coordinate of pixel constitutes difference field pyramid.The generally square block of pixels of block size, such as 8*8 then will Difference field pyramid obtains block grid difference field gold by being divided into the block of several 8*8 for the block grid of 8*8 with grid block size Word tower.
In the present embodiment, when generating every layer of image pyramid, this first can be generated according to the acquisition of the size of 3D frame images 3D frame image sectional drawings are the size that can be divided exactly by the block length of side, i.e., if cannot divide exactly by the size divided by the block length of side of layer The insufficient block length of side pixel row or column is had lost, the block grid difference field pyramid for presetting block size is generated.
Such as：For the 3D frame images of 1080*960 (long * wide) pixel, pyramid is 3 layers, will be generated from bottom to up successively For：The image pyramid of 1080*960,540*480,270*240 pixel；It can directly judge that the picture size of 3D frame images is When block size uses 8*8 pixels, each layer block grid difference field pyramid size is respectively 135*120 blocks, 67*60 blocks and 33*30 Block.Difference field pyramid has abandoned the part less than 8 pixels when generating, and such as a length of 540 pixels/8=67.5 of the second layer, loses Less than the part of 8 pixels, become 67.
Searching route setting unit 32, the searching route for default iterations to be arranged, wherein the search of each iteration Path is to be searched for by row/column, and the other end, the adjacent rows/opposite search of row searching route are searched from one end of every row/column Path.
Specifically, if setting is searched for line by line, the other end, adjacent rows searching route phase will be searched from one end of every row Instead, even a behavior is searched for from left to right, then next behavior is searched for from right to left；If setting is searched for by column, from one end of each column The other end is searched, adjacent two row searching route is on the contrary, even one is classified as from top to bottom, then next to be classified as from bottom to up.
By taking the searching route searched for line by line as an example, if iterations are 4, if setting is searched for line by line, from top to bottom search line by line Rope, to search for from left to right, even number line search is to search for from right to left for odd-numbered line search.
Further, adjacent searching route twice is on the contrary, and the last terminal searched for is the starting point searched for next time Searching route, example as above, if the search starting point of odd-times iteration is the upper left corner, the search terminal of iteration is the lower right corner, even-times The search starting point of iteration is the lower right corner, the search terminal upper left corner of iteration.
Unit 33 is iterated to calculate, is connected with pyramid generation unit and searching route setting unit, for according to preset Estimator successively uses described search path to complete to preset the multilayer 3DRS of iterations to each layer block grid difference field pyramid Block estimation iterative calculation, obtains each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Specifically, successively using described search path to each layer block map declination since top according to preset estimator Different field pyramid is completed to preset the multilayer 3DRS blocks estimation iterative calculation of iterations, obtains in the pyramid of block grid difference field most Each piece of difference estimation value of bottom.
Wherein, multilayer 3DRS blocks estimation iterative calculation is each of the acquisition difference field pyramid bottom in multilayer 3DRS algorithms The block estimation iterative calculation step of block difference estimation value, but the step change in multilayer 3DRS algorithms the structure of estimator and Searching route.
As shown in attached drawing 5a and 5c, estimator is will be with matched tilting Y estimators (the Oblique Y of searching route Estimator the spatial prediction block that this in) does not iterate to calculate is placed on and symmetrical iteration in the tilting Y estimators Block position.
Based on tilting Y estimators, the design of estimated below device has been done.For the searching route searched for line by line is set, For odd-times iteration (Fig. 5 a), when left-to-right is searched for, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction Block (X, Y), time prediction block (X+2, Y+2) structure；Right-to-left search for when, estimator structure be spatial prediction block (X-1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2).For even-times iteration (Fig. 5 c), right-to-left search When, estimator is spatial prediction block (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), time prediction block (X-2, Y-2) structure； When left-to-right is searched for, estimator structure is spatial prediction block (X+1, Y), (X, Y-1), hierarchical prediction block (X, Y), time prediction block (X+2, Y+2).
It should be noted that in the interlaced alternative direction of search, according to tilting Y estimators, since block (X+1, Y) is (strange Secondary iteration), (X-1, Y) (even iteration) can not be used as spatial prediction block (not yet calculate and arrive), therefore, selected (X+1, Y) (strange Secondary iteration), (X+1, Y) (even iteration) replaces.
According to each candidate vector for the current calculation block that estimated value obtains, candidate vector includes：Spatial candidate vector, when Between candidate vector, layering candidate vector, renewal vector.
Compare by region centered on each candidate vector direction band of position in the estimator centered on current calculation block Compare the difference between the comparison window in region centered on window and current calculation block, vector corresponding to selection differential reckling is as working as Preceding piece of result of calculation.It is respectively directed to the band of position, difference calculates in certain relatively window, and is carried out about by comparing range Beam.A kind of conventional way be will compare window be dimensioned to it is consistent with block size, such as 8*8 pixels in example.Difference is in pixel It completes to calculate (calculating as unit of pixel) under grid, frequently with absolute difference and (SAD：sum of absolute Difference) or absolute difference is averaged (MAD：Mean of absolute difference) form.
Renewal vector is superimposed with random vector in ASC tables by spatial candidate vector and is formed.Wherein, since ASC tables have Asia Pixel precision (such as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is directed toward with non-integer center When position, need to the value of non-integer coordinates position be obtained using interpolation algorithm.
Include in each multilayer 3DRS blocks estimation iterative process：Judge the every of the current calculation block that estimated value obtains Whether the direction band of position of a candidate vector is integer (being directed toward whether the corresponding center position coordinates in the band of position are integer), If so, directly calculating the comparison window in region and region centered on current calculation block centered on the candidate vector direction band of position Comparison window between difference；If it is not, then passing through the dull area for being segmented cube interpolation method interpolation and going out candidate vector direction The comparison window at domain center, then the comparison window of regional center that the obtained candidate vector of calculating difference is directed toward is with current calculation block Difference between the comparison window of central area；The candidate of selection and the difference minimum of the comparison window in region centered on current calculation block Difference estimation result of the vector as current calculation block current iteration.
The device of the present embodiment, compared with the searching route searched in the same direction line by line, block grid difference field obtained by the present embodiment Each piece of difference estimation value of the bottom generally more uniformly, effectively prevents causing since searching route is single in pyramid " abnormal point " generation.
Fig. 4 is shown according to a kind of knot of second embodiment of difference estimation device based on multilayer 3DRS provided by the invention Structure block diagram.A kind of difference estimation device based on multilayer 3DRS of the present embodiment is mainly executed by intelligent terminal.Wherein, intelligence Terminal includes but not limited to：PC, laptop, mobile phone, tablet computer, video playback apparatus etc., intelligent terminal can The intelligent terminal of 3D videos can be played or parse by thinking.The device includes pyramid generation unit 41, searching route setting unit 42, unit 43 and depth map generating unit 44 are iterated to calculate.
Pyramid generation unit 41, the image gold for building the default number of plies respectively according to the left figure and right figure of 3D frame images Word tower generates the block grid difference field pyramid for presetting block size according to described image pyramid.
Specifically, building the figure of the default number of plies respectively using average weighted method according to the left figure of 3D frame images and right figure As pyramid, described image pyramid is initialized as to the block grid difference field pyramid of default block size.In image pyramid Including left figure pyramid and right figure pyramid.The left figure of 3D frame images builds the left figure pyramid of the default number of plies, 3D frame images Right figure also builds the right figure pyramid of the default number of plies, wherein the high pyramidal length and width of a tomographic image is bottom one tomographic image gold respectively 1/2 times of the length and width of word tower.The block grid difference field pyramid for presetting block size, difference field gold are generated according to image pyramid The generation method of word tower is by identical layer image pixel coordinates in right figure pyramid compared to identical tomographic image in left layer pyramid The phasor difference of the corresponding coordinate of pixel constitutes difference field pyramid.
In the present embodiment, pyramid generation unit 41 includes building image pyramid module 411 and generation difference field gold word Tower module 412.
Image pyramid module 411 is built, the length of left figure and right figure for judging 3D frame images and wide whether respectively energy By the 2 of the block length of sidenDivide exactly again, n is the default number of plies, if so, building default layer respectively according to the left figure of 3D frame images and right figure Several multilayer 3DRS pyramids, if it is not, being then worth to energy in the left figure of 3D frame images and the surrounding benefit of right figure using nearest interpolation By the 2 of the block length of sidenThe size divided exactly again, left figure and right figure further according to the 3D frame images after benefit value build the golden word of the default number of plies Tower.
Difference field pyramid module 412 is generated, the block net for the multilayer pyramid to be initialized as to default block size Lattice difference field pyramid.
Searching route setting unit 42, the searching route for default iterations to be arranged, each iteration is compared to upper one Searching route is rotated by 90 ° by secondary iteration relative to the relevant image of current iteration and each difference field figure, the search road of each iteration Diameter is to be searched for by row/column, and the other end, the opposite search road of adjacent rows/row searching route are searched from one end of every row/column Diameter.
Specifically, every time iteration compared to last iteration by searching route relative to the relevant image of current iteration and each Difference field figure is rotated by 90 °, and can be to rotate clockwise or rotate counterclockwise, wherein can be related by rotating current iteration Image and each difference field realize, for rotating counterclockwise, including：When each iteration, relevant image and each difference field After figure is rotated by 90 ° counterclockwise, left-right reversed is carried out to each vector in each difference field after overturning, and to each vector x after exchange Durection component negates processing and realizes.When each iteration, after relevant image and each difference field figure are rotated by 90 ° counterclockwise, to turning over Each vector after turning in each difference field carries out left-right reversed, and negates processing to each vector x durection component after exchange and realize. Specific implementation is shown in two kinds of realization methods in iterative calculation unit 43.
Unit 43 is iterated to calculate, for successively using described search path to each layer block map declination according to preset estimator Different field pyramid is completed to preset the multilayer 3DRS blocks estimation iterative calculation of iterations, obtains in the pyramid of block grid difference field most Each piece of difference estimation value of bottom.
Specifically, successively using described search path to each layer block map declination since top according to preset estimator Different field pyramid is completed to preset the multilayer 3DRS blocks estimation iterative calculation of iterations, obtains in the pyramid of block grid difference field most Each piece of difference estimation value of bottom.
Wherein, multilayer 3DRS blocks estimation iterative calculation is each of the acquisition difference field pyramid bottom in multilayer 3DRS algorithms The block estimation iterative calculation step of block difference estimation value, but the step change in multilayer 3DRS algorithms the structure of estimator and Searching route.
Further, it includes iteration module 431 and estimation of Depth value acquisition module that unit 43 is iterated to calculate in the present embodiment 432。
Iteration module 431, the multilayer 3DRS blocks for completing default iterations to each layer block grid difference field pyramid If searching route is to search for line by line when estimation iterative calculation, multilayer each time is completed to each layer block grid difference field pyramid After the estimation iterative calculation of 3DRS blocks, judgement is described to work as whether time iterations reach default iterations, if so, stopping to this The pyramid iterative calculation of layer block grid difference field；If it is not, described search path is then used to change the estimation of multilayer 3DRS blocks next time In generation, calculates, and repeats current iteration process and is calculated as next iteration, until reaching default iterations.
Two ways may be used to realize in iteration module 431：
Example one, estimator are to be placed on the spatial prediction block that this in tilting Y estimators does not iterate to calculate and the tilting Y The block position of symmetrical iteration in estimator, then estimator need to be matched with searching route.It is to search for line by line with searching route For, as shown in attached drawing 5a, 5b, 5c and 5d, for example, using searching for line by line, if default iterations are 4, first time iteration is 5a, to have done the design of estimated below device with the matched tilting Y estimators of searching route.By taking 4 iteration as an example, the 1st iteration (figure 5a), estimator design is identical as the estimator of example odd-times iteration in the step 23 of first embodiment.When the 2nd iteration (Fig. 5 b) is searched for from top to bottom, and from the upper left corner to the lower right corner, being equal in this step will change to second of multilayer 3DRS blocks estimation After the generation relevant image of calculating and each difference field figure are rotated by 90 ° counterclockwise, each vector in each difference field after overturning is carried out left Right exchange negates processing to each vector x durection component after exchange, repeats current iteration process as second of iteration meter It calculates, and such realization method that need not rotate image and each difference field figure, it is more easy to by intuitivism apprehension, such realization method is estimated Gauge is that spatial prediction block is (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), the structure of time prediction block (X-2, Y+2). It can similarly obtain, attached drawing (Fig. 5 c), estimator design and first embodiment are seen in the iterative search path of third time in this realization method Step 23 in example even-times iteration estimator it is identical.It can similarly obtain, the iteration of third time is searched in this realization method Attached drawing (Fig. 5 d) is seen in rope path, and when up to lower search, estimator is spatial prediction block (X-1, Y), (X, Y-1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2).It is identical as in the search of interlaced alternative direction, for can not when estimator designs Block as spatial prediction block (not yet calculate) uses symmetry blocks to replace.
Example two is estimated using the search pattern and estimator of first time iteration in attached drawing 5a.Complete the 1st iteration Afterwards, second of multilayer 3DRS blocks estimation is iterated to calculate into relevant image and each difference field figure turns over turn 90 degrees counterclockwise.For turning over Each vector carries out left and right exchange in each difference field after turning, and negates processing to vector x durection component, i.e.,：
V (x, y)=(Vx(x,y),Vy(x,y))T
V ' (x, y)=(- Vy(y,x),Vx(y,x))T
Wherein, V (x, y) is first time iterative search result vector, and V ' (x, y) is vector after overturning negates.Repeat first Secondary iterative process is iterated calculating according to attached drawing 5a searching routes and estimator and changes as second as second of iteration Generation.Iterating to calculate relevant image and each difference field figure, which turns over and turn 90 degrees, similarly is estimated to third time multilayer 3DRS blocks according to above-mentioned, Each vector carries out left and right exchange, and negates processing to vector x durection component, repeats the 1st iterative process as second Iteration carries out third time iteration, similarly carries out being sequentially completed 4 iteration or higher number of iterations again.
Wherein, it is for obtaining that the above-mentioned 3DRS blocks of multilayer next time estimation, which iterates to calculate relevant image and each difference field figure, Next iteration calculate needed for spatial candidate vector, time candidate vector, layering candidate vector, renewal vector image and respectively Difference field figure, further include when time search iteration obtain as a result, i.e. when secondary iteration after this layer of block grid difference field pyramid in respectively Block difference estimation value.
Estimator is the spatial prediction for not iterating to calculate this in tilting Y estimators (Oblique Y estimator) Block is placed on the block position with symmetrical iteration in the tilting Y estimators, then estimator need to be matched with searching route.To search Rope path is that as shown in attached drawing 5a, 5b, 5c and 5d, the design of estimated below device has been done with tilting Y estimators for searching for line by line. By taking 4 iteration as an example, the 1st iteration (Fig. 5 a), estimator design changes with the example odd-times in the step 23 of first embodiment The estimator in generation is identical.When the 2nd iteration (Fig. 5 b), when up to lower search, estimator be spatial prediction block be (X-1, Y), (X, Y+1), hierarchical prediction block (X, Y), the structure of time prediction block (X+2, Y-2)；Under supreme search when, estimator is spatial prediction Block is (X+1, Y), (X, Y+1), hierarchical prediction block (X, Y), the structure of time prediction block (X-2, Y+2).3rd iteration (figure 5c), estimator design is identical as the estimator of even-times iteration of the example in the step 23 of first embodiment.The 4th iteration When (Fig. 5 d), under supreme search when, estimator be spatial prediction block be (X+1, Y), (X, Y-1), hierarchical prediction block (X, Y), when Between prediction block (X-2, Y+2) structure；When up to lower search, estimator is spatial prediction block (X-1, Y), (X, Y-1), and layering is pre- Survey block (X, Y), the structure of time prediction block (X+2, Y-2).It is identical as in the search of interlaced alternative direction, estimator design when for Symmetry blocks can not be used to replace as the block of spatial prediction block (not yet calculate).
Example two is estimated using the search pattern and estimator of first time iteration in attached drawing 5a.Complete the 1st iteration Afterwards, second of multilayer 3DRS blocks estimation is iterated to calculate into relevant image and each difference field figure turns over turn 90 degrees counterclockwise.For turning over Each vector carries out left and right exchange in each difference field after turning, and negates processing to vector x durection component, i.e.,：
V (x, y)=(Vx(x,y),Vy(x,y))T
V ' (x, y)=(- Vy(y,x),Vx(y,x))T
Wherein, V (x, y) is first time iterative search result vector, and V ' (x, y) is vector after overturning negates.Repeat first Secondary iterative process is iterated calculating according to attached drawing 5a searching routes and estimator and changes as second as second of iteration Generation.Iterating to calculate relevant image and each difference field figure, which turns over and turn 90 degrees, similarly is estimated to third time multilayer 3DRS blocks according to above-mentioned, Each vector carries out left and right exchange, and negates processing to vector x durection component, repeats the 1st iterative process as second Iteration carries out third time iteration, similarly carries out being sequentially completed 4 iteration or higher number of iterations again.
Wherein, it is for obtaining that the above-mentioned 3DRS blocks of multilayer next time estimation, which iterates to calculate relevant image and each difference field figure, Next iteration calculate needed for spatial candidate vector, time candidate vector, layering candidate vector, renewal vector image and respectively Difference field figure, further include when time search iteration obtain as a result, i.e. when secondary iteration after this layer of block grid difference field pyramid in respectively Block difference estimation value.
According to each candidate vector for the current calculation block that estimated value obtains, candidate vector includes：Spatial candidate vector (Spatial Candidate), time candidate vector (Temporal Candidate), layering candidate vector (Hierarchy Candidate), renewal vector (Update Candidate).Wherein, estimated by the difference of last layer corresponding blocks by spatial prediction block Evaluation is multiplied by and 2 obtains spatial candidate vectorial (Spatial Candidate), by time prediction block by previous frame or last time iteration Difference estimation result obtain time candidate vector (Temporal Candidate), complete by current iteration by hierarchical prediction block Blocking difference estimation is worth to layering candidate vector (Hierarchy Candidate), renewal vector (Update Candidate) random vector is generated with ASC tables be superimposed generation by spatial candidate vector.
Compare by region centered on each candidate vector direction band of position in the estimator centered on current calculation block Compare the difference (Distortion) between the comparison window in region centered on window and current calculation block, selection differential reckling institute is right Answer result of calculation of the vector as current block.It is respectively directed to the band of position, difference is centainly comparing in window (Search window) It calculates, and is constrained by comparing range (Search Area).A kind of conventional way is will to compare being dimensioned to for window It is consistent with block size, such as 8*8 pixels in example.Difference is completed to calculate (i.e. to calculate as unit of pixel) under pixel grid, Chang Cai With absolute difference and (SAD：Sum of absolute difference) or absolute difference be averaged (MAD：mean of absolute Difference form).
Renewal vector is superimposed with random vector in ASC tables by spatial candidate vector and is formed.Wherein, since ASC tables have Asia Pixel precision (such as 0.25 pixel), and other vectors are passed to by space vector.When candidate vector is directed toward with non-integer center When position, need to the value of non-integer coordinates position be obtained using interpolation algorithm.
Include in each multilayer 3DRS blocks estimation iterative process：Judge the every of the current calculation block that estimated value obtains Whether the direction band of position of a candidate vector is integer (being directed toward whether the corresponding center position coordinates in the band of position are integer), If so, directly calculating the comparison window in region and region centered on current calculation block centered on the candidate vector direction band of position Comparison window between difference；If it is not, then passing through the dull area for being segmented cube interpolation method interpolation and going out candidate vector direction The comparison window at domain center, then the comparison window of regional center that the obtained candidate vector of calculating difference is directed toward is with current calculation block Difference between the comparison window of central area；The candidate of selection and the difference minimum of the comparison window in region centered on current calculation block Difference estimation result of the vector as current calculation block current iteration.
Estimation of Depth value acquisition module 432, is connected with iteration module 431, for determining that block grid difference field is pyramidal The estimation iterative calculation of multilayer 3DRS blocks stops, and obtains each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
Depth map generating unit 44, for being given birth to according to each piece of difference estimation value of the bottom in the pyramid of block grid difference field At depth map.
Due to the bottom in block grid difference field pyramid obtained by the present embodiment each piece of difference estimation value generally more It is balanced, reliable, improve the generation for effectively inhibiting 3DRS algorithms to estimate singular point in difference field after searching route.
This step has many modes may be implemented, and can usually be corroded by block and extend block grid difference field to obtain pixel Difference field figure, then by the way that pixel difference field figure is carried out difference deep conversion in a manner of tabling look-up or is linear etc., finally by pixel difference Different field figure, which is quantized in the section of 0 to 255 (unit 1), obtains depth map, can convert 3D frame images after generating depth map 2D+Z (depth) format generated at can be used for various visual angles (Multi-view) 3D rendering.
For estimation of Depth, one more compact, that range is smaller difference field is conducive in final 0~255 range The quantization of smaller error.Contrast experiment has been carried out using two width difference pictures, has been shown by experimental data：Four-way in the present embodiment Searching route multilayer 3DRS is compared with the searching route in searching route multilayer 3DRS in the same direction line by line or first embodiment, difference Range is minimum, and (the first width figure, is searched for line by line：72.25 pixels, the search of interlaced alternative direction：76 pixels improve searching route： 63.5 pixels；Second width figure, is searched for line by line：89.5 pixels, the search of interlaced alternative direction：100 pixels improve searching route： 84.5 pixels), and the generation of singular point in 3DRS algorithms estimation difference field is effectively inhibited in the present embodiment.
In conclusion a kind of difference estimation method and device based on multilayer 3DRS of the present invention, by using adjacent two The opposite searching route of row/column searching route, and each iteration is made to rotate image and searching route compared to last iteration 90 degree, Iterative path is made to become four-way iteration, keeps the difference estimation result of 3D frame images more balanced, reliable, further strengthen It is convergent in different directions that multilayer 3DRS algorithms uniformity convergent in different directions further strengthens multilayer 3DRS algorithms Uniformity, to obtain more accurate, uniform depth estimation result.
The above-described embodiments merely illustrate the principles and effects of the present invention, and is not intended to limit the present invention.It is any ripe The personage for knowing this technology can all carry out modifications and changes to above-described embodiment without violating the spirit and scope of the present invention.Cause This, institute is complete without departing from the spirit and technical ideas disclosed in the present invention by those of ordinary skill in the art such as At all equivalent modifications or change, should by the present invention claim be covered.

Claims (8)

1. a kind of difference estimation method based on multilayer 3DRS, which is characterized in that including：
The image pyramid for building the default number of plies respectively according to the left figure of 3D frame images and right figure is given birth to according to described image pyramid At the block grid difference field pyramid of default block size；
The searching route of default iterations is set, and each iteration changes searching route relative to this compared to last iteration It is rotated by 90 ° for relevant image and each difference field figure, wherein the searching route of each iteration is to be searched for by row/column, from every One end of row/column searches the other end, the adjacent rows/opposite searching route of row searching route；
Described search path is successively used to complete to preset iteration to each layer block grid difference field pyramid according to preset estimator The multilayer 3DRS blocks estimation iterative calculation of number, obtains each piece of difference estimation value of the bottom in the pyramid of block grid difference field, The estimator be with the matched tilting Y estimators of searching route, for will in the matched tilting Y estimators of searching route when The secondary spatial prediction block not iterated to calculate is placed on the block position with symmetrical iteration in the tilting Y estimators.
2. the method as described in claim 1, which is characterized in that described to obtain each of the bottom in the pyramid of block grid difference field After block difference estimation value, further include being given birth to according to each piece of difference estimation value of the bottom in described piece of grid difference field pyramid At depth map.
3. the method as described in claim 1, which is characterized in that described to the pyramidal default iteration in each layer block grid difference field The multilayer 3DRS blocks estimation iterative calculation of number, including：
After completing the iterative calculation of multilayer 3DRS blocks estimation each time to each layer block grid difference field pyramid, judge when time iteration time Whether number reaches default iterations, if so, stopping to this layer of block grid difference field pyramid iterative calculation；If it is not, then adopting Multilayer 3DRS blocks next time are estimated to iterate to calculate with described search path, repeat current iteration process as next iteration It calculates, until reaching default iterations.
4. the method as described in claim 1, which is characterized in that the left figure and right figure according to 3D frame images is built pre- respectively If the pyramid of the number of plies, including：
Judge the left figure of 3D frame images and the length of right figure and it is wide whether respectively can be by the 2 of the block length of sidenTo divide exactly again, n is the default number of plies, If so, building the multilayer 3DRS pyramids of the default number of plies respectively according to the left figure of 3D frame images and right figure；If it is not, then using most Nearly interpolation is worth in the left figure of 3D frame images and the surrounding benefit of right figure to can be by the 2 of the block length of sidenThe size divided exactly again, further according to benefit value The left figure and right figure of 3D frame images afterwards build the pyramid of the default number of plies.
5. a kind of difference estimation device based on multilayer 3DRS, which is characterized in that including：
Pyramid generation unit, the image pyramid for building the default number of plies respectively according to the left figure and right figure of 3D frame images, The block grid difference field pyramid for presetting block size is generated according to described image pyramid；
Searching route setting unit, the searching route for default iterations to be arranged, each iteration is compared to last iteration Searching route is rotated by 90 ° relative to the relevant image of current iteration and each difference field figure, wherein the searching route of each iteration It is to be searched for by row/column, the other end, the opposite search road of adjacent rows/row searching route is searched from one end of every row/column Diameter；
Unit is iterated to calculate, is connected with pyramid generation unit and searching route setting unit, for according to preset estimator The multilayer 3DRS blocks that iterations are completed to preset in described search path to each layer block grid difference field pyramid are successively used to estimate Iterative calculation, obtains each piece of difference estimation value of the bottom in the pyramid of block grid difference field, and the estimator is and search road The matched tilting Y estimators of diameter, for by in the matched tilting Y estimators of searching route when the secondary space not iterated to calculate is pre- It surveys block and is placed on the block position with symmetrical iteration in the tilting Y estimators.
6. device as claimed in claim 5, which is characterized in that further include depth map generating unit, for according to described piece of net Each piece of difference estimation value of the bottom generates depth map in the pyramid of lattice difference field.
7. device as claimed in claim 5, which is characterized in that the iterative calculation unit, including：
Iteration module, for successively using described search path to each layer block grid difference field pyramid according to preset estimator The multilayer 3DRS blocks estimation iterative calculation of the default iterations carried out：
After completing the iterative calculation of multilayer 3DRS blocks estimation each time to each layer block grid difference field pyramid, judge when time iteration time Whether number reaches default iterations, if so, stopping to this layer of block grid difference field pyramid iterative calculation；If it is not, then adopting Multilayer 3DRS blocks next time are estimated to iterate to calculate with described search path, repeat current iteration process as next iteration It calculates, until reaching default iterations；
Estimation of Depth value acquisition module, is connected with iteration module, for determining the pyramidal multilayer 3DRS blocks in block grid difference field Estimation iterative calculation stops, and obtains each piece of difference estimation value of the bottom in the pyramid of block grid difference field.
8. device as claimed in claim 5, which is characterized in that the pyramid generation unit, including：
Image pyramid module is built, whether the length and width of left figure and right figure for judging 3D frame images respectively can be by the block length of sides 2nDivide exactly again, n is the default number of plies, if so, building the multilayer of the default number of plies respectively according to the left figure of 3D frame images and right figure 3DRS pyramids；If it is not, being then worth to can be by the block length of side in the left figure of 3D frame images and the surrounding benefit of right figure using nearest interpolation 2nThe size divided exactly again, left figure and right figure further according to the 3D frame images after benefit value build the pyramid of the default number of plies；
Difference field pyramid module is generated, for generating the block grid difference field gold for presetting block size according to described image pyramid Word tower.
CN201510514518.1A 2015-08-20 2015-08-20 A kind of difference estimation method and device based on multilayer 3DRS Active CN105007495B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510514518.1A CN105007495B (en) 2015-08-20 2015-08-20 A kind of difference estimation method and device based on multilayer 3DRS

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510514518.1A CN105007495B (en) 2015-08-20 2015-08-20 A kind of difference estimation method and device based on multilayer 3DRS

Publications (2)

Publication Number Publication Date
CN105007495A CN105007495A (en) 2015-10-28
CN105007495B true CN105007495B (en) 2018-07-20

Family

ID=54379983

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510514518.1A Active CN105007495B (en) 2015-08-20 2015-08-20 A kind of difference estimation method and device based on multilayer 3DRS

Country Status (1)

Country Link
CN (1) CN105007495B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105809717B (en) * 2016-03-10 2019-01-11 上海玮舟微电子科技有限公司 A kind of depth estimation method, system and electronic equipment
CN105952233B (en) * 2016-05-10 2019-08-27 中国电力科学研究院 The mouth width of adjacent tower body determines method and device in power transmission tower
CN106993108B (en) * 2017-04-07 2020-08-28 上海顺久电子科技有限公司 Method and device for determining random quantity of video image in motion estimation
CN112215880B (en) * 2019-07-10 2022-05-06 浙江商汤科技开发有限公司 Image depth estimation method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011001364A1 (en) * 2009-06-29 2011-01-06 Nxp B.V. Parallel three-dimensional recursive search (3drs) meandering algorithm
CN102447892A (en) * 2010-09-30 2012-05-09 宝利微电子系统控股公司 Motion estimation method and device
CN102881014A (en) * 2012-09-07 2013-01-16 航天恒星科技有限公司 Quick stereo matching method based on graph cut
CN103679680A (en) * 2012-08-31 2014-03-26 富士通株式会社 Stereo matching method and system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090304293A1 (en) * 2008-06-08 2009-12-10 Te-Hao Chang Motion estimation method and related apparatus for efficiently selecting motion vector
JP2015503253A (en) * 2011-10-10 2015-01-29 コーニンクレッカ フィリップス エヌ ヴェ Depth map processing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011001364A1 (en) * 2009-06-29 2011-01-06 Nxp B.V. Parallel three-dimensional recursive search (3drs) meandering algorithm
CN102447892A (en) * 2010-09-30 2012-05-09 宝利微电子系统控股公司 Motion estimation method and device
CN103679680A (en) * 2012-08-31 2014-03-26 富士通株式会社 Stereo matching method and system
CN102881014A (en) * 2012-09-07 2013-01-16 航天恒星科技有限公司 Quick stereo matching method based on graph cut

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Optimization of Hierarchical 3DRS Motion Estimators for Picture Rate Conversion;Aderenne Heinrich et.al.;《IEEE Journal of Selected Topics in Signal Processing》;20110430;第262页第2栏最后一段-第266页倒数第二段，图1-3 *
基于三维递归搜索的多级运动估计视频帧率上转换算法;贾茜等;《电子与信息学报》;20121030;第34卷(第10期);全文 *
应用于格式转换的三维递归搜索块匹配算法;王建国，周源华;《计算机工程与应用》;20031231;全文 *

Also Published As

Publication number Publication date
CN105007495A (en) 2015-10-28

Similar Documents

Publication Publication Date Title
CN105007495B (en) A kind of difference estimation method and device based on multilayer 3DRS
CN107071416B (en) HEVC intra-frame prediction mode rapid selection method
CN105100807B (en) A kind of frame per second method for improving based on motion vector post-processing
TWI685250B (en) Method, device and encoder for selecting the best reference frame of HEVC encoding
US20130072299A1 (en) Three-dimensional motion mapping for cloud gaming
CN106331703A (en) Video coding and decoding method, and video coding and decoding device
CN106303543A (en) Encoding video pictures and the method for decoding, encoding device and decoding device
CN107734335A (en) Image prediction method and relevant apparatus
CN102801976B (en) Inter-frame module selecting method based on three-dimensional wavelet video code
CN106028047B (en) A method of the fast mode decision and PU of the intra prediction based on Hadamard transformation divide
CN104219532B (en) The method and apparatus for determining interpolation frame between the method in wisp region, frame of video
CN107888928A (en) Motion prediction compensation method and equipment
CN105469386B (en) A kind of method and device of determining stereoscopic camera height and pitch angle
TWI297576B (en) Method of motion vector determination in digital video compression
CN106464865A (en) Block-based static region detection for video processing
CN108235031A (en) A kind of motion vector decoder method and decoder
CN115118977B (en) Intra-frame prediction encoding method, system, and medium for 360-degree video
CN105957003B (en) The super dimension method for reconstructing of porous media based on study
TW201929550A (en) Method and image processing apparatus for video coding
US10448048B2 (en) Method of motion estimation and image processing apparatus using the same
CN110113608A (en) Video coding method for fast searching in a kind of GPU based on rate-distortion optimization
Kolesnikov Efficient online algorithms for the polygonal approximation of trajectory data
CN105809717B (en) A kind of depth estimation method, system and electronic equipment
CN110111380A (en) 3D rendering transmission and method for reconstructing based on depth camera
CN108024113A (en) A kind of adaptive compression domain small target tracking method of target accounting

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20200331

Address after: 215634 north side of Chengang road and west side of Ganghua Road, Jiangsu environmental protection new material industrial park, Zhangjiagang City, Suzhou City, Jiangsu Province

Patentee after: ZHANGJIAGANG KANGDE XIN OPTRONICS MATERIAL Co.,Ltd.

Address before: 201203, room 5, building 690, No. 202 blue wave road, Zhangjiang hi tech park, Shanghai, Pudong New Area

Patentee before: WZ TECHNOLOGY Inc.