CN106791770B - A kind of depth map fusion method suitable for DIBR preprocessing process - Google Patents
A kind of depth map fusion method suitable for DIBR preprocessing process Download PDFInfo
- Publication number
- CN106791770B CN106791770B CN201611185808.7A CN201611185808A CN106791770B CN 106791770 B CN106791770 B CN 106791770B CN 201611185808 A CN201611185808 A CN 201611185808A CN 106791770 B CN106791770 B CN 106791770B
- Authority
- CN
- China
- Prior art keywords
- map
- depth
- depth map
- dibr
- degree distribution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000008569 process Effects 0.000 title claims abstract description 25
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 13
- 238000007781 pre-processing Methods 0.000 title claims abstract description 13
- 238000009826 distribution Methods 0.000 claims abstract description 37
- 238000009792 diffusion process Methods 0.000 claims abstract description 21
- 230000000977 initiatory effect Effects 0.000 claims abstract description 19
- 238000005457 optimization Methods 0.000 claims abstract description 15
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 230000008859 change Effects 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 230000000644 propagated effect Effects 0.000 claims description 3
- 238000012804 iterative process Methods 0.000 claims description 2
- 238000004519 manufacturing process Methods 0.000 claims description 2
- 239000004576 sand Substances 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 21
- 238000009499 grossing Methods 0.000 abstract description 7
- 230000009466 transformation Effects 0.000 abstract description 6
- 230000003044 adaptive effect Effects 0.000 abstract description 5
- 230000000717 retained effect Effects 0.000 abstract description 4
- 238000009877 rendering Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 10
- 238000001914 filtration Methods 0.000 description 8
- 238000002474 experimental method Methods 0.000 description 5
- 238000013507 mapping Methods 0.000 description 5
- 230000001154 acute effect Effects 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000013442 quality metrics Methods 0.000 description 3
- 230000002146 bilateral effect Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
- G06T15/205—Image-based rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a kind of depth map fusion methods for DIBR preprocessing process, include the following steps：Gradient analysis is carried out to original depth-map, estimates out the hole region in new viewpoint image；Initial effects degree distribution map is generated according to hole region distribution is estimated；Initial effects degree distribution map is diffused using texture image under dimension transformation space；The depth map of original depth-map and pre-smoothed is merged based on the disturbance degree distribution map after diffusion, generates the depth map after optimization.The method of the present invention is under efficient dimension transformation space, the depth map of original depth-map and pre-smoothed is merged by the disturbance degree distribution map spread based on textural characteristics, to realize adaptive adjustment of the depth map after optimization in different zones smoothed intensity, the smoothing effect of hole region has not only been effectively retained to eliminate empty influence, and effectively prevent to there is not the excessively smooth of hole region and caused by additional twisted phenomena.
Description
Technical field
The invention belongs to 3 D video technical fields, and in particular to the Video Quality Metric technology of 2D/3D, it is especially a kind of to be applicable in
In the depth map fusion method of DIBR preprocessing process.
Background technology
Currently, three-dimensional (3D) video is gradually popularized, Chinese Central Television (CCTV) also pilots when New Year's Day in 2012
3D channels, 3D videos have been increasingly becoming a kind of trend of current development.However, video source deficiency, which becomes, restricts this production
The main bottleneck that industry is risen.In this case, it is to solve the problems, such as this effective way 2D videos to be switched to 3D videos.
2D videos are switched into 3D videos and generally speaking there are two kinds of rendering modes：One of which is by using some way
The right and left eyes image pair for providing parallax is directly reconstructed from single video frame；Another kind is the rendering based on depth map
(Depth Image-based Rendering, DIBR), its transformation result is to attached each frame on the basis of original video
Corresponding depth map, after being finally converted to binocular tri-dimensional video by the display terminal output of embedded DIBR processing modules
Watched (referring to " film 2D/3D switch technologies summarize [J] ", Liu Wei, Wu Yihong, Hu Zhanyi,《CAD with
Graphics journal》, 2012,24 (1):14-28).Compared with the former, three original features that the latter has with it：Efficient pressure
Contracting efficiency of transmission, the depth of field having by force and in real time tridimensional video generation with existing 2D technologies and the compatibility of distinct device
The technical advantages such as adjustment and Fast rendering synthesis, occupy absolute leading position in markets such as emerging 3DTV, 3D mobile terminals,
It is the direction of 3D Rendering future developments.
DIBR renderings are the important steps in the 2D/3D conversion methods based on depth map, it can utilize depth information wash with watercolours
Virtual three-dimensional video-frequency is dyed, to be finally completed 2D to 3D " fundamental change ".Although this technology has many advantages,
Still there is its limitation.Since the DIBR mapping relations converted according to depth map from reference picture fictionalize right and left eyes image,
The variation of viewpoint may cause to be exposed in new images by the part background area that foreground object blocks in original image, and this
Subregion does not have corresponding texture mapping in conversion process, therefore cavitation will be generated on target image.This
Problem is the research hotspot of DIBR technologies in recent years, and improves the importance of 3D rendering quality.It is current for this problem
Frequently with process flow as shown in Fig. 1, link is pre-processed and after DIBR based on figure by the way that depth map is added before DIBR
Filling up for cavity is completed as recovery technique.
Depth map pretreatment is typically to carry out smoothly, drawing obtained new viewpoint in this way to depth map using all kinds of filters
It is middle comprising smaller cavity, to be conducive to further fill up.Such methods operational efficiency is high, and effect is apparent, but smooth filter
Wave may result in the object edges areas (the especially edge of vertical direction) in virtual image and generate geometric deformation.Therefore exist
Undistorted turn of virtual image can not be effectively ensured in depth map pretreatment link in existing DIBR technologies in 2D/3D Video Quality Metrics
Synthesis is changed, to affect the practical conversion effect of 3D videos.
Invention content
In view of this, the purpose of the present invention is the deficiency for existing DIBR depth maps preconditioning technique link, pass through dimension
The disturbance degree distribution map of generation is spread to be merged to the depth map of original depth-map and pre-smoothed under degree transform domain, thus
Realize that the depth map after optimization in the adaptive adjustment of different zones smoothed intensity, has not only been effectively retained the smooth of hole region
Effect to eliminate empty influence, and effectively prevent to there is not the excessively smooth of hole region and caused by additionally distortion it is existing
As to promote 3D virtual image rendering effects.
In order to achieve the above objectives, the present invention uses following technical scheme：
A kind of depth map fusion method suitable for DIBR preprocessing process includes the following steps：
A) in original depth-map DOriIn, estimate change in depth rule of the operator according to right and left eyes virtual image using cavity
To the hole region R of generationHIt is estimated；
B initial effects degree distribution map I) is generatedf-init；
C) to initial effects degree distribution map I under dimension transform domainf-initIt is diffused, generates the disturbance degree point after optimization
Butut If；
D) with the disturbance degree distribution map I after optimizationfTo original depth-map DOriWith the depth map D of pre-smoothedPreMelted
It closes, generates the depth map D after optimizationFin。
Wherein, step A) described in cavity estimate operator and be：
Wherein, RHIndicate that the cavity of prediction, r (x, y) are indicated in original depth-map DOriDepth value at middle coordinate (x, y),
DmaxBe setting generate virtual image parallax maximum value number of pixels, α be normalization factor (in 8bit gray level images, α=
255), DwidthIt is the number of pixels of picture traverse, λHIt is preset threshold factor；It is regarded if newly synthesized virtual view is left eye
Scheme, then i=l, otherwise, i=r.
Wherein, step B) in initial effects degree distribution map If-initIt is specifically defined as：
Wherein, RHIndicate the cavity of prediction, De(p) indicate point p to the distance for estimating empty edge.
Wherein, step C) under dimension transform domain to initial effects degree distribution map If-initIt is diffused specially:Diffusion
Function is defined as follows：
If[n]=(1-ad)If-init[n]+adIf[n-1]
Wherein, If-init[n] indicates that the pixel value of initial effects degree distribution map lastrow or a row, a ∈ (0,1) are diffusions
The feedback factor of function, d indicate adjacent sample x in dimension transform domainnAnd xn-1The distance between.
Wherein, adjacent sample x in dimension transform domainnAnd xn-1The distance between be defined as：
D=ct (xn)-ct(xn-1)
Wherein, ct (u) indicates that dimension transform domain, dimension conversion process are:
Wherein, Itexture(x) texture image of input is indicated, | I 'texture(x) | indicate the gradient intensity of texture image, σs
And σrIt is transmission device space and codomain parameter respectively, for adjusting the influence of propagation.σsValue range is 200~2500, σrIt takes
Value ranging from 0.1~10.
Wherein, be diffused as iterative process, and to realize symmetric propagation, if in an iteration diffusion according to from a left side to
The right side, sequence from top to bottom are propagated in the picture, then diffusion is passed according to sequence from right to left, from top to bottom in next iteration
It broadcasts.Iterations are 2~10 times.
Wherein, step D) in depth map fusion formula be：
DFin=IfDPre+(1-If)DOri。
The beneficial effects of the invention are as follows：Disturbance degree distribution map in the present invention is based on line under efficient dimension transformation space
The original cavity diffusion for managing feature and prediction, can reflect the distributed intelligence of structuring.Depth integration is carried out with this, it can be with
Replace traditional artificial parameter limited way come the more efficient determining filtering effect for reinforcing area to be repaired with adaptive diffusion way
Fruit, and exclude to reduce the filter effect of non-hole region, to overcome the depth map in conventional depth figure preprocessing process to distort
With smoothing problasm excessively, realizes and significantly improve the virtual rendering effects of 3D while cavity is repaired.
Description of the drawings
Fig. 1 is existing DIBR systems process chart；
Fig. 2 is flow chart of the method for the present invention；
Fig. 3 is the depth map and virtual image effect contrast figure using the method for the present invention.
Specific implementation mode
The invention will be further described with reference to the accompanying drawings and examples：
Fig. 1 shows existing DIBR systems process flow, for the original depth-map of input, first in pretreatment link
Smothing filtering is carried out to depth map, reduces the generation in most of cavity when rendering by the optimization of depth map part-structure with this；
Then, processing method is rendered using DIBR to reflect the pixel in reference picture using the camera parameter of depth image and calibration
It is mapped in target image；Finally, using hole-filling method, a small amount of cavity retained in render process is repaired, and export conversion
The right and left eyes virtual image gone out.
Wherein, DIBR renderings processing is the important step in 2D/3D conversion methods, it describes an accurate point-to-point
Mapping relations, virtual three-dimensional video-frequency can be rendered using depth information, to be finally completed 2D to 3D " fundamental change ".
Although this technology there are many advantages, still there is its limitation.Since DIBR turns from reference picture according to depth map
The mapping relations changed fictionalize right and left eyes image, and the variation of viewpoint may cause the part blocked by foreground object in original image to be carried on the back
Scene area is exposed in new images, and this subregion does not have corresponding texture mapping in conversion process, therefore will
Cavitation is generated on target image.This problem is the research hotspot of DIBR technologies in recent years, and improves 3D rendering matter
The importance of amount.
There is three classes solution for this problem is currently used：
1) depth of seam division video (LDV) format.Such method is fundamentally solved by new data Layer due to blocking
And the cavitation generated in depth map.However this technology when requiring video acquisition using special equipment, so not
It is suitable for 2D/3D conversions；
2) hole-filling.Such method is the hole-filling link after DIBR shown in Fig. 1.Such method it is main
Thought is the textural characteristics according to image, chooses sizeable texture block, is then found therewith around region to be repaired
Most similar Texture Matching block substitutes the texture block.This kind of method can repair large area region cavity, but block when reparation
Matching is based on greedy search, is likely to result in apparent reparation mistake.In addition to this, such method calculation amount is larger, therefore often
Auxiliary depth map pre-processes link to repair a small amount of cavity of reservation；
3) pretreatment of depth image, such method are the link before DIBR shown in Fig. 1.Such methods can be smooth
Discontinuous (depth acute variation) region in depth map increases the intensity energy of gaussian filtering to reduce the cavity in depth map
Improve the quality for generating stereotome.It, can be advance by the local optimum of depth map because such methods computational efficiency is higher
The most of cavity being likely to occur is eliminated, only includes smaller cavity, is conducive to further fill up, therefore be in DIBR systems
The important link of hole-filling.But on the other hand filtering be easy to cause the torsional deformation of object rectilinear direction fringe region.Though
Asymmetric smothing filtering and bilateral filtering so have been proposed at present to alleviate this problem, but the setting of these global parameters is filtered
Wave device still fully effective can not avoid the excessively smooth phenomenon in part, when the smoothing effect of part is excessive, still result in synthesis
The subregion object of new viewpoint view generate geometric deformation.
Therefore the depth map pretreatment link in 2D/3D Video Quality Metrics in existing DIBR systems still can not have completely
Effect ensures the undistorted conversion synthesis of virtual image, to affect the practical conversion effect of 3D videos.For this purpose, the method for the present invention
Secondary fusion is carried out to pretreated depth map and original depth-map by introducing disturbance degree distribution map, is realized adaptive
Part filter carries out DIBR renderings with depth map after further being optimized.
The method of the present invention is with texture image, original depth image and the pre-smoothed obtained by certain filtering method
Depth map data source as input generates the depth map after fusion optimization after treatment.Fig. 2 is the method stream of the present invention
Cheng Tu is described the specific implementation mode of the present invention in conjunction with Fig. 2.
If smooth discontinuous (depth acute variation) region that may only occur in cavity of depth map carries out, non-
The depth plot quality of hole region can be effectively maintained, and crossing smoothing effect also can preferably be inhibited.It is based on
This thought, the present invention propose a kind of new depth map fusion method, are realized by the disturbance degree distribution map of diffusion adaptive
The local smoothing method answered, specifically includes following steps：
A) in original depth-map DOriIn, estimate change in depth rule of the operator according to right and left eyes virtual image using cavity
To the hole region R of generationHIt is estimated.If defining the larger value of distance numerical value closer apart from observer in depth map
It indicates, is indicated with the smaller value of numerical value apart from observer's larger distance.So specifically, in left eye virtual image
In, cavity concentrates on the region of the ascending acute variation of depth value；In right eye virtual image, cavity is concentrated in depth value
The region of descending acute variation.Based on this, cavity estimates operator definitions and is：
Wherein, RHIndicate that the cavity of prediction, r (x, y) are indicated in original depth-map DOriDepth value at middle coordinate (x, y),
DmaxBe setting generate virtual image parallax maximum value number of pixels, α be normalization factor (such as in 8bit gray level images,
α=255), DwidthIt is the number of pixels of picture traverse, λHIt is preset threshold factor, value range is 1~5, in emulation experiment
λHTake 2；If newly synthesized virtual view is left-eye view, i=l, otherwise, i=r；
B initial effects degree distribution map I) is generatedf-init.It is influenced to eliminate cavity, the disturbance degree distribution map that the present invention defines encloses
Around the hole region expansion estimated.Based on this, initial effects degree distribution map If-initIt is specifically defined as：
Wherein, RHIndicate the cavity of prediction, De(p) indicate point p to the distance for estimating empty edge.From De(p) definition
As can be seen that closer to the region of hollow center, the depth map of corresponding pre-smoothed is bigger to the effect for eliminating cavity；
In non-hole region, the depth map of corresponding pre-smoothed is to eliminating empty effect very little；
C) to initial effects degree distribution map I under dimension transform domainf-initIt is diffused；
Because initial effects degree distribution map only only accounts for the direct distribution in cavity, if there is edge line in virtual image
Near cavity, then being allowed to generate deformation to the edge that may smoothly influence in cavity.In order to solve this problem, it needs
Diffusion appropriate is carried out according to the textural characteristics of image to initial effects degree distribution map, make cavity structure similar area tool nearby
There is similar disturbance degree, overcomes the generation of deformation while realizing local smoothing method with this；
Spread function is defined as follows：
If[n]=(1-ad)If-init[n]+adIf[n-1]
Wherein, If-init[n] indicates that the pixel value of initial effects degree distribution map lastrow or a row, a ∈ (0,1) are diffusions
The feedback factor of function, d=ct (xn)-ct(xn-1) indicate adjacent sample x in dimension transform domainnAnd xn-1The distance between.Here
Dimension transform domain be with Eduardo S.L.Gastal in 2011 et al. in article " Domain transform for edge-
The transformation space that the method proposed in aware image and video processing " obtains, its sharpest edges be
It can ensure that hyperspace is being reduced to the one-dimensional space under the premise of image texture characteristic, to substantially increase computational efficiency.
Specifically, ct (u) indicates that dimension transform domain, dimension conversion process are:
Wherein, Itexture(x) texture image of input is indicated, | I 'texture(x) | indicate the gradient intensity of texture image, σs
And σrIt is transmission device space and codomain parameter respectively, for adjusting the influence of propagation.σsValue range is 200~2500, σrIt takes
Value ranging from 0.1~10；
It can be seen that in dimension conversion process, the scene structure feature reflected in texture image is taken into account, and is become
The important evidence of initial effects degree distribution maps diffusion.Entire diffusion process effect is similar to two-sided filter, disturbance degree distribution map
As the scene characteristic of image is propagated further diffusion near cavity, but due to the reduction of dimension under dimension transformation space,
Its operation efficiency is far longer than traditional two-sided filter, and traditional two-sided filter is run under two bit spaces, on
Although the dimension conversion process for stating definition substantially increases operation efficiency, but the only filter under the one-dimensional space.In order to reach
Same effect, in the particular embodiment, the mode of diffusion iteration is realized.Again because dimension defined above is transformed
Journey is asymmetric, so to realize symmetric propagation, if according to from left to right, sequence from top to bottom exists for diffusion in an iteration
It propagates, is then spread according to sequence spread from right to left, from top to bottom in next iteration in image.Iterations are 2~10
Secondary, 3 diffusion effects of general iteration can reach stabilization, and iterations are 3 times in emulation experiment；
D) with the disturbance degree distribution map I after optimizationfTo original depth-map DOriWith the depth map D of pre-smoothedPreMelted
It closes, generates the depth map D after optimizationFin.Depth map fusion formula is：
DFin=IfDPre+(1-If)DOri
Using the disturbance degree distribution map after diffusion, by after smooth depth map and original depth-map carry out secondary fusion,
It retains and advantageous smooth region is eliminated to cavity.Due to disturbance degree distribution map IfCodomain may become in diffusion process
Change, therefore needs to normalize to region [0,1] when implementing；
Depth map after fusion optimization has inhibited a large amount of cavities to generate, so finally being carried out with it and texture image
DIBR is rendered, then the cavity retained on a small quantity is filled up by using simple interpolation method, so that it may to generate right and left eyes virtual image.
It is the experimental verification of the DIBR virtual image restorative procedures of the present invention below；
1) experiment condition：
It is in CPU CoreTM2 Quad CPU Q9400@2.66GHz, in 7 system of memory 4G, Windows
It is tested；
2) experiment content：
Experiment realization details according to the method for the present invention is specifically described referring to Fig. 3 and quality institute band is rendered to 3D
To be promoted.
The case where Fig. 3 is when handling one group of experimental image.Wherein, Fig. 3 (a) is original texture image, here we
It directly as right eye virtual image, then needing to generate left eye virtual image by DIBR methods.Fig. 3 (b) is shown not
By depth map pretreatment and the left eye virtual image that is directly rendered with DIBR of hole-filling, in newly-generated virtual image
Black discontinuous section be cavity, it is seen that cavity is mainly distributed in Fig. 3 (c) in original depth-map depth value from small to large
The foreground object left side edge of drastic change.Fig. 3 (d) is the depth map of the pre-smoothed obtained by bilateral filtering, it is seen that after smooth
The region of depth value drastic change is significantly less in depth map, therefore can obviously inhibit the generation in cavity.But such as Fig. 3
(b) shown in, cavity is concentrated mainly on foreground object left side edge, thus in Fig. 3 (d) smooth on the right side of foreground object to cavity
Elimination has no obvious effect.Shown in disturbance degree distribution map such as Fig. 3 (e) after the diffusion that the method for the present invention proposes, it is seen that its point
Cloth and intensity can be diffused using hole region as core according to the structure feature that image reflects.Such as man is left in image
Texture in the background of side on wall is vertically distributed, and the background shutter texture on the left of Ms is distributed in the horizontal direction,
It is obtained for good embodiment in the disturbance degree distribution map of these features after the diffusion.Fig. 3 (f) is based on Fig. 3 (e) by original depth
Depth map after the optimization obtained after the secondary fusions of depth map Fig. 3 (d) of degree figure Fig. 3 (c) and pre-smoothed, with Fig. 3 (d) phases
Than, it can be seen that smooth region is crossed on the right side of foreground object to be inhibited, and the smooth effect of hole region is then according to surrounding figure
As feature is effectively maintained.Fig. 3 (g) and Fig. 3 (h) is that Fig. 3 (d) and Fig. 3 (f) is used to input to obtain as depth map respectively
DIBR rendering results, it can be seen that in Fig. 3 (d) non-hole region cross smoothing effect in Fig. 3 (g) ellipse mark
Produce more apparent pattern distortion in region, and through the method for the present invention, treated that depth map renders image then in Fig. 3 (h)
There is no this problems.It can be seen that depth map fusion method proposed by the present invention in DIBR depth map preprocessing process to changing
The castering action that kind 3D visual effect quality is played.
Finally illustrate, the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, this field is common
Other modifications or equivalent replacement that technical staff makes technical scheme of the present invention, without departing from technical solution of the present invention
Spirit and scope, be intended to be within the scope of the claims of the invention.
Claims (6)
1. a kind of depth map fusion method for DIBR preprocessing process, it is characterised in that：Include the following steps：
A) in original depth-map DOriIn, operator is estimated according to the change in depth rule of right and left eyes virtual image to production using cavity
Raw hole region RHIt is estimated；
B initial effects degree distribution map I) is generatedf-init；
C) to initial effects degree distribution map I under dimension transform domainf-initIt is diffused, generates the disturbance degree distribution map after optimization
If；
D) with the disturbance degree distribution map I after optimizationfTo original depth-map DOriWith the depth map D of pre-smoothedPreIt is merged, it is raw
At the depth map D after optimizationFin；
Step B) described in initial effects degree distribution map If-initIt is specifically defined as：
Wherein, RHIndicate the cavity of prediction, De(p) indicate point p to the distance for estimating empty edge.
2. a kind of depth map fusion method for DIBR preprocessing process as described in claim 1, which is characterized in that step
A operator is estimated in the cavity described in)：
Wherein, RHIndicate that the cavity of prediction, r (x, y) are indicated in original depth-map DOriDepth value at middle coordinate (x, y), DmaxIt is
Setting generates the number of pixels of virtual image parallax maximum value, and α is normalization factor, DwidthIt is the number of pixels of picture traverse,
λHIt is preset threshold factor；If newly synthesized virtual view is left-eye view, i=l, otherwise, i=r.
3. a kind of depth map fusion method for DIBR preprocessing process as described in claim 1, which is characterized in that step
C described in) under dimension transform domain to initial effects degree distribution map If-initIt is diffused specially:Spread function is defined as follows：
If[n]=(1-ad)If-init[n]+adIf[n-1]
Wherein, If-init[n] indicates the pixel value of initial effects degree distribution map lastrow or a row, and a ∈ (0,1) are spread functions
Feedback factor, d indicate dimension transform domain in adjacent sample xnAnd xn-1The distance between.
4. a kind of depth map fusion method for DIBR preprocessing process as claimed in claim 3, which is characterized in that
Adjacent sample x in the dimension transform domainnAnd xn-1The distance between be defined as：
D=ct (xn)-ct(xn-1)
Wherein, ct (u) indicates that dimension transform domain, dimension conversion process are:
Wherein, Itexture(x) texture image of input, I are indicatedt′exture(x) gradient intensity of texture image, σ are indicatedsAnd σrPoint
It is not transmission device space and codomain parameter, for adjusting the influence of propagation, σsValue range is 200~2500, σrValue range is
0.1~10.
5. a kind of depth map fusion method for DIBR preprocessing process as described in claim 1 or claim 3, special
Sign is,
It is described to be diffused as iterative process, and to realize symmetric propagation, if diffusion is according to from left to right in an iteration, from upper
Sequence under is propagated in the picture, then is spread according to sequence spread from right to left, from top to bottom, iteration in next iteration
Number is 2~10 times.
6. a kind of depth map fusion method for DIBR preprocessing process as described in claim 1, which is characterized in that step
D the depth map fusion formula described in) is：
DFin=IfDPre+(1-If)DOri。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611185808.7A CN106791770B (en) | 2016-12-20 | 2016-12-20 | A kind of depth map fusion method suitable for DIBR preprocessing process |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611185808.7A CN106791770B (en) | 2016-12-20 | 2016-12-20 | A kind of depth map fusion method suitable for DIBR preprocessing process |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106791770A CN106791770A (en) | 2017-05-31 |
| CN106791770B true CN106791770B (en) | 2018-08-10 |
Family
ID=58896179
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611185808.7A Expired - Fee Related CN106791770B (en) | 2016-12-20 | 2016-12-20 | A kind of depth map fusion method suitable for DIBR preprocessing process |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106791770B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE112018004891T5 (en) * | 2017-09-01 | 2020-06-10 | Sony Corporation | IMAGE PROCESSING DEVICE, IMAGE PROCESSING PROCESS, PROGRAM AND MOBILE BODY |
| CN107592519A (en) * | 2017-09-30 | 2018-01-16 | 南阳师范学院 | Depth map preprocess method based on directional filtering under a kind of dimension transformation space |
| CN109951705B (en) * | 2019-03-15 | 2020-10-30 | 武汉大学 | Reference frame synthesis method and device for vehicle object coding in surveillance video |
| CN112203074B (en) * | 2020-12-07 | 2021-03-02 | 南京爱奇艺智能科技有限公司 | Camera translation new viewpoint image generation method and system based on two-step iteration |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307312A (en) * | 2011-08-31 | 2012-01-04 | 四川虹微技术有限公司 | Method for performing hole filling on destination image generated by depth-image-based rendering (DIBR) technology |
| CN103248909A (en) * | 2013-05-21 | 2013-08-14 | 清华大学 | Method and system of converting monocular video into stereoscopic video |
| CN104954780A (en) * | 2015-07-01 | 2015-09-30 | 南阳师范学院 | DIBR (depth image-based rendering) virtual image restoration method applicable to high-definition 2D/3D (two-dimensional/three-dimensional) conversion |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2160037A3 (en) * | 2006-06-23 | 2010-11-17 | Imax Corporation | Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition |
| US20120274626A1 (en) * | 2011-04-29 | 2012-11-01 | Himax Media Solutions, Inc. | Stereoscopic Image Generating Apparatus and Method |
| US9471988B2 (en) * | 2011-11-02 | 2016-10-18 | Google Inc. | Depth-map generation for an input image using an example approximate depth-map associated with an example similar image |
| KR101855980B1 (en) * | 2011-12-14 | 2018-05-10 | 연세대학교 산학협력단 | Hole filling method and apparatus |
| CN102595167B (en) * | 2012-03-07 | 2014-06-04 | 中国科学院自动化研究所 | Depth uniformization method and device for 2D/3D video conversion |
| CN104240275A (en) * | 2013-06-13 | 2014-12-24 | 深圳深讯和科技有限公司 | Image repairing method and device |
| CN103957402B (en) * | 2014-05-07 | 2015-10-21 | 四川虹微技术有限公司 | A kind of real-time full HD 2D turns 3D system row read-write sequence method for designing |
| CN104052990B (en) * | 2014-06-30 | 2016-08-24 | 山东大学 | A kind of based on the full-automatic D reconstruction method and apparatus merging Depth cue |
| CN104506872B (en) * | 2014-11-26 | 2017-09-29 | 深圳凯澳斯科技有限公司 | A kind of method and device of converting plane video into stereoscopic video |
| KR101618776B1 (en) * | 2015-02-11 | 2016-05-12 | 광주과학기술원 | Method for Enhancing 3-Dimensional Depth Image |
| CN104751508B (en) * | 2015-03-14 | 2017-07-14 | 杭州道玄影视科技有限公司 | The full-automatic of new view is quickly generated and complementing method in the making of 3D three-dimensional films |
-
2016
- 2016-12-20 CN CN201611185808.7A patent/CN106791770B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307312A (en) * | 2011-08-31 | 2012-01-04 | 四川虹微技术有限公司 | Method for performing hole filling on destination image generated by depth-image-based rendering (DIBR) technology |
| CN103248909A (en) * | 2013-05-21 | 2013-08-14 | 清华大学 | Method and system of converting monocular video into stereoscopic video |
| CN104954780A (en) * | 2015-07-01 | 2015-09-30 | 南阳师范学院 | DIBR (depth image-based rendering) virtual image restoration method applicable to high-definition 2D/3D (two-dimensional/three-dimensional) conversion |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度图像绘制的视图合成;许小艳等;《系统仿真学报》;20111031;第23卷(第10期);第2263-2268页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106791770A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106791770B (en) | A kind of depth map fusion method suitable for DIBR preprocessing process | |
| CN102592275B (en) | Virtual viewpoint rendering method | |
| CN108234985B (en) | Filtering method under dimension transformation space for rendering processing of reverse depth map | |
| CN103581648B (en) | Draw the hole-filling method in new viewpoint | |
| CN103905813B (en) | Based on the DIBR hole-filling method of background extracting and divisional reconstruction | |
| US9578312B2 (en) | Method of integrating binocular stereo video scenes with maintaining time consistency | |
| US20140002595A1 (en) | Apparatus, system and method for foreground biased depth map refinement method for dibr view synthesis | |
| CN104378619B (en) | A kind of hole-filling algorithm rapidly and efficiently based on front and back's scape gradient transition | |
| CN111047709B (en) | Binocular vision naked eye 3D image generation method | |
| CN104065946B (en) | Based on the gap filling method of image sequence | |
| CN102625127A (en) | Optimization method suitable for virtual viewpoint generation of 3D television | |
| CN104954780A (en) | DIBR (depth image-based rendering) virtual image restoration method applicable to high-definition 2D/3D (two-dimensional/three-dimensional) conversion | |
| CN111899295B (en) | Monocular scene depth prediction method based on deep learning | |
| CN104506872B (en) | A kind of method and device of converting plane video into stereoscopic video | |
| Zhang et al. | A unified scheme for super-resolution and depth estimation from asymmetric stereoscopic video | |
| CN106028020B (en) | A kind of virtual perspective image cavity complementing method based on multi-direction prediction | |
| Riechert et al. | Real-time disparity estimation using line-wise hybrid recursive matching and cross-bilateral median up-sampling | |
| Pham et al. | Efficient spatio-temporal local stereo matching using information permeability filtering | |
| Liu et al. | An enhanced depth map based rendering method with directional depth filter and image inpainting | |
| CN104661014B (en) | The gap filling method that space-time combines | |
| CN107592519A (en) | Depth map preprocess method based on directional filtering under a kind of dimension transformation space | |
| CN106780705B (en) | Depth map robust smooth filtering method suitable for DIBR preprocessing process | |
| Yao et al. | Fast and high-quality virtual view synthesis from multi-view plus depth videos | |
| CN103945209A (en) | DIBR method based on block projection | |
| CN110149508A (en) | A kind of array of figure generation and complementing method based on one-dimensional integrated imaging system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180810 Termination date: 20191220 |