CN106993178A - 2D turns the computational methods that 3D video images lift depth plot quality - Google Patents
2D turns the computational methods that 3D video images lift depth plot quality Download PDFInfo
- Publication number
- CN106993178A CN106993178A CN201710216444.2A CN201710216444A CN106993178A CN 106993178 A CN106993178 A CN 106993178A CN 201710216444 A CN201710216444 A CN 201710216444A CN 106993178 A CN106993178 A CN 106993178A
- Authority
- CN
- China
- Prior art keywords
- scene
- contrast
- depth
- depth map
- computational methods
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/128—Adjusting depth or disparity
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
Turn the computational methods that 3D video images lift depth plot quality the invention discloses a kind of 2D, it comprises the following steps：Step one, by estimation and motion vector computation ID figure；Step 2, carries out the coherent degree detection of scene between present frame source images and former frame source images；Step 3, carries out the detection of present frame source images scene naturalness；Step 4, carries out the contrast differentiation threshold calculations in effective depth graph region；Step 5, carries out depth map contrast lifting processing.The computational methods that 2D of the present invention turns 3D video images lifting depth plot quality are based on the coherent degree of scene and scene naturalness algorithm, substantially increase the contrast of depth map, and computing cost is kept within zone of reasonableness, operand relatively reasonable advantage obvious with the lifting of depth map contrast.
Description
Technical field
The present invention relates to a kind of computational methods for lifting depth plot quality, more particularly to a kind of 2D turns 3D video images and carried
Rise the computational methods of depth plot quality.
Background technology
During video image 2D turns 3D, two big links are broadly divided into：The extraction of depth image, 3D disparity maps
Picture is rendered.The extraction of wherein depth image is that video image 2D turns emphasis and difficult point in 3D calculating process, if not to institute
The depth map of extraction carries out strict optimization processing, then the 3D visions that the 3D anaglyphs that thus depth map is rendered are produced
Effect will have a greatly reduced quality.Some traditional depth map optimized treatment methods are mainly carried out at simple LPF to depth map
Reason, although this method can improve the high-frequency noise of depth map to a certain extent, make depth map balanced to a certain extent
It is unified.But it have been found that the gray value contrast of the gray value of moving object and background is not obvious in depth map, contrast
The depth of field gap that small depth map renders moving object and background in the 3D anaglyphs come is just small, and 3D visual effects are paid no attention to very much
Think.The ways and means for optimizing processing to depth map at present is various, but mostly exists among this numerous technology serious
Shortcoming, or contrast lifting is not obvious, or amount of calculation is too big and can not handle in real time, be thus extremely difficult to actual life
The demand of production.
The content of the invention
The technical problems to be solved by the invention are to provide a kind of 2D and turn the calculating that 3D video images lift depth plot quality
Method, it is based on the coherent degree of scene and scene naturalness algorithm, substantially increases the contrast of depth map, and keep calculating to open
Pin is obvious with the lifting of depth map contrast within zone of reasonableness, the relatively reasonable advantage of operand.
The present invention is to solve above-mentioned technical problem by following technical proposals：It is deep that a kind of 2D turns the lifting of 3D video images
The computational methods of plot quality are spent, it comprises the following steps：
Step one, by estimation and motion vector computation ID figure；
Step 2, carries out the coherent degree detection of scene between present frame source images and former frame source images；
Step 3, carries out the detection of present frame source images scene naturalness；
Step 4, carries out the contrast differentiation threshold calculations in effective depth graph region；
Step 5, carries out depth map contrast lifting processing.
Preferably, the coherent degree of the step 2 Scene refers to the content of present frame picture and the company of former frame image content
Pass through degree.
Preferably, the step 3 Scene naturalness refer to judge present frame content whether be natural scene reference mark
It is accurate.
Preferably, in the step 4 calculating in the significance arithmetic region of contrast differentiation threshold calculations including depth map and
The calculating of depth map contrast threshold.
Preferably, contrast threshold of the step 5 based on depth map marks off different critical points, and at these not
Assignment again is carried out with the interval pixel value to depth map of critical point.
The positive effect of the present invention is：2D of the present invention turns the computational methods that 3D video images lift depth plot quality
Based on the coherent degree of scene and scene naturalness algorithm, the contrast of depth map is substantially increased, and keeps computing cost in conjunction
It is obvious with the lifting of depth map contrast within the scope of reason, the relatively reasonable advantage of operand.
Brief description of the drawings
Fig. 1 is the flow chart for the computational methods that 2D of the present invention turns 3D video images lifting depth plot quality.
Fig. 2 is the phase of the coherent degree detection of scene for the computational methods that 2D of the present invention turns 3D video images lifting depth plot quality
Adjacent frame schematic diagram.
Embodiment
Present pre-ferred embodiments are provided below in conjunction with the accompanying drawings, to describe technical scheme in detail.
As shown in Figure 1 to Figure 2,2D of the present invention turn 3D video images lifting depth plot quality computational methods include following step
Suddenly：
Step one, by estimation and motion vector computation ID figure；
Step 2, carries out the coherent degree detection of scene between present frame source images and former frame source images；
Step 3, carries out the detection of present frame source images scene naturalness；
Step 4, carries out the contrast differentiation threshold calculations in effective depth graph region；
Step 5, carries out depth map contrast lifting processing.
The coherent degree of step 2 Scene refers to the content of present frame picture and the coherent degree of former frame image content, we
Usually the video watched all is generally continuous scenic picture, and the difference between frame and frame is often smaller.But
In the scene switching of video lens, the difference between the consecutive frame of switching point is just very big.The coherent degree detection of scene is main
Exactly detect that present frame is to be between one section of continuous frame sequence in transition frames when scene switches.Certainly, this
It is not the setting that absolute this additionally depends on the coherent degree threshold value of scene.
As illustrated in fig. 2, it is assumed that the width of two field picture is W, it is highly H.Represent that coordinate is in present framePlace
Pixel gray value size,Represent that coordinate is in former frameThe gray value size of the pixel at place, and
And If, such as following formula（1）：
（1）So such as following formula（2）：
（2）
Wherein,Threshold value is spent for scene is coherent,Scene pixel point changes number counter, and initial value is set to
0.That is whenever corresponding pixel Y in consecutive frame, U, V component absolute difference and more than the coherent degree threshold value of scene
When, scene change number counter just Jia one automatically, and certain pixel mentioned here includes tri- points of YUV.Corresponding scene
The percentage accounting that coherent pixel accounts for total pixel is following formula（3）：
（3）
The coherent degree of scene is represented, if generallyLess than a certain fixed percentage（70%）,
We are considered as front and rear two field pictures just without correlation, are considered as present frame if greater than the percentage similar to former frame
Degree is larger, and previous frame image also has certain reference value to present image.
Step 3 Scene naturalness refer to judge present frame content whether be natural scene normative reference.It is so-called from
Right scene, which generally refers to image content, does not have the homogeneity of large area, completely black or complete white, if the image of our demands certainly
Except if it is particularly the case.It is generally believed that being to have between picture captured under nature state, the value of adjacent pixel
Size difference, this difference is generally larger than some fixed constant.If the difference between neighbor pixel is solid less than this
Permanent number, and such pixel number reaches certain order of magnitude, and it is not nature picture that we, which are considered as this width scene, this
Sample processing has certain erroneous judgement, but the confidence level of testing result or satisfactory for one very long frame sequence
's.The number for meeting nature picture photo vegetarian refreshments in scene naturalness detection one two field picture of main calculating accounts for the hundred of total pixel number
Divide ratio, often naturalness detection is vital in film video sequence.
Top and bottom are all often the cropping of black in film video picture, and cropping is not wrapped for the scene of center
Containing any image information.If the ratio of the pixel shared by cropping is too high, great deviation can be brought to subsequent algorithm
, this algorithm is according to the significance arithmetic regions of the automatic selected depth figures of the difference of scene naturalness.
We still assume that the width of two field picture is W, are highly H.Represent that coordinate is in present framePlace
The gray value size of pixel, and,,, such as following formula（4）And following formula（5）.
<1>In the horizontal direction, if:
（4）
So（5）
Wherein：For scene naturalness threshold value,Refer to natural pixel counter, its initial value in horizontal scene
It is that vertical coordinate, y are horizontal coordinate for 0, x.The gray value for also having used tri- components of Y, U, V in original image herein participates in meter
Calculate.When the poor absolute value sum of the alternate corresponding Y of two pixels, U, V component is more than the scene naturalness threshold value that we set
When, level counterAutomatically Jia 1.With same method, x, y is allowed to have been circulated always in the range of above-mentioned H, W, just
The number of natural pixel in whole horizontal direction, such as following formula can be obtained（6）, following formula（7）And following formula（8）.
<2>In vertical direction, if:
（6）
So
（7）
Wherein：Refer in vertical scene natural pixel counter, its initial value is that 0, x is that vertical coordinate, y are water
Flat coordinate.The gray value for also having used tri- components of Y, U, V in original image herein participates in calculating.One on principle same level direction
Sample, whenever scene naturalness threshold condition is met, vertical counterAutomatically Jia 1.With same method, x, y are allowed
Circulated always in the range of above-mentioned H, W, it is possible to obtain the number of natural pixel in whole horizontal direction.
So corresponding scene naturalness is：
（8）
Wherein,Refer to scene naturalness,Represent that scene is total in the horizontal and vertical directions
The number of natural pixel,The number of corresponding total pixel.That just as described in algorithm flow
Sample, the effective coverage of the different corresponding depth maps of scene naturalness is different, and artwork can be thus avoided to a certain extent
The influence of middle black cropping.
Contrast differentiation threshold calculations include the calculating and depth map contrast in the significance arithmetic region of depth map in step 4
Spend the calculating of threshold value.
The computational rules in the significance arithmetic region of depth map：When scene naturalness is less than or equal to 50%, depth map
The effective coverage of horizontal direction is：, it is in vertical direction：；When scene naturalness 50% ~ 65% it
Between when, the effective coverage in horizontal direction is：, it is in vertical direction：；When scene naturalness is 65%
When between ~ 80%, the effective coverage in horizontal direction is：, it is in vertical direction：；When scene is natural
Degree is more than or equal to 80%, and entire image is all defaulted as effective coverage.Above-mentioned treatment mechanism is as shown in the table：
Table 1
Naturalness be less than 50% in the case of, give tacit consent in the scene of the width image include a large amount of inactive pixels points, for example film or
The scene confidence level of black cropping in person's television video frame sequence, only center section is higher, thus now we select
Effective coverage on horizontally and vertically is narrow；In the case where naturalness is more than 80%, the institute of the two field picture is given tacit consent to
There is pixel to be all rich in effective information, there is no black cropping etc. in picture, so selection entire image is as effectively calculating area
Domain.
The calculating of depth map contrast threshold is used to obtain the threshold reference of picture contrast boosting algorithm, be object with
Background, close shot region and distant view region further discriminate between the Main Basiss come.The contrast threshold in effective scene areas
Computational methods are：The gray value of pixel all in depth map effective coverage is added up and summed, is then added up and is removed with this again
The average gray value of pixel is obtained with the number of effective pixel points, finally with this average gray value divided by respective regions from
So the degree upper limit just obtains final contrast threshold, such as following formula（9）, following formula（10）, following formula（11）And following formula（12）.
When scene naturalnessWhen：
（9）
When scene naturalness is 50% ~ 65%：
（10）
When scene naturalness is 65% ~ 80%：
（11）
When scene naturalnessWhen：
（12）
Wherein,The contrast threshold of depth map is represented, H, W represent the height and width of depth map respectively,
Represent the gray value size in corresponding coordinate in depth map.Above-mentioned treatment mechanism is as shown in table 2 below：
Table 2
So far, contrast threshold has been obtained.Why above formula is all multiplied by the interval maximum naturalness of corresponding naturalness
Inverse, is to make up the error that the pixel region that we cast out is likely to result in as a compensating factor.
Contrast threshold of the step 5 based on depth map marks off different critical points, and in these different critical points interval to depth
The pixel value for spending figure carries out assignment again.
Specifically processing method is：Last point calculates obtained contrast threshold and is, select respectively、、、、This five points are drawn as five neighbor points for repartitioning depth map gray value region
The interval divided is：、、、、、.Then withFor separation, if depth image vegetarian refreshments gray value existsIn the interval in left side, take respectively
The lower limit of interval separation where the gray value of pixel in subinterval replaces the gray scale of all pixels point in the subinterval
Value, if gray valueIf, former ash angle value keeps constant；If depth image vegetarian refreshments gray value existsRight side
Interval in, the higher limit of the interval separation where the gray value of capture vegetarian refreshments in subinterval replaces institute in the subinterval respectively
There is the gray value of pixel, if gray valueIf, former ash angle value similarly keeps constant.This processing method,
The gray value allowed in different subintervals is in two kinds of different retraction trend, and the gray value of low gray scale interval is less and less, high gray scale
Interval gray value is increasing, and the contrast of such entire depth figure will be significantly improved, main formulas for calculating such as following formula
（13）：
（13）Wherein,、,、The respectively height and width of depth map.
Introduced by this paper contrasts boosting algorithm, this algorithm passes sequentially through the coherent degree detection of scene, the inspection of scene naturalness
The links such as survey, contrast differentiation threshold calculations and contrast lifting processing, carry out calculation process to Depth figure, greatly improve
The contrast effect of moving object and background in depth map, the 3D anaglyph depth stereovision that so renders substantially, depending on
Feel effect protrude, and this algorithm scene link up degree mechanism allow the computing cost of whole process be maintained at rational scope it
It is interior.So, compared to some other depth map contrast optimized treatment methods, the above-mentioned advantage of this paper algorithms can be realized more
Gratifying depth map effect.
Algorithm can be automatically according to the coherent size spent of scene（The coherent degree threshold value of scene）Situation judges that depth map contrast is carried
Rise in processing procedure is the contrast differentiation threshold value from present frame or the contrast differentiation threshold value from former frame.Simply
Say that contrast when the coherent degree of current scene is larger from former frame breaks up threshold value, the coherent degree of scene recalculates current when smaller
The contrast differentiation threshold value of frame, this selectivity processing based on the coherent degree of scene can effectively reduce algorithm to a certain extent
Operand, improve calculation process speed.
Size of the present invention automatically according to scene naturalness（Scene naturalness threshold value）Situation determines to fall into a trap in depth map
Calculate the significance arithmetic region that contrast breaks up threshold value.The selection in significance arithmetic region is most important, because it determines contrast
Whether the result of calculation for breaking up threshold value is accurate.If operating region selects mistake, some non-natural factor interference are allowed so not only
Depth plot quality can not be lifted, the effective information of depth map is reduced on the contrary, allows final 3D anaglyphs to be damaged.Effectively transporting
Calculate the differentiation threshold value that contrast is calculated in region.The process is an adaptive threshold determination process, not only with depth map computing
The size in region is relevant, relevant also with the particular content of depth map（The particularly particular location of moving object in the current frame）.
By selecting suitable contrast to break up threshold value, contrast lifting processing further increases moving object and background in depth map
The drop of gray value, need not artificially be set during processing in specific threshold value, processing procedure it is limited because
Element is small.The coherent degree detection of scene is handled between present frame and former frame, the attribute with time dimension；Scene is certainly
So degree detection is handled between different pixels point inside present frame, the attribute with Spatial Dimension.Time dimension
Combine the big feature that processing is also the present invention with Spatial Dimension.
In summary, 2D of the present invention turn 3D video images lifting depth plot quality computational methods be based on the coherent degree of scene and
Scene naturalness algorithm, substantially increases the contrast of depth map, and keeps computing cost within zone of reasonableness, with depth
Spend the lifting of figure contrast obvious, the relatively reasonable advantage of operand.
Particular embodiments described above, technical problem, technical scheme and beneficial effect to the solution of the present invention are carried out
It is further described, should be understood that the specific embodiment that the foregoing is only of the invention, be not limited to
The present invention, within the spirit and principles of the invention, any modification, equivalent substitution and improvements done etc. should be included in this
Within the protection domain of invention.
Claims (5)
1. a kind of 2D turns the computational methods that 3D video images lift depth plot quality, it is characterised in that it comprises the following steps：
Step one, by estimation and motion vector computation ID figure；
Step 2, carries out the coherent degree detection of scene between present frame source images and former frame source images；
Step 3, carries out the detection of present frame source images scene naturalness；
Step 4, carries out the contrast differentiation threshold calculations in effective depth graph region；
Step 5, carries out depth map contrast lifting processing.
2. 2D as claimed in claim 1 turns the computational methods that 3D video images lift depth plot quality, it is characterised in that described
The coherent degree of step 2 Scene refers to the content of present frame picture and the coherent degree of former frame image content.
3. 2D as claimed in claim 1 turns the computational methods that 3D video images lift depth plot quality, it is characterised in that described
Step 3 Scene naturalness refer to judge present frame content whether be natural scene normative reference.
4. 2D as claimed in claim 1 turns the computational methods that 3D video images lift depth plot quality, it is characterised in that described
The calculating in the significance arithmetic region of contrast differentiation threshold calculations including depth map and depth map contrast threshold in step 4
Calculate.
5. 2D as claimed in claim 1 turns the computational methods that 3D video images lift depth plot quality, it is characterised in that described
Contrast threshold of the step 5 based on depth map marks off different critical points, and in these different critical points interval to depth map
Pixel value carry out assignment again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710216444.2A CN106993178A (en) | 2017-04-05 | 2017-04-05 | 2D turns the computational methods that 3D video images lift depth plot quality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710216444.2A CN106993178A (en) | 2017-04-05 | 2017-04-05 | 2D turns the computational methods that 3D video images lift depth plot quality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106993178A true CN106993178A (en) | 2017-07-28 |
Family
ID=59415250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710216444.2A Pending CN106993178A (en) | 2017-04-05 | 2017-04-05 | 2D turns the computational methods that 3D video images lift depth plot quality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106993178A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102685533A (en) * | 2006-06-23 | 2012-09-19 | 图象公司 | Methods and systems for converting 2d motion pictures for stereoscopic 3d exhibition |
| CN103024419A (en) * | 2012-12-31 | 2013-04-03 | 青岛海信信芯科技有限公司 | Video image processing method and system |
| CN104125446A (en) * | 2013-04-27 | 2014-10-29 | 瑞智半导体（上海）有限公司 | Depth image optimization processing method and device in the 2D-to-3D conversion of video image |
-
2017
- 2017-04-05 CN CN201710216444.2A patent/CN106993178A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102685533A (en) * | 2006-06-23 | 2012-09-19 | 图象公司 | Methods and systems for converting 2d motion pictures for stereoscopic 3d exhibition |
| CN103024419A (en) * | 2012-12-31 | 2013-04-03 | 青岛海信信芯科技有限公司 | Video image processing method and system |
| CN104125446A (en) * | 2013-04-27 | 2014-10-29 | 瑞智半导体（上海）有限公司 | Depth image optimization processing method and device in the 2D-to-3D conversion of video image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9171372B2 (en) | Depth estimation based on global motion | |
| Conze et al. | Objective view synthesis quality assessment | |
| US8854425B2 (en) | Method and apparatus for depth-related information propagation | |
| US9123115B2 (en) | Depth estimation based on global motion and optical flow | |
| US9401039B2 (en) | Image processing device, image processing method, program, and integrated circuit | |
| CN109360235A (en) | A kind of interacting depth estimation method based on light field data | |
| US20130106837A1 (en) | Depth-map generation for an input image using an example approximate depth-map associated with an example similar image | |
| TW201328315A (en) | 2D to 3D video conversion system | |
| CN110268712A (en) | Method and apparatus for handling image attributes figure | |
| Furihata et al. | Novel view synthesis with residual error feedback for FTV | |
| Riechert et al. | Fully automatic stereo-to-multiview conversion in autostereoscopic displays | |
| Pham et al. | Efficient spatio-temporal local stereo matching using information permeability filtering | |
| JP5210416B2 (en) | Stereoscopic image generating apparatus, stereoscopic image generating method, program, and recording medium | |
| CN106993178A (en) | 2D turns the computational methods that 3D video images lift depth plot quality | |
| CN104240179B (en) | 2D images turn figure layer method of adjustment and device in 3D rendering | |
| US9787980B2 (en) | Auxiliary information map upsampling | |
| CN104125446A (en) | Depth image optimization processing method and device in the 2D-to-3D conversion of video image | |
| Wei et al. | Iterative depth recovery for multi-view video synthesis from stereo videos | |
| CN108712642B (en) | Automatic selection method for adding position of three-dimensional subtitle suitable for three-dimensional video | |
| JP6708131B2 (en) | Video processing device, video processing method and program | |
| Wang et al. | Depth filter design by jointly utilizing spatial-temporal depth and texture information | |
| Wei et al. | Video synthesis from stereo videos with iterative depth refinement | |
| JP2013165306A (en) | Stereoscopic video display device | |
| Zhao et al. | 2D to 3D video conversion based on interframe pixel matching | |
| JP2011199382A (en) | Image evaluation apparatus, method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180724 Address after: 200125 Shanghai Pudong New Area free trade pilot area 115, 2, 8, 8-34 rooms. Applicant after: Shanghai Hong Si semiconductor Co., Ltd. Address before: 6 Galileo Road, Pudong New District, Shanghai, 201203 Applicant before: Shanghai Silicon Micro Electronics Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170728 |
|
| RJ01 | Rejection of invention patent application after publication |