CN103581650B - Binocular 3D video turns the method for many orders 3D video - Google Patents
Binocular 3D video turns the method for many orders 3D video Download PDFInfo
- Publication number
- CN103581650B CN103581650B CN201310495786.4A CN201310495786A CN103581650B CN 103581650 B CN103581650 B CN 103581650B CN 201310495786 A CN201310495786 A CN 201310495786A CN 103581650 B CN103581650 B CN 103581650B
- Authority
- CN
- China
- Prior art keywords
- video
- image
- parameter
- algorithm
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The present invention relates to the method that binocular 3D video turns many orders 3D video, comprising: Video Quality Metric is sequence frame by a.; B. rendering parameter is set, present frame figure is split into left and right two-way image and carry out convergent-divergent; C., filtering and Iamge Segmentation parameter are set, filtering and Iamge Segmentation are carried out to image; D. initial parallax value is calculated; Message iterative parameter is set, carries out plane fitting; E. using disparity plane parameter time minimum for the energy function of each cut zone as this region best disparity plane parameter; F. the parallax value of each cut zone is reappraised, and carry out gaussian filtering process; G. repeat n step c ~ f, calculate the motion vector of n two field picture, and carry out interframe smooth operation; H. play up virtual view and be spliced into many palaces table images, being compressed into video.The present invention efficiently solves the problem of bore hole 3D film source scarcity, transferring in multiple views 3D video common 3D format video, effectively reducing 3D cost of manufacture, improve 3D display effect.
Description
Technical field
The present invention relates to Video processing, is the method that binocular 3D video turns many orders 3D video concretely.
Background technology
Along with the development of free 3 D display technology, bore hole 3D Display Technique becomes a popular research topic gradually.The increasing concern with its distinctive visual impact gravitational attraction of bore hole 3D Display Technique, nowadays most bore hole 3D shows solution is all scheme based on multiple views, therefore has special demand to program film source.On the one hand can by modeling software customizing programming film sources such as 3DMAX, MAYA, but can not satisfy the demand far away, on the other hand can by transferring common 3D format video to multiple views 3D video, to meet multiple views 3D scheme to the demand of program film source, turn many objects scheme by binocular and enriched program category greatly, reduce program making cost and the bore hole 3D display effect compared favourably with customization film source can be reached.
Summary of the invention
The invention provides a kind of method that binocular 3D video turns many orders 3D video, to solve the serious deficient problem of existing bore hole 3D film source, and 3D cost of manufacture can be reduced, improve 3D display effect.
Binocular 3D video of the present invention turns the method for many orders 3D video, comprising:
A. extract the frame of video of video to be converted, and the order of pressing video preserves frame of video, such as, use AVS script to change video;
B., the number needing the virtual view played up and the resolution exporting many orders frame of video are set, because incoming frame is generally left-right format, therefore need present frame figure to be split into independent left and right two-way image, and adopt cube interpolation algorithm to carry out convergent-divergent left and right two-way image;
C., after quick bilateral filtering algorithm parameter is set, the filtering of quick bilateral filtering algorithm is carried out respectively to described left and right two-way image; Then image segmentation algorithm parameter is set, Iamge Segmentation is carried out to the left and right road image after quick bilateral filtering;
D. arrange disparity range and the Stereo matching pixel reference windows size of left and right road image, disparity range is [-dispdisp], and disp is positive integer, and the value of disp is an empirical value.SAD algorithm (Sumof absolute differences, a kind of image matching algorithm) being performed in disparity range to left images, and adopts WTA(Winer Take All) policy calculation goes out initial parallax value; Arrange belief propagation algorithm message iterative parameter again, utilize Iamge Segmentation information to carry out plane fitting, each cut zone all can obtain one group of plane parameter;
E. to each cut zone, utilize belief propagation algorithm to calculate energy function summation that this region is adjacent region, and disparity plane parameter corresponding when getting minimum using each region energy function is as the best disparity plane parameter in this region, concrete grammar is in Nanjing Aero-Space University's Master's thesis " the belief propagation Study on Stereo Matching Algorithm based on Iamge Segmentation ", author: Li Binbin, has a detailed description in a literary composition;
F. the best disparity plane parameter described in each cut zone step e is reappraised all parallax value in cut zone, and gaussian filtering process is carried out to the disparity map after reappraising;
G. repeat n step c ~ f, obtain the original left right wing video frame images that continuous n frame disparity map is corresponding, utilize feature point detection and Feature Correspondence Algorithm to calculate the motion vector of described n two field picture; According to the motion vector calculated, the n frame anaglyph calculated is carried out interframe smooth operation and preserved; Wherein n is natural number, and general value is 6 ~ 10;
H. use original left right wing image and combine corresponding anaglyph, formula is played up according to multiple views, the number of the virtual view that the needs arranged in integrating step b are played up, carry out playing up of virtual view, then rule corresponding according to the number of viewpoint for the multiway images after playing up is spliced into many palaces table images, as: 4 viewpoints are spliced into 4 palace lattice, and 8 viewpoints are spliced into 9 palace lattice.Finally many palaces lattice sequence frame is compressed into video, can plays in bore hole 3D player.
Further, the number of the virtual view played up is needed to be 4 viewpoints or 8 viewpoints in step b; The resolution exporting many orders frame of video is 1920 × 1080; Left and right two-way image scaling is to 960 × 540.
Concrete, the quick bilateral filtering algorithm parameter arranged in step c comprises: filter window size, color component value and location components value; Image segmentation algorithm parameter comprises: the pixel number of color threshold value, radius threshold and Minimum Area.
Preferably, before the filtering of quick bilateral filtering algorithm is carried out respectively to described left and right two-way image, transfer the color space of left and right road image to Lab colour model from RGB, transfer the color space of filter result to RGB from Lab colour model after filtering and preserve filter result.Better filter effect will be obtained through color space conversion.
Concrete, in steps d, belief propagation algorithm message iterative parameter comprises: the scale factor after discontinuous penalty factor, message iteration and the number of times after message iteration.
Further, in steps d before arranging belief propagation algorithm message iterative parameter, also will apply left and right consistency detection rule and detect initial parallax value, what meet this rule is labeled as credible parallax, and what do not meet this rule is labeled as insincere parallax; When carrying out plane fitting, Iamge Segmentation information is utilized to carry out plane fitting in conjunction with parallax value being labeled as the pixel of credible parallax all in this cut zone.
Further, in step h after virtual view is played up, if play up rear image in the disparity range of left and right road, carry out hole region fill up by fill a vacancy the mutually method in hole of left and right road, if image is outside disparity range, use image mending algorithm to fill up hole region, and then carry out described many palaces table images splicing.
Binocular 3D video of the present invention turns the method for many orders 3D video, effectively solves the problem of existing bore hole 3D film source scarcity, transferring in multiple views 3D video to common binocular 3D format video, can effectively reduce 3D cost of manufacture, and improve 3D display effect.
Below in conjunction with the embodiment of embodiment, foregoing of the present invention is described in further detail again.But this should be interpreted as that the scope of the above-mentioned theme of the present invention is only limitted to following example.Without departing from the idea case in the present invention described above, the various replacement made according to ordinary skill knowledge and customary means or change, all should comprise within the scope of the invention.Embodiment
Binocular 3D video of the present invention turns the method for many orders 3D video, comprising:
A. use AVS script to be sequence frame by Video Quality Metric to be converted, and the order of pressing video preserve frame of video;
B., the number needing the virtual view played up is set, is generally 4 viewpoints or 8 viewpoints, then the resolution arranging output many orders frame of video is 1920 × 1080.Because incoming frame is generally left-right format, therefore need present frame figure to be split into independent left and right two-way image, and adopt cube interpolation algorithm to zoom to 960 × 540 left and right two-way image;
C., quick bilateral filtering algorithm parameter is set, comprises: the sigma value of filter window size, color component and the sigma value (sigma is a parameter, is a technical term in bilateral filtering) of location components.Transfer the color space of left and right road image to Lab colour model from RGB, the filtering of quick bilateral filtering algorithm is carried out respectively to described left and right two-way image, transfer the color space of filter result to RGB from Lab colour model after filtering and preserve filter result.Better filter effect will be obtained through color space conversion.Then image segmentation algorithm parameter is set, comprises: the pixel number of color threshold value, radius threshold and Minimum Area, Iamge Segmentation is carried out to the left and right road image after quick bilateral filtering;
D. arrange disparity range and the Stereo matching pixel reference windows size of left and right road image, disparity range is [-dispdisp], and disp is positive integer, and the value of disp is an empirical value.SAD algorithm (Sumof absolute differences, a kind of image matching algorithm) is performed in disparity range to left images, and adopts the algorithm of WTA to calculate initial parallax value; The computing formula of SAD algorithm is:
, wherein left is left image, and right is right image, and r is window size, and d is current parallax.WTA(Winer Take All) strategy is in all SAD (X, Y, d), choose the parallax value of d value corresponding to minimum value as current pixel point.
Application left and right consistency detection rule detects initial parallax value: the parallax value dLeft (x of current point, y)=d, if dReft is (x+d, y)=dLeft (x, y)=d, then meet disparity consistency constraint, this parallax value is labeled as credible parallax value, otherwise is labeled as insincere parallax value; Belief propagation algorithm message iterative parameter is set again, comprises: the scale factor after discontinuous penalty factor, message iteration and the number of times after message iteration.Utilize Iamge Segmentation information to carry out plane fitting in conjunction with parallax value being labeled as the pixel of credible parallax all in this cut zone, each cut zone all can obtain one group of plane parameter.If plane equation is d (x, y)=ax+by+c, wherein a, b and c are disparity plane parameter, x and y is coordinate figure, respectively according to the situation of change of parallax in x direction and y direction, utilize Voting Algorithm to simulate most possible a, b value, combine according to existing depth value a, b value simulated and can calculate each c value of n, wherein n is the number of pixel, again utilizes Voting Algorithm to find out most possible c value.
E. to each cut zone, utilize belief propagation algorithm to calculate energy function summation that this region is adjacent region, and disparity plane parameter corresponding when getting minimum using each region energy function is as the best disparity plane parameter in this region.The global energy function of definition is
wherein d represents that the parallax of entire image distributes, four neighborhood point sets of all pixels in N presentation video, d
prepresent the parallax value that some p distributes, level and smooth item V (d
p, d
q) represent that two neighbor pixel p and some q distribute parallax d
pand d
ptime parallax discontinuity punishment, the set of pixel in p presentation video, data item D
p(d
p) represent that p point parallax is d
ptime non-similarity estimate.Make global energy minimum if certain parallax distributes, then this parallax distributes the final parallax being image.Specifically refer to Nanjing Aero-Space University's Master's thesis " the belief propagation Study on Stereo Matching Algorithm based on Iamge Segmentation ", author: Li Binbin;
F. the best disparity plane parameter of each cut zone is reappraised parallax value all in cut zone, according to the result of step e, the plane parameter that each region will have after an optimization, the parallax value of all pixels in region is now recalculated according to formula d (x, y)=ax+by+c.X and y is coordinate figure.Reappraise all parallax value in cut zone the described best disparity plane parameter of each cut zone, and carry out gaussian filtering process to the disparity map after reappraising, the computational methods of Gaussian kernel are:
wherein, δ is standard deviation, x, y be respectively current point distance center point in x direction, the distance in y direction;
G. repeat n step c ~ f, obtain the original left right wing video frame images that continuous n frame disparity map is corresponding, utilize feature point detection and Feature Correspondence Algorithm to calculate the motion vector of described n two field picture, the value of general n is 6 ~ 10.Concrete grammar is: the harris angle point first detecting consecutive frame image, calculates the feature interpretation vector of each characteristic point, then utilizes characteristic point description vectors to carry out the coupling of characteristic point, calculated the motion vector of consecutive frame by the matching relationship of characteristic point.According to the motion vector calculated, image is carried out the operations such as corresponding translation, convergent-divergent according to motion vector, then the residual information of consecutive frame is asked for, when residual information is less than certain threshold value threash, adopt weighting smoothing method that the n frame anaglyph calculated is carried out to interframe smooth operation and preserved;
H. use original left right wing image and combine corresponding anaglyph, playing up formula according to multiple views, the number of the virtual view that the needs arranged in integrating step b are played up, carry out playing up of virtual view, multiple views plays up the translation that essence is pixel, and the computing formula of translation vector is: Shift (x, y)=dScale*d (x, y), wherein x, y are pixel coordinate, and dScale is shift factor, with the resolution of depth map, mode of playing up is correlated with.If play up rear image in the disparity range of left and right road, carry out hole region fill up by fill a vacancy the mutually method in hole of left and right road, if image is outside disparity range, use image mending algorithm to fill up hole region.Wherein hole of filling a vacancy mutually, left and right road first navigates to cavity, then utilizes the information before and after cavity in another road image, find the region of the hole region in the image of most possible coupling current road.And image mending algorithm only utilizes the current road existing information of image to repair out hole region.Then rule corresponding according to the number of viewpoint for the multiway images after playing up is spliced into many palaces table images, as: 4 viewpoints are spliced into 4 palace lattice, and 8 viewpoints are spliced into 9 palace lattice, put in order as from left to right, from top to bottom.Finally many palaces lattice sequence frame is compressed into video, can plays in bore hole 3D player.
Claims (7)
1. binocular 3D video turns the method for many orders 3D video, and its feature comprises:
A. extract the frame of video of video to be converted, and the order of pressing video preserves frame of video;
B., the number needing the virtual view played up and the resolution exporting many orders frame of video are set, present frame figure are split into independent left and right two-way image, and adopt cube interpolation algorithm to carry out convergent-divergent left and right two-way image;
C., after quick bilateral filtering algorithm parameter is set, the filtering of quick bilateral filtering algorithm is carried out respectively to described left and right two-way image; Then image segmentation algorithm parameter is set, Iamge Segmentation is carried out to the left and right road image after quick bilateral filtering;
D., disparity range and the Stereo matching pixel reference windows size of left and right road image are set, SAD algorithm are performed in disparity range to left images, and adopts WTA policy calculation to go out initial parallax value; Arrange belief propagation algorithm message iterative parameter again, utilize Iamge Segmentation information to carry out plane fitting, each cut zone all can obtain one group of plane parameter;
E. to each cut zone, utilize belief propagation algorithm to calculate energy function summation that this region is adjacent region, and disparity plane parameter corresponding when getting minimum using each region energy function is as the best disparity plane parameter in this region;
F. the best disparity plane parameter described in each cut zone step e is reappraised all parallax value in cut zone, and gaussian filtering process is carried out to the disparity map after reappraising;
G. repeat n step c ~ f, obtain the original left right wing video frame images that continuous n frame disparity map is corresponding, utilize feature point detection and Feature Correspondence Algorithm to calculate the motion vector of original left right wing video frame images corresponding to described continuous n frame disparity map; According to the motion vector calculated, the n frame anaglyph calculated is carried out interframe smooth operation and preserved; Wherein n is natural number;
H. use original left right wing image and combine corresponding anaglyph, formula is played up according to multiple views, the number of the virtual view that the needs arranged in integrating step b are played up, carry out playing up of virtual view, then rule corresponding according to the number of viewpoint for the multiway images after playing up is spliced into many palaces table images, finally many palaces lattice sequence frame is compressed into video.
2. binocular 3D video as claimed in claim 1 turns the method for many orders 3D video, it is characterized by: need the number of the virtual view played up to be 4 viewpoints or 8 viewpoints in step b; The resolution exporting many orders frame of video is 1920 × 1080; Left and right two-way image scaling is to 960 × 540.
3. binocular 3D video as claimed in claim 1 turns the method for many orders 3D video, it is characterized by: the quick bilateral filtering algorithm parameter arranged in step c comprises: filter window size, color component value and location components value; Image segmentation algorithm parameter comprises: the pixel number of color threshold value, radius threshold and Minimum Area.
4. binocular 3D video as claimed in claim 1 turns the method for many orders 3D video, it is characterized by: before the filtering of quick bilateral filtering algorithm is carried out respectively to described left and right two-way image, transfer the color space of left and right road image to Lab colour model from RGB, transfer the color space of filter result to RGB from Lab colour model after filtering and preserve filter result.
5. binocular 3D video as claimed in claim 1 turns the method for many orders 3D video, it is characterized by: in steps d, belief propagation algorithm message iterative parameter comprises: the scale factor after discontinuous penalty factor, message iteration and the number of times after message iteration.
6. binocular 3D video as claimed in claim 1 turns the method for many orders 3D video, it is characterized by: in steps d before belief propagation algorithm message iterative parameter is set, also to apply left and right consistency detection rule to detect initial parallax value, what meet this rule is labeled as credible parallax, and what do not meet this rule is labeled as insincere parallax; When carrying out plane fitting, Iamge Segmentation information is utilized to carry out plane fitting in conjunction with parallax value being labeled as the pixel of credible parallax all in this cut zone.
7. binocular 3D video as claimed in claim 1 turns the method for many orders 3D video, it is characterized by: in step h after virtual view is played up, if play up rear image in the disparity range of left and right road, carry out hole region fill up by fill a vacancy the mutually method in hole of left and right road, if image is outside disparity range, use image mending algorithm to fill up hole region, and then carry out described many palaces table images splicing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310495786.4A CN103581650B (en) | 2013-10-21 | 2013-10-21 | Binocular 3D video turns the method for many orders 3D video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310495786.4A CN103581650B (en) | 2013-10-21 | 2013-10-21 | Binocular 3D video turns the method for many orders 3D video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103581650A CN103581650A (en) | 2014-02-12 |
| CN103581650B true CN103581650B (en) | 2015-08-19 |
Family
ID=50052432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310495786.4A Active CN103581650B (en) | 2013-10-21 | 2013-10-21 | Binocular 3D video turns the method for many orders 3D video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103581650B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104469440B (en) * | 2014-04-16 | 2018-10-30 | 成都理想境界科技有限公司 | Video broadcasting method, video player and corresponding playback equipment |
| CN103945208B (en) * | 2014-04-24 | 2015-10-28 | 西安交通大学 | A kind of parallel synchronous zooming engine for multiple views bore hole 3D display and method |
| CN104378616B (en) * | 2014-09-03 | 2017-06-16 | 王元庆 | A kind of flush system multi-view image frame packaging structure and building method |
| CN104717514B (en) * | 2015-02-04 | 2018-03-02 | 四川长虹电器股份有限公司 | Rendering multi-viewpoint images system and method |
| CN108377376B (en) * | 2016-11-04 | 2021-01-26 | 宁波舜宇光电信息有限公司 | Parallax calculation method, double-camera module and electronic equipment |
| WO2018082604A1 (en) * | 2016-11-04 | 2018-05-11 | 宁波舜宇光电信息有限公司 | Parallax and distance parameter calculation methods, dual camera module and electronic device |
| CN107493465B (en) * | 2017-09-18 | 2019-06-07 | 郑州轻工业学院 | A kind of virtual multi-view point video generation method |
| CN109688397B (en) * | 2017-10-18 | 2021-10-22 | 上海质尊文化传媒发展有限公司 | Method for converting 2D (two-dimensional) video into 3D video |
| CN108492326B (en) * | 2018-01-31 | 2021-11-23 | 北京大学深圳研究生院 | Stereo matching method and system with gradually refined resolution ratio from low to high |
| CN110310317A (en) * | 2019-06-28 | 2019-10-08 | 西北工业大学 | A method of the monocular vision scene depth estimation based on deep learning |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101483788B (en) * | 2009-01-20 | 2011-03-23 | 清华大学 | Method and apparatus for converting plane video into tridimensional video |
| CN102752616A (en) * | 2012-06-20 | 2012-10-24 | 四川长虹电器股份有限公司 | Method for converting double-view three-dimensional video to multi-view three-dimensional video |
| CN102831601A (en) * | 2012-07-26 | 2012-12-19 | 中北大学 | Three-dimensional matching method based on union similarity measure and self-adaptive support weighting |
| CN103269435A (en) * | 2013-04-19 | 2013-08-28 | 四川长虹电器股份有限公司 | Binocular to multi-view virtual viewpoint synthetic method |
-
2013
- 2013-10-21 CN CN201310495786.4A patent/CN103581650B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN103581650A (en) | 2014-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103581650B (en) | Binocular 3D video turns the method for many orders 3D video | |
| CN100576934C (en) | Virtual visual point synthesizing method based on the degree of depth and block information | |
| CN101742349B (en) | Method for expressing three-dimensional scenes and television system thereof | |
| CN106504190B (en) | A kind of three-dimensional video-frequency generation method based on 3D convolutional neural networks | |
| CN102254348B (en) | Virtual viewpoint mapping method based o adaptive disparity estimation | |
| CN102026013B (en) | Stereo video matching method based on affine transformation | |
| CN102263979B (en) | Depth map generation method and device for plane video three-dimensional conversion | |
| CN103702098B (en) | Three viewpoint three-dimensional video-frequency depth extraction methods of constraint are combined in a kind of time-space domain | |
| KR20110090958A (en) | Generation of occlusion data for image properties | |
| CN103081476A (en) | Method and device for converting three-dimensional image using depth map information | |
| CN104639933A (en) | Real-time acquisition method and real-time acquisition system for depth maps of three-dimensional views | |
| CN104506872B (en) | A kind of method and device of converting plane video into stereoscopic video | |
| CN103269438A (en) | Method for drawing depth image on the basis of 3D video and free-viewpoint television | |
| US10127714B1 (en) | Spherical three-dimensional video rendering for virtual reality | |
| CN110113593A (en) | Wide baseline multi-view point video synthetic method based on convolutional neural networks | |
| CN112019828B (en) | Method for converting 2D (two-dimensional) video into 3D video | |
| CN103716615B (en) | 2D video three-dimensional method based on sample learning and depth image transmission | |
| Zhang et al. | Visual pertinent 2D-to-3D video conversion by multi-cue fusion | |
| CN103679739A (en) | Virtual view generating method based on shielding region detection | |
| CN104159098B (en) | The translucent edge extracting method of time domain consistence of a kind of video | |
| WO2008152607A1 (en) | Method, apparatus, system and computer program product for depth-related information propagation | |
| CN104717514A (en) | Multi-viewpoint image rendering system and method | |
| Akin et al. | Real-time free viewpoint synthesis using three-camera disparity estimation hardware | |
| CN103247027B (en) | Image processing method and electric terminal | |
| CN103379350A (en) | Virtual viewpoint image post-processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |