CN101483788B - Method and apparatus for converting plane video into tridimensional video - Google Patents
Method and apparatus for converting plane video into tridimensional video Download PDFInfo
- Publication number
- CN101483788B CN101483788B CN200910077228XA CN200910077228A CN101483788B CN 101483788 B CN101483788 B CN 101483788B CN 200910077228X A CN200910077228X A CN 200910077228XA CN 200910077228 A CN200910077228 A CN 200910077228A CN 101483788 B CN101483788 B CN 101483788B
- Authority
- CN
- China
- Prior art keywords
- video
- sequence
- image sequence
- frame image
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000009825 accumulation Methods 0.000 claims abstract description 55
- 230000009466 transformation Effects 0.000 claims abstract description 10
- 238000009499 grossing Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 abstract description 16
- 238000009877 rendering Methods 0.000 abstract description 2
- 239000007787 solid Substances 0.000 abstract 6
- 230000004069 differentiation Effects 0.000 abstract 1
- 238000007792 addition Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
This invention discloses a method and a device for converting plane video into solid video, belonging to multimedia technology field, wherein the method comprises: decoding plane video sequence to obtain frame image sequence of plane video; carrying out grey level transformation, differentiation on the frame image sequence of plane video to obtain frame difference image sequence; carrying out frame difference adding accumulation on the frame image sequence to obtain depth image sequence; based on plane video frame image sequence and depth image sequence, generating solid video frame image sequence with depth information rendering algorithm; decoding the solid video frame image sequence to obtain solid video sequence. During process of converting plane video into solid video, this invention can effectively save manpower and time to enhance efficiency of converting plane video to solid video; meanwhile, hard drive space can be effectively saved without storing temporary results therein.
Description
Technical field
The present invention relates to Computer Multimedia Technology, particularly a kind of method and apparatus of converting plane video into stereoscopic video.
Background technology
Development along with Computer Multimedia Technology, three-dimensional video-frequency is more and more promoted and is favored, so-called three-dimensional video-frequency, be exactly binocular or multi-channel video sequence, promptly comprise the above video sequence of two-way, and have parallax between the corresponding frame of each video sequence, can provide left view sequence and the right view sequence that has parallax information for the beholder, this also is to produce relief reason as if on the spot in person when watching three-dimensional video-frequency; But the difficulty of three-dimensional video-frequency is big, equipment cost is higher owing to directly obtain, and existing planar video content quantity is huge, so the converting plane video into stereoscopic video technology has been obtained increasing attention.
The process of converting plane video into stereoscopic video, one is converted to the planar video frame image sequence with the planar video sequence exactly, the depth information that extracts planar video frame image sequence correspondence then obtains corresponding depth map sequence, (Depth Image Based Rendering DIBR) plays up the process that generates binocular or multichannel stereoscopic video sequence with planar video frame image sequence and depth map sequence to utilize depth information to play up algorithm.Prior art mainly is to realize planar video is converted to three-dimensional video-frequency by man-machine interaction, detailed process is as follows: at first the planar video sequence is changed into uncompressed format and be kept in the hard disk, utilize the program of opencv that uncompressed format planar video sequence is become the planar video two field picture of a frame frame then and be kept in the hard disk, utilization obtains the depth map of every frame planar video two field picture by hand and is kept in the hard disk again, then utilize depth information to play up algorithm planar video frame image sequence and depth map sequence compound stereoscopic video frame images sequence and be kept in the hard disk, at last with the three-dimensional video-frequency frame sequence output of the program of opencv with the synthetic uncompressed format of three-dimensional video-frequency frame image sequence.
In realizing process of the present invention, the inventor finds that there is following shortcoming at least in prior art:
In the process of prior art converting plane video into stereoscopic video, each step all needs artificial participation, and artificial participation height, workload are big, and the efficient of converting plane video into stereoscopic video is low; And the intermediate object program of each step all needs to be kept in the hard disk, and the video of uncompressed format and image file are all very big, so can take a large amount of memory space of hard disk.
Summary of the invention
In order to reduce the efficient of artificial participation, raising converting plane video into stereoscopic video, the embodiment of the invention provides a kind of method and apparatus of converting plane video into stereoscopic video.Described technical scheme is as follows:
On the one hand, the embodiment of the invention provides a kind of method of converting plane video into stereoscopic video, and described method comprises:
Open the planar video sequence, obtain the information of described planar video sequence;
Call the Video Decoder of ffmpeg the information of described planar video sequence is decoded, obtain described planar video frame image sequence;
Described planar video frame image sequence is carried out greyscale transformation, poor, obtain the frame difference image sequence;
According to described frame difference image sequence, described planar video frame image sequence is divided into the relevant a plurality of subsequences of content;
The frame difference image sequence of each subsequence correspondence in described a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in described a plurality of subsequence;
With the accumulation frame difference image of each subsequence in described a plurality of subsequences depth map, thereby obtain the described depth map sequence of described planar video sequence correspondence as all planar video sequences that each subsequence comprised in described a plurality of subsequences;
Based on described planar video frame image sequence and described depth map sequence, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence;
The parameter of described stereoscopic video sequence is set;
Read described three-dimensional video-frequency frame image sequence, obtain the information of described three-dimensional video-frequency frame image sequence;
Call the video encoder of ffmpeg the information of described three-dimensional video-frequency frame image sequence is encoded, obtain described stereoscopic video sequence.
On the other hand, the embodiment of the invention provides a kind of device of converting plane video into stereoscopic video, and described device comprises:
Planar video frame image sequence acquisition module, specifically comprise planar video sequence information acquiring unit and planar video frame image sequence acquiring unit, wherein, described planar video sequence information acquiring unit, be used to open the planar video sequence, obtain the information of described planar video sequence; Described planar video frame image sequence acquiring unit, be used for after described planar video sequence information acquiring unit obtains the information of described planar video sequence, call the Video Decoder of ffmpeg the information of described planar video sequence is decoded, obtain described planar video frame image sequence;
Frame difference image sequence acquisition module is used for after described planar video frame image sequence acquisition module obtains described planar video frame image sequence described planar video frame image sequence being carried out greyscale transformation, poor, obtains the frame difference image sequence;
The depth map sequence acquisition module, specifically comprise subsequence division unit, accumulation frame difference image acquiring unit and depth map sequence acquiring unit, wherein, described subsequence division unit, be used for after described frame difference image sequence acquisition module obtains described frame difference image sequence, according to described frame difference image sequence, the described planar video frame image sequence that described planar video frame image sequence acquiring unit is obtained is divided into the relevant a plurality of subsequences of content; Described accumulation frame difference image acquiring unit, be used for after described subsequence division unit marks off described a plurality of subsequence, the frame difference image sequence of each subsequence correspondence in described a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in described a plurality of subsequence; Described depth map sequence acquiring unit, be used for after described accumulation frame difference image acquiring unit obtains described accumulation frame difference image, with the accumulation frame difference image of each subsequence in described a plurality of subsequences depth map, thereby obtain the described depth map sequence of described planar video sequence correspondence as all planar video sequences that each subsequence comprised in described a plurality of subsequences;
Three-dimensional video-frequency frame image sequence acquisition module, be used for after described depth map sequence acquisition module obtains described depth map sequence, based on described planar video frame image sequence and described depth map sequence, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence;
The stereoscopic video sequence acquisition module, specifically comprise module, stereo video frame image sequence information acquiring unit and stereoscopic video sequence acquiring unit are set, wherein, the described module that is provided with, be used for after described three-dimensional video-frequency frame image sequence acquisition module obtains described three-dimensional video-frequency frame image sequence, the parameter of described stereoscopic video sequence being set; Described stereo video frame image sequence information acquiring unit, be used for described the parameter that module is provided with described stereoscopic video sequence is set after, read described three-dimensional video-frequency frame image sequence, obtain the information of described three-dimensional video-frequency frame image sequence; Described stereoscopic video sequence acquiring unit, be used for after described stereo video frame image sequence information acquiring unit obtains the information of described three-dimensional video-frequency frame image sequence, call the video encoder of ffmpeg the information of described three-dimensional video-frequency frame image sequence is encoded, obtain stereoscopic video sequence.
The beneficial effect of the technical scheme that the embodiment of the invention provides is:
Planar video is being converted in the process of three-dimensional video-frequency, is saving a large amount of artificial participations, saving a large amount of manpowers and time, improving the efficient of converting plane video into stereoscopic video; And, do not need intermediate object program is kept in the hard disk, saved the memory space of hard disk.
Description of drawings
Fig. 1 is the method flow diagram of a kind of converting plane video into stereoscopic video of providing of the embodiment of the invention 1;
Fig. 2 is the method flow diagram of a kind of converting plane video into stereoscopic video of providing of the embodiment of the invention 2;
Fig. 3 is the apparatus structure schematic diagram of a kind of converting plane video into stereoscopic video of providing of the embodiment of the invention 3;
Fig. 4 is the apparatus structure schematic diagram of the another kind of converting plane video into stereoscopic video that provides of the embodiment of the invention 3.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, embodiment of the present invention is described further in detail below in conjunction with accompanying drawing.
The embodiment of the invention proposes ffmpeg is applied in the process of converting plane video into stereoscopic video.Wherein, ffmpeg is that collection audio frequency and video are recorded, audio frequency and video are changed and the audio/video coding decoding function is the complete scheme of increasing income of one, ffmpeg almost can support the audio frequency and video of current all forms, ffmpeg mainly comprises libavcodec storehouse and libavformat storehouse, comprise all ffmpeg audio/video coders and ffmpeg audio/video decoder in the libavcodec storehouse, comprise the resolver and the generator of all common audio frequency and video forms in the libavformat storehouse.The embodiment of the invention is by being applied to ffmpeg in the process of converting plane video into stereoscopic video, and realized planar video is automatically converted to three-dimensional video-frequency in conjunction with various technological means, the method for describing the described a kind of converting plane video into stereoscopic video of the embodiment of the invention below in conjunction with specific embodiments in detail is how to realize automatically planar video being converted to three-dimensional video-frequency.
Embodiment 1
Referring to Fig. 1, the embodiment of the invention provides a kind of method of converting plane video into stereoscopic video, specifically comprises:
101: the planar video sequence is decoded, obtain the planar video frame image sequence;
102: the planar video frame image sequence is carried out greyscale transformation, poor, obtain the frame difference image sequence;
103: the frame difference image sequence is carried out frame difference addition accumulation, obtain depth map sequence;
104:, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence based on planar video frame image sequence and depth map sequence;
105: the stereoscopic video frame image sequence is encoded, and obtains stereoscopic video sequence.
Wherein, the planar video sequence is decoded, obtains the planar video frame image sequence, specifically comprise:
Open the planar video sequence, obtain the information of planar video sequence;
Call the Video Decoder of ffmpeg the information of planar video sequence is decoded, obtain the planar video frame image sequence.
Wherein, the frame difference image sequence is carried out frame difference addition accumulation, obtains depth map sequence, specifically comprise:
According to the frame difference image sequence, the planar video frame image sequence is divided into the relevant a plurality of subsequences of content;
The frame difference image sequence of each subsequence correspondence in a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in a plurality of subsequences;
With the accumulation frame difference image of each subsequence in a plurality of subsequences depth map, thereby obtain the depth map sequence of planar video sequence correspondence as all planar video sequences that each subsequence comprised in a plurality of subsequences.
Wherein, the stereoscopic video frame image sequence is encoded, and obtains stereoscopic video sequence, specifically comprises:
The parameter of stereoscopic video sequence is set;
Read the three-dimensional video-frequency frame image sequence, obtain the information of stereoscopic frequency frame image sequence;
Call the information of the video encoder stereoscopic video frame image sequence of ffmpeg and encode, obtain stereoscopic video sequence.
Further, the frame difference image sequence is carried out frame difference addition accumulation, obtains after the depth map sequence,, utilize depth information to play up algorithm and generate before the three-dimensional video-frequency frame image sequence, also comprise based on planar video frame image sequence and depth map sequence:
Depth map sequence is carried out smoothing processing.
The described method of present embodiment is being converted to planar video in the process of three-dimensional video-frequency, has save a large amount of artificial participations, has saved a large amount of manpowers and time, has improved the efficient of converting plane video into stereoscopic video; And, do not need intermediate object program is kept in the hard disk, saved the memory space of hard disk; And, the accumulation frame difference image of subsequence as depth map, is obtained depth map and compares with manual, reduced workload, improved obtain depth map efficient; In addition,, eliminated the rough discontinuous part of noise and depth map edge of depth map inside, reduced the influence of accumulation frame mistake difference, thereby avoided the antemarginal shake of generation stereoscopic video sequence by depth map sequence is carried out smoothing processing.
Embodiment 2
Referring to Fig. 2, the embodiment of the invention provides a kind of method of converting plane video into stereoscopic video, specifically comprises:
201: the planar video sequence is decoded, obtain the planar video frame image sequence.
Wherein, the planar video sequence is decoded, obtain the planar video frame image sequence and comprise: open the planar video sequence that will be converted to stereoscopic video sequence, obtain the planar video sequence information; Call the Video Decoder of ffmpeg the planar video sequence information is decoded, obtain the planar video frame image sequence; Wherein the planar video sequence information mainly comprises: the sequence number of the width of the frame number of planar video sequence, every frame planar video, the height of every frame planar video and every frame planar video etc.
Need to prove that may there be the planar video that cannot decode that has damaged in practical application midplane video sequence, so before the Video Decoder that calls ffmpeg is decoded to the planar video sequence information, also need to judge whether and can decode according to the planar video sequence information that obtains, and if could the Video Decoder that would call ffmpeg the planar video sequence information is decoded; Otherwise, close this planar video.And need to prove, ffmpeg provides the interface of Video Decoder and obtains the function of Video Decoder, the function that provides of ffmpeg just can get access to the Video Decoder of ffmpeg by reference, just can realize the planar video sequence information is decoded then, obtain the planar video frame image sequence by the Video Decoder that calls ffmpeg; When the function that wherein quoting ffmpeg provides gets access to the Video Decoder of ffmpeg, need use the various parameters that comprise in the planar video sequence information, if planar video damages, some parameter in the planar video sequence information just may obtain less than, when correspondingly just the function that provides of ffmpeg gets access to the Video Decoder of ffmpeg by reference, cause the Video Decoder that can't call ffmpeg that this planar video sequence information is decoded, so the planar video sequence information that above-mentioned basis is obtained judges whether and can decode, concrete just according in the planar video sequence information that obtains, the function that judging whether to quote ffmpeg provides gets access to the Video Decoder of ffmpeg, and if could the Video Decoder that would call ffmpeg the planar video sequence information is decoded; Otherwise, close this planar video.And, after obtaining the planar video frame image sequence, the planar video frame image sequence can be kept in the internal memory, wait for follow-up processing.
Further need to prove,,, need not in advance the planar video sequence to be decompressed so, also the planar video sequential decoding can be become the planar video frame image sequence if the planar video sequence is a compressed format because ffmpeg supports all format videos; With needing artificial elder generation before each conversion in the prior art planar video of compressed format is decompressed, be converted into the planar video frame image sequence again and compare, can save manually, save a large amount of time and hard-disc storage space.
Need to prove in addition, when the Video Decoder that calls ffmpeg becomes the planar video frame image sequence with the planar video sequential decoding, can select start frame and frame number in advance, the Video Decoder that utilizes ffmpeg to provide then becomes the planar video frame image sequence with the planar video sequential decoding of choosing, that is to say can be according to the actual needs, selection will be converted to the planar video sequence of stereoscopic video sequence, for example: the planar video sequence comprises 100 frames, the part of 50-80 frame need be converted to stereoscopic video sequence now, so when the Video Decoder that utilizes ffmpeg to provide becomes planar video frame image sequence sequence with the planar video sequential decoding, can select start frame be 50 and frame number be 31, the Video Decoder that provides of ffmpeg will be converted to the planar video sequence of 50-80 frame corresponding planar video frame image sequence then; And when in the prior art planar video sequence being converted into the planar video frame image sequence, planar video sequence with the 50-80 frame is converted to stereoscopic video sequence if desired, so planar video sequence (100 frame) all can be converted to earlier the planar video frame image sequence so, and then, carry out follow-up conversion again by manually from the planar video frame image sequence, selecting the planar video frame image sequence of 50-80 frame.
202: the planar video frame image sequence that obtains is carried out greyscale transformation, obtain grayscale image sequence.
Wherein, the planar video frame image sequence that obtains is carried out greyscale transformation, obtain grayscale image sequence, specifically be, utilize RGB (Red Green Blue, RGB) the image transform formula carries out greyscale transformation to the planar video frame image sequence, obtains the grayscale image sequence of planar video frame image sequence correspondence.RGB image transform formula is as follows:
Y＝0.212671×R+0.715160×G+0.072169×B (1)
Wherein, Y is each gray values of pixel points of gray level image, and R, G, B are respectively R, G, the B components of each pixel in the planar video two field picture.
203: every adjacent two frame gray level images in the grayscale image sequence are poor, obtain the frame difference image sequence.
Wherein, every adjacent two frame gray level images in the grayscale image sequence are poor, obtain the frame difference image sequence specifically: corresponding per two gray values of pixel points of adjacent two frame gray level images are done difference and asked absolute value; The absolute value result that obtains as corresponding each gray values of pixel points of frame difference image, is generated the frame difference image of adjacent two frame gray level images; According to the method described above, obtain the frame difference image of all adjacent two frame gray level images in the grayscale image sequence successively, thereby obtain the frame difference image sequence of grayscale image sequence correspondence.
204: the frame difference image sequence is carried out frame difference addition accumulation, obtain the depth map sequence of planar video frame image sequence correspondence.
Wherein,, obtain the depth map sequence of planar video frame image sequence correspondence, specifically comprise:, the planar video frame image sequence is divided into the relevant a plurality of subsequences of content according to the frame difference image sequence according to the frame difference image sequence; The frame difference image sequence of each subsequence correspondence in a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in a plurality of subsequences; With the accumulation frame difference image of each subsequence in a plurality of subsequences depth map, thereby obtain the depth map sequence of planar video sequence correspondence as all planar video sequences that each subsequence comprised in a plurality of subsequences.
Wherein, the planar video frame image sequence is divided into the relevant a plurality of subsequences of content, specifically: according to the context relation of adjacent video frames picture material, the planar video frame image sequence is divided into the relevant a plurality of subsequences of content, for example: according to high correlation (personage's number unanimity of video frame images content context, personage's action is continuous and amplitude is smaller), the planar video sequence is divided into a plurality of subsequences of height correlation.If adjacent two frame planar video two field pictures are complete contents relevant (supposing that promptly two frame planar video two field pictures are duplicate), so each gray values of pixel points of the frame difference image of these adjacent two frame planar video two field picture correspondences should very little (being assumed to be 0～20) or the very little pixel number of gray value can in total pixel number, account for significant proportion; If adjacent two frame planar video two field pictures are contents irrelevant (supposing that promptly two frame planar video two field pictures are different fully), so each gray values of pixel points of the frame difference image of these adjacent two frame planar video two field picture correspondences should very big (being assumed to be 200～256) or the very big pixel number of gray value can in total pixel number, account for significant proportion; Above-mentioned characteristic according to the frame difference image of adjacent two frame planar video two field picture correspondences, present embodiment is according to the frame difference image sequence, when the planar video frame image sequence was divided into the relevant a plurality of subsequence of content, detailed process was as follows: set in advance a gray threshold (as being set to 50) and proportion threshold value (as being set to 70%); Each gray values of pixel points and this gray threshold of frame difference image are compared, and statistics is less than the number of this gray threshold; Calculate the ratio that accounts for total pixel number less than the number of this gray threshold then, obtain corresponding ratio value; Again this ratio value and proportion threshold value are compared,, then can be divided into same subsequence two frame planar video two field pictures that should frame difference image if greater than proportion threshold value; All frame difference images that successively the frame difference image sequence comprised are judged according to the method described above, correspondingly just the planar video frame image sequence can be divided into the relevant a plurality of subsequences of content.Need to prove that gray value threshold value and proportion threshold value can be provided with according to actual conditions, correlation between the subsequence that obtains as needs big (be between the planar video frame image sequence difference less), relatively littler (as being set to 20) that the gray value threshold value can be provided with is with big (as being set to 80%) relatively of proportion threshold value setting.For example: the planar video frame image sequence includes 5 frames, setting in advance gray threshold is 50, proportion threshold value is 70%, each gray values of pixel points and gray threshold in the frame difference image of the 1st frame planar video two field picture and the 2nd frame planar video two field picture correspondence are compared, the number that counts on less than this gray threshold is 240, calculate 240 then and account for the ratio of total pixel number (being assumed to be 300), obtaining corresponding ratio value is 80%, this ratio value is greater than proportion threshold value, so the 1st frame planar video two field picture and the 2nd frame planar video two field picture can be divided into same subsequence; The ratio value that correspondingly obtains the frame difference image of the 2nd frame planar video two field picture and the 3rd frame planar video two field picture correspondence is 90%, the ratio value of the frame difference image of the 3rd frame planar video two field picture and the 4th frame planar video two field picture correspondence is 40%, the ratio value of the frame difference image of the 4th frame planar video two field picture and the 5th frame planar video two field picture correspondence is 85%, so with the 1st frame planar video two field picture, the 2nd frame planar video two field picture and the 3rd frame planar video two field picture are divided into the 1st subsequence, the 4th frame planar video two field picture and the 5th frame planar video two field picture are divided into the 2nd subsequence.
Wherein, the frame difference image sequence of each subsequence correspondence in a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in a plurality of subsequences, specifically: corresponding each gray values of pixel points addition of all frame difference images of all video frame images correspondences that each subsequence in a plurality of subsequences is comprised self, with the addition result that obtains as the corresponding gray values of pixel points of each subsequence oneself accumulation frame difference image in a plurality of subsequences, thereby obtain the accumulation frame difference image of each subsequence self correspondence in a plurality of subsequences.For example: above-mentioned the 1st subsequence comprises the 1st frame planar video two field picture, the 2nd frame planar video two field picture and the 3rd frame planar video two field picture, if the frame difference image of the 1st frame planar video two field picture and the 2nd frame planar video two field picture correspondence is the 1st frame difference image, the frame difference image of the 2nd frame planar video two field picture and the 3rd frame planar video two field picture correspondence is the 2nd frame difference image, with the 1st frame difference image and corresponding per two the gray values of pixel points additions of the 2nd frame difference image, the addition result that obtains is accumulated corresponding each gray values of pixel points of frame difference image as the 1st subsequence, thereby obtain the accumulation frame difference image of the 1st subsequence correspondence; Above-mentioned the 2nd subsequence comprises the 3rd frame planar video two field picture and the 4th frame planar video two field picture, because the frame difference image of the 3rd frame planar video two field picture and the 4th frame planar video two field picture correspondence has only 1 frame, so the frame difference image of the 3rd frame planar video two field picture and the 4th frame planar video two field picture correspondence is the accumulation frame difference image of the 2nd subsequence.
Wherein, with the accumulation frame difference image of each subsequence in a plurality of subsequences depth map as all planar video sequences that each subsequence comprised in a plurality of subsequences, thereby obtain the depth map sequence of planar video sequence correspondence, for example: above-mentioned the 1st subsequence comprises the 1st frame planar video two field picture, the 2nd frame planar video two field picture and the 3rd frame planar video two field picture, with the accumulation frame difference image of the 1st subsequence depth map as the 1st frame planar video two field picture, the 2nd frame planar video two field picture and the 3rd frame planar video two field picture; Above-mentioned the 2nd subsequence comprises the 3rd frame planar video two field picture and the 4th frame planar video two field picture, with the accumulation frame difference image of the 2nd subsequence depth map as the 3rd frame planar video two field picture and the 4th frame planar video two field picture; Thereby obtained the depth map sequence (5 frame) of planar video sequence (5 frame).
Need to prove, the planar video frame image sequence is divided into the relevant a plurality of subsequences of content, with the accumulation frame difference image of each subsequence in a plurality of subsequences depth map, thereby obtain the depth map sequence of planar video sequence correspondence as all planar video sequences that each subsequence comprised in a plurality of subsequences; With compare by the manual depth map that obtains each planar video in the prior art, reduced workload, improved the efficient of converting plane video into stereoscopic video sequence.
205: depth map sequence is carried out smoothing processing.
Wherein, depth map sequence is carried out smoothing processing specifically is, with low pass filter depth map sequence is carried out filtering, present embodiment adopts Gaussian (Gauss) low pass filter to carry out smoothing processing, also can adopt other low pass filter in the practical application, as Laplce's low pass filter etc.Depth map sequence is carried out smoothing processing, can eliminate the rough discontinuous part of noise and depth map edge of depth map inside, and reduced the influence of accumulation frame mistake difference, thereby avoided the antemarginal shake of generation stereoscopic video sequence.
206: the depth map sequence based on planar video frame image sequence and planar video frame image sequence correspondence, utilize depth information to play up algorithm, obtain the three-dimensional video-frequency frame image sequence.
Wherein, depth map sequence based on planar video frame image sequence and planar video frame image sequence correspondence, utilize depth information to play up algorithm, the process that obtains the three-dimensional video-frequency frame image sequence is: with every frame planar video two field picture of planar video frame image sequence as left view, left view and depth map synthetic the obtain right view corresponding with left view, and then the synthetic three-dimensional video-frequency frame image sequence (odd number of three-dimensional video-frequency frame image sequence is classified the odd column of left view as, and the even number of three-dimensional video-frequency frame image sequence is classified the even column of right view as) that obtains of mode that left and right sides view is intersected with the odd even ordered series of numbers.
207: the stereoscopic video frame image sequence is encoded, and obtains stereoscopic video sequence and output.
Wherein, the stereoscopic video frame image sequence is encoded, and obtains stereoscopic video sequence and export comprising: the stereoscopic video sequence parameter is set; Read the three-dimensional video-frequency frame image sequence, obtain the stereo video frame image sequence information; Call the video encoder stereoscopic video frame image sequence information of ffmpeg and encode, obtain stereoscopic video sequence and output.The stereo video frame image sequence information comprises: the sequence number of the width of the frame number of stereoscopic video sequence, every frame three-dimensional video-frequency, the height of every frame three-dimensional video-frequency and every frame three-dimensional video-frequency etc.
Need to prove, the stereoscopic video sequence parameter is set specifically is meant the parameters such as file format, stereoscopic video sequence frame number, stereoscopic video sequence resolution of setting the output stereoscopic video sequence according to actual conditions (as the parameter of playing up algorithm according to the parameter and the depth information of three-dimensional display); The stereoscopic video sequence that correspondingly obtains at last and export is to export according to the stereoscopic video sequence parameter that is provided with.
And need to prove, may there be the three-dimensional video-frequency two field picture that cannot encode that has damaged in practical application neutral body video frame images sequence, so before the video encoder stereoscopic video frame image sequence information of calling ffmpeg is decoded, also need to judge whether and can encode according to the stereo video frame image sequence information obtained, and if could the video encoder stereoscopic video frame image sequence information of calling ffmpeg encode; Otherwise, close this three-dimensional video-frequency two field picture.Wherein, judge whether that according to the planar video sequence information that obtains the principle that can decode is similar in the principle and 201 that judges whether to encode according to the stereo video frame image sequence information of obtaining, repeat no more herein.
Need to prove that in addition the process of video encoder that realizes calling ffmpeg is similar with the process of the Video Decoder of realizing calling ffmpeg, is not giving unnecessary details herein.
The described method of present embodiment by ffmpeg is referred in the process of converting plane video into stereoscopic video, has been save a large amount of artificial participations, has saved a large amount of manpowers and time, has improved the efficient of converting plane video into stereoscopic video; And can be according to the video parameter that is provided with, the three-dimensional video-frequency of output corresponding format does not need manually to carry out the conversion of form, makes simple to operately, has increased user's experience; And, do not need intermediate object program is kept in the hard disk, saved the memory space of hard disk; And, the accumulation frame difference image of subsequence as depth map, is obtained depth map and compares with manual, reduced workload, improved obtain depth map efficient; In addition,, eliminated the rough discontinuous part of noise and depth map edge of depth map inside, reduced the influence of accumulation frame mistake difference, thereby avoided the antemarginal shake of generation stereoscopic video sequence by depth map sequence is carried out smoothing processing.
Embodiment 3
Referring to Fig. 3, the embodiment of the invention provides a kind of device of converting plane video into stereoscopic video, and this device comprises:
Planar video frame image sequence acquisition module 301 is used for the planar video sequence is decoded, and obtains the planar video frame image sequence;
Frame difference image sequence acquisition module 302 is used for after planar video frame image sequence 301 acquisition modules obtain the planar video frame image sequence planar video frame image sequence being carried out greyscale transformation, poor, obtains the frame difference image sequence;
Depth map sequence acquisition module 303 is used for after frame difference image sequence acquisition module 302 obtains the frame difference image sequence, and the frame difference image sequence is carried out frame difference addition accumulation, obtains depth map sequence;
Three-dimensional video-frequency frame image sequence acquisition module 304 is used for after depth map sequence acquisition module 303 obtains depth map sequence, based on planar video frame image sequence and depth map sequence, utilizes depth information to play up algorithm and generates the three-dimensional video-frequency frame image sequence;
Stereoscopic video sequence acquisition module 305 is used for after three-dimensional video-frequency frame image sequence acquisition module 304 obtains the three-dimensional video-frequency frame image sequence, and the stereoscopic video frame image sequence is encoded, and obtains stereoscopic video sequence.
Wherein, planar video frame image sequence acquisition module 301 specifically comprises:
Planar video sequence information acquiring unit is used to open the planar video sequence, obtains the information of planar video sequence;
Planar video frame image sequence acquiring unit is used for after planar video sequence information acquiring unit obtains the information of plane video sequence, calls the Video Decoder of ffmpeg the information of planar video sequence is decoded, and obtains the planar video frame image sequence.
Wherein, depth map sequence acquisition module 303 specifically comprises:
The subsequence division unit is used for after frame difference image sequence acquiring unit obtains the frame difference image sequence, and according to the frame difference image sequence, the planar video frame image sequence that planar video frame image sequence acquiring unit is obtained is divided into the relevant a plurality of subsequences of content;
Accumulation frame difference image acquiring unit is used for after the subsequence division singly marks off a plurality of subsequences the frame difference image sequence of each subsequence correspondence in a plurality of subsequences being carried out frame difference addition accumulation, obtains the accumulation frame difference image of each subsequence in a plurality of subsequences;
The depth map sequence acquiring unit, be used for after accumulation frame difference image acquiring unit obtains accumulating frame difference image, with the accumulation frame difference image of each subsequence in a plurality of subsequences depth map, thereby obtain the depth map sequence of planar video sequence correspondence as all planar video sequences that each subsequence comprised in a plurality of subsequences.
Wherein, stereoscopic video sequence acquisition module 305 specifically comprises:
Module is set, is used for after three-dimensional video-frequency frame image sequence acquisition module 304 obtains the three-dimensional video-frequency frame image sequence, the parameter of stereoscopic video sequence being set;
Stereo video frame image sequence information acquiring unit is used for reading the three-dimensional video-frequency frame image sequence after the parameter that module is provided with stereoscopic video sequence is set, and obtains the information of three-dimensional video-frequency frame image sequence;
The stereoscopic video sequence acquiring unit, be used for after stereo video frame image sequence information acquiring unit obtains the information of stereoscopic frequency frame image sequence, call the information of the video encoder stereoscopic video frame image sequence of ffmpeg and encode, obtain stereoscopic video sequence.
Further, referring to Fig. 4, this device also comprises:
Smoothing processing module 306, be used for after depth map sequence acquisition module 303 obtains depth map sequence, depth map sequence is carried out smoothing processing, notify three-dimensional video-frequency frame image sequence acquisition module 304 based on planar video frame image sequence and depth map sequence then, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence.
The described device of present embodiment is being converted to planar video in the process of three-dimensional video-frequency, has save a large amount of artificial participations, has saved a large amount of manpowers and time, has improved the efficient of converting plane video into stereoscopic video; And, do not need intermediate object program is kept in the hard disk, saved the memory space of hard disk; And, the accumulation frame difference image of subsequence as depth map, is obtained depth map and compares with manual, reduced workload, improved obtain depth map efficient; In addition,, eliminated the rough discontinuous part of noise and depth map edge of depth map inside, reduced the influence of accumulation frame mistake difference, thereby avoided the antemarginal shake of generation stereoscopic video sequence by depth map sequence is carried out smoothing processing.
All or part of content in the technical scheme that above embodiment provides can realize that its software program is stored in the storage medium that can read by software programming, storage medium for example: the hard disk in the computer, CD or floppy disk.
The above only is preferred embodiment of the present invention, and is in order to restriction the present invention, within the spirit and principles in the present invention not all, any modification of being done, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (4)
1. the method for a converting plane video into stereoscopic video is characterized in that, described method comprises:
Open the planar video sequence, obtain the information of described planar video sequence;
Call the Video Decoder of ffmpeg the information of described planar video sequence is decoded, obtain described planar video frame image sequence;
Described planar video frame image sequence is carried out greyscale transformation, poor, obtain the frame difference image sequence;
According to described frame difference image sequence, described planar video frame image sequence is divided into the relevant a plurality of subsequences of content;
The frame difference image sequence of each subsequence correspondence in described a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in described a plurality of subsequence;
With the accumulation frame difference image of each subsequence in described a plurality of subsequences depth map, thereby obtain the described depth map sequence of described planar video sequence correspondence as all planar video sequences that each subsequence comprised in described a plurality of subsequences;
Based on described planar video frame image sequence and described depth map sequence, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence;
The parameter of described stereoscopic video sequence is set;
Read described three-dimensional video-frequency frame image sequence, obtain the information of described three-dimensional video-frequency frame image sequence;
Call the video encoder of ffmpeg the information of described three-dimensional video-frequency frame image sequence is encoded, obtain described stereoscopic video sequence.
2. the method for converting plane video into stereoscopic video according to claim 1, it is characterized in that, described described frame difference image sequence is carried out frame difference addition accumulation, obtain after the depth map sequence, based on described planar video frame image sequence and described depth map sequence, utilize depth information to play up algorithm and generate before the three-dimensional video-frequency frame image sequence, also comprise:
Described depth map sequence is carried out smoothing processing.
3. the device of a converting plane video into stereoscopic video is characterized in that, described device comprises:
Planar video frame image sequence acquisition module, specifically comprise planar video sequence information acquiring unit and planar video frame image sequence acquiring unit, wherein, described planar video sequence information acquiring unit, be used to open the planar video sequence, obtain the information of described planar video sequence; Described planar video frame image sequence acquiring unit, be used for after described planar video sequence information acquiring unit obtains the information of described planar video sequence, call the Video Decoder of ffmpeg the information of described planar video sequence is decoded, obtain described planar video frame image sequence;
Frame difference image sequence acquisition module is used for after described planar video frame image sequence acquisition module obtains described planar video frame image sequence described planar video frame image sequence being carried out greyscale transformation, poor, obtains the frame difference image sequence;
The depth map sequence acquisition module, specifically comprise subsequence division unit, accumulation frame difference image acquiring unit and depth map sequence acquiring unit, wherein, described subsequence division unit, be used for after described frame difference image sequence acquisition module obtains described frame difference image sequence, according to described frame difference image sequence, the described planar video frame image sequence that described planar video frame image sequence acquiring unit is obtained is divided into the relevant a plurality of subsequences of content; Described accumulation frame difference image acquiring unit, be used for after described subsequence division unit marks off described a plurality of subsequence, the frame difference image sequence of each subsequence correspondence in described a plurality of subsequences is carried out frame difference addition accumulation, obtain the accumulation frame difference image of each subsequence in described a plurality of subsequence; Described depth map sequence acquiring unit, be used for after described accumulation frame difference image acquiring unit obtains described accumulation frame difference image, with the accumulation frame difference image of each subsequence in described a plurality of subsequences depth map, thereby obtain the described depth map sequence of described planar video sequence correspondence as all planar video sequences that each subsequence comprised in described a plurality of subsequences;
Three-dimensional video-frequency frame image sequence acquisition module, be used for after described depth map sequence acquisition module obtains described depth map sequence, based on described planar video frame image sequence and described depth map sequence, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence;
The stereoscopic video sequence acquisition module, specifically comprise module, stereo video frame image sequence information acquiring unit and stereoscopic video sequence acquiring unit are set, wherein, the described module that is provided with, be used for after described three-dimensional video-frequency frame image sequence acquisition module obtains described three-dimensional video-frequency frame image sequence, the parameter of described stereoscopic video sequence being set; Described stereo video frame image sequence information acquiring unit, be used for described the parameter that module is provided with described stereoscopic video sequence is set after, read described three-dimensional video-frequency frame image sequence, obtain the information of described three-dimensional video-frequency frame image sequence; Described stereoscopic video sequence acquiring unit, be used for after described stereo video frame image sequence information acquiring unit obtains the information of described three-dimensional video-frequency frame image sequence, call the video encoder of ffmpeg the information of described three-dimensional video-frequency frame image sequence is encoded, obtain stereoscopic video sequence.
4. the device of converting plane video into stereoscopic video according to claim 3 is characterized in that, described device also comprises:
The smoothing processing module, be used for after described depth map sequence acquisition module obtains described depth map sequence, described depth map sequence is carried out smoothing processing, notify described three-dimensional video-frequency frame image sequence acquisition module based on described planar video frame image sequence and described depth map sequence then, utilize depth information to play up algorithm and generate the three-dimensional video-frequency frame image sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910077228XA CN101483788B (en) | 2009-01-20 | 2009-01-20 | Method and apparatus for converting plane video into tridimensional video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910077228XA CN101483788B (en) | 2009-01-20 | 2009-01-20 | Method and apparatus for converting plane video into tridimensional video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101483788A CN101483788A (en) | 2009-07-15 |
| CN101483788B true CN101483788B (en) | 2011-03-23 |
Family
ID=40880684
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910077228XA Expired - Fee Related CN101483788B (en) | 2009-01-20 | 2009-01-20 | Method and apparatus for converting plane video into tridimensional video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101483788B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101662695B (en) * | 2009-09-24 | 2011-06-15 | 清华大学 | Method and device for acquiring virtual viewport |
| US8537200B2 (en) * | 2009-10-23 | 2013-09-17 | Qualcomm Incorporated | Depth map generation techniques for conversion of 2D video data to 3D video data |
| CN101729919B (en) * | 2009-10-30 | 2011-09-07 | 无锡景象数字技术有限公司 | System for full-automatically converting planar video into stereoscopic video based on FPGA |
| CN101873509B (en) * | 2010-06-30 | 2013-03-27 | 清华大学 | Method for eliminating background and edge shake of depth map sequence |
| CN102469322B (en) * | 2010-11-18 | 2014-08-13 | Tcl集团股份有限公司 | Image processing method for plane stereoscopic bodies |
| CN102036090B (en) * | 2010-12-16 | 2012-05-02 | 清华大学 | Television signal conversion device for digital television terminal |
| US9470778B2 (en) * | 2011-03-29 | 2016-10-18 | Microsoft Technology Licensing, Llc | Learning from high quality depth measurements |
| CN102223554B (en) * | 2011-06-09 | 2013-04-03 | 清华大学 | Depth image sequence generating method and device of plane image sequence |
| US20130257851A1 (en) * | 2012-04-01 | 2013-10-03 | Chao-Hua Lee | Pipeline web-based process for 3d animation |
| US10075689B2 (en) | 2012-07-04 | 2018-09-11 | Intel Corporation | Region-of-interest based 3D video coding |
| CN103248909B (en) * | 2013-05-21 | 2015-05-20 | 清华大学 | Method and system of converting monocular video into stereoscopic video |
| CN103581650B (en) * | 2013-10-21 | 2015-08-19 | 四川长虹电器股份有限公司 | Binocular 3D video turns the method for many orders 3D video |
| WO2016038811A1 (en) * | 2014-09-12 | 2016-03-17 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | Recording medium, playback device, and playback method |
| CN104717514B (en) * | 2015-02-04 | 2018-03-02 | 四川长虹电器股份有限公司 | Rendering multi-viewpoint images system and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5717415A (en) * | 1994-02-01 | 1998-02-10 | Sanyo Electric Co., Ltd. | Display system with 2D/3D image conversion where left and right eye images have a delay and luminance difference base upon a horizontal component of a motion vector |
| JP2006191357A (en) * | 2005-01-06 | 2006-07-20 | Victor Co Of Japan Ltd | Reproduction device and reproduction program |
| CN101257641A (en) * | 2008-03-14 | 2008-09-03 | 清华大学 | Method for converting plane video into stereoscopic video based on human-machine interaction |
-
2009
- 2009-01-20 CN CN200910077228XA patent/CN101483788B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5717415A (en) * | 1994-02-01 | 1998-02-10 | Sanyo Electric Co., Ltd. | Display system with 2D/3D image conversion where left and right eye images have a delay and luminance difference base upon a horizontal component of a motion vector |
| JP2006191357A (en) * | 2005-01-06 | 2006-07-20 | Victor Co Of Japan Ltd | Reproduction device and reproduction program |
| CN101257641A (en) * | 2008-03-14 | 2008-09-03 | 清华大学 | Method for converting plane video into stereoscopic video based on human-machine interaction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101483788A (en) | 2009-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101483788B (en) | Method and apparatus for converting plane video into tridimensional video | |
| CN103688533B (en) | Chroma intra prediction method and the device of line storage can be reduced | |
| CN109951715B (en) | H.264-based YUV444 image coding and decoding method | |
| CN107071417B (en) | A kind of intra-frame prediction method for Video coding | |
| CN106412579B (en) | A kind of coding of image, coding/decoding method and device | |
| CN1416650A (en) | method for filtering digital images, and filetring device | |
| CN102724492B (en) | Method and system for transmitting and playing video images | |
| CN102045578B (en) | Image processing apparatus and image processing method | |
| CN100348045C (en) | Resolution changing method and apparatus | |
| CN1010735B (en) | The display packing and the device of image | |
| CN104272736A (en) | Context determination for entropy coding of run-length encoded transform coefficients | |
| CN103716626A (en) | Image processing apparatus and method | |
| CN104412595B (en) | The method of three-dimensional video stream and corresponding generation and reconstructing arrangement are generated and reconstruct according to the use of masking figure | |
| CN106170093B (en) | Intra-frame prediction performance improving coding method | |
| CN101325718A (en) | Image processing apparatus and method | |
| CN106464890A (en) | Scalable video encoding/decoding method and apparatus | |
| CN1523893A (en) | Video encoding method and apparatus | |
| CN104253998A (en) | Hardware on-chip storage method of deblocking effect filter applying to HEVC (High Efficiency Video Coding) standard | |
| CN1214648C (en) | Method and apparatus for performing motion compensation in a texture mapping engine | |
| US10999604B2 (en) | Adaptive implicit transform setting | |
| CN1129868C (en) | Image processing apparatus and method | |
| CN1224310A (en) | Image encoding/decoding by eliminating color components in pixels | |
| CN104935928B (en) | A kind of efficient image compression method based on spatial domain down-sampling pattern | |
| CN110572713B (en) | Transcoding method and processing terminal for adaptive video bandwidth ratio | |
| CN1143553C (en) | Apparatus and method of coding/decoding moving picture and storage medium storing moving picture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110323 Termination date: 20190120 |