CN1204757C - Stereo video stream coder/decoder and stereo video coding/decoding system - Google Patents
Stereo video stream coder/decoder and stereo video coding/decoding system Download PDFInfo
- Publication number
- CN1204757C CN1204757C CN 03116541 CN03116541A CN1204757C CN 1204757 C CN1204757 C CN 1204757C CN 03116541 CN03116541 CN 03116541 CN 03116541 A CN03116541 A CN 03116541A CN 1204757 C CN1204757 C CN 1204757C
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- video flowing
- auxilliary
- estimation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention relates to data compression technique for stereo videos. When the present invention carries out encoding for binocular video streams collected by a stereo camera system, one path of videos carry out encoding according to the standard of compatible MPEG series, and another path of videos respectively uses parallax compensation prediction, and combined parallax and motion compensation prediction, and carry out coding transmission at decoding ends through frame estimation and interpolation, wherein the parallax estimation uses multilevel segmentation block estimation based on Markov model; in order to utilize reference frame pictures recovered the decoding end, corresponding parallax and motion vector, the method of the frame estimation and the interpolation uses a method based on a frame estimation probability model to carry out estimation and interpolation. The decoding end has two stage decoding, wherein one state decodes main video streams to obtain the single video displayed on common display equipment, and the other stage decodes all double video streams; recovered stereo video signals are synthesized into displayed stereo pictures by an automatic stereo display.
Description
Technical field
The present invention relates to the moving image treatment technology, be specifically related to a kind of coding/decoding method and device of stereo video data.
Background technology
Human when watching the world around, can not only see the width and the height of object, and can know their degree of depth, can judgment object between or the distance between beholder and the object.The main cause that this 3D vision characteristic produces is: people usually always binocular watch object simultaneously, and because the spacing (about 65mm) of two eyes optical axis, left eye and right eye are when seeing the object of certain distance, received sensed image is different, thereby brain is by motion, the adjustment of eyeball, combine the information of this two images, produce third dimension.When list was watched object with left eye and right eye, the visual referred sensation that is produced just was parallax.
The 3D vision characteristic that comes from the binocular structure to we provide one from about obtain two images real world relative depth sense directly and simple method, and this relative depth information is such as telecommunication (tele-medicine, teleconference), tele-robotic (Remote, autonomous aviation, supervision), be vital in the application of amusement (interactive HDTV, three-dimensional film) and virtual reality and so on.But this to introduce the obvious cost of relative depth information be to make its transmission and data quantity stored double above than mono-vision system in order to increase authenticity.In order to satisfy the increase of data volume, settling mode increases channel width nothing more than, improves channel utilization and reduce these approach of information source code check with compress technique efficiently with agreement efficiently.But, therefore must adopt effective image compression technology owing to increase the diseconomy of the memory span and the network bandwidth.
The method of stereo scopic video coding all is to utilize the correlation between the binocular video stream to come the whole code efficiency that improves the two-path video signal in itself under the prior art.Two class methods are arranged substantially, and the first kind is the three-dimensional video-frequency stream encoding method based on the MPEG video encoding standard, and its basic principle is that one road video flowing is wherein encoded separately, and another road video flowing then adopts disparity estimation and compensation technique to encode.These class methods mostly adopt the hybrid coding mode, for example with the mixing definition of operating such coding (definition of one of them stream is relative relatively poor), based on the Bit Allocation in Discrete method of psychological characteristics, based on the D encoding of multiresolution and adopt frame to estimate that interpolation rebuilds right B frame and clap, right wing video flowing B frame does not transmit, and recovers but make interpolation in decoding end) etc. mode.The problem that these class methods exist is: the efficient of disparity estimation compensation haves much room for improvement; Ignored the effective utilization to right wing stream movable information when utilizing binocular parallax information, binary encoding efficient also has the bigger rising space; Though adopt frame to estimate that the D encoding compression ratio of interpositioning is high, existing frame estimates that interpositioning is fairly simple, the image reconstruction quality is undesirable; Also lack ripe perfect D encoding system generally.
Second class is object-based D encoding method, and its basic principle is the object in the scene to be cut apart to extract also encode in conjunction with motion and binocular depth information.But when having a plurality of objects to occur in the scene, the coding effect of these class methods is also bad, and owing to its complexity of calculation, real-time is also relatively poor, and is still far away from the requirement of real time system application simultaneously.
Summary of the invention
The purpose of this invention is to provide a kind of stereo video streaming coder/decoder, it have compression rates height, decode rate fast and can with the advantage of single video coded system compatibility.
Above-mentioned purpose of the present invention is achieved through the following technical solutions:
A kind of three-dimensional video-frequency stream encoder comprises:
Main video flowing coding unit, the road video flowing that is used for stereoscopic video stream is encoded to generate main video code flow according to the MPEG agreement;
Auxilliary video flowing coding unit, it comprises:
Parallax/motion-compensated estimation is single not to be had, another road video flowing interior corresponding intracoded frame and the MB of prediction frame of stereoscopic video stream are carried out disparity-compensated estimation respectively to be used for utilizing interior intracoded frame of main video flowing and MB of prediction frame, and utilize intracoded frame previous in the auxilliary video flowing and/or MB of prediction frame that current MB of prediction frame in the auxilliary video flowing is carried out motion-compensated estimation, the initial value of disparity-compensated estimation obtains according to following manner: utilizes by the MB of prediction frame in the auxilliary video flowing being carried out motion vector that motion-compensated estimation obtains the optical parallax field of previous intracoded frame or MB of prediction frame carried out motion-compensated estimation, and with the initial value of new optical parallax field as disparity-compensated estimation:
The compensation prediction coding unit is used for the disparity-compensated estimation information of the intracoded frame in the auxilliary video flowing and the disparity-compensated estimation information and the motion-compensated estimation information of MB of prediction frame are encoded to generate auxilliary video code flow;
Multiplexer is used for main video code flow and auxilliary video code flow are generated the three-dimensional video-frequency code stream with time division multiplexing mode.
Reasonable is that in above-mentioned three-dimensional video-frequency stream encoder, described parallax/motion estimation unit adopts carries out disparity estimation based on layering markov probabilistic model and multistage matching way.Be more preferably, in described layering markov probabilistic model and overlapping piece matching way, stratum level is set at two-stage, and the block size is divided into 8 * 8 and 16 * 16 two kinds.
Reasonable is in above-mentioned three-dimensional video-frequency stream encoder, to adjust the transmission channel bandwidth that auxilliary video code flow takies by the residual error image DCT quantization parameter that changes in the disparity-compensated estimation information.
A kind of three-dimensional video-frequency stream decoder comprises:
Demultiplexer is used for the three-dimensional video-frequency code stream is decomposed into main video code flow and auxilliary video code flow;
Main video code flow decoding unit is used for main video code flow is decoded to generate main video flowing according to the MPEG agreement;
Auxilliary video code flow decoding unit, it comprises:
Parallax/motion compensated prediction unit is used for rebuilding according to the disparity-compensated estimation information that comprises in the intracoded frame of main video flowing and MB of prediction frame and the auxilliary video code flow and motion-compensated estimation information the intracoded frame and the I picture predictive frame of auxilliary video flowing;
Frame estimates and interpolation unit, be used for according to corresponding main video flowing bi-directional predicted/disparity-compensated estimation information that intracoded frame in the interpolation frame, auxilliary video flowing and MB of prediction frame and auxilliary video code flow comprise and motion-compensated estimation information rebuilds the bi-directional predicted/interpolation frame in the auxilliary video flowing;
Assist the video flowing reconstruction unit, be used for the intracoded frame of parallax/motion compensated prediction unit reconstruction and the bi-directional predicted/two-way interpolation frame of I picture predictive frame and frame estimation and interpolation unit reconstruction are sorted to generate auxilliary video flowing according to time order and function.
Reasonable is that in above-mentioned three-dimensional video-frequency stream decoder, frame estimates to estimate based on the three-dimensional frame of Bayes's minimum cost equation with the interpolation unit employing and interpolating method is rebuild bi-directional predicted/interpolation frame.
In three-dimensional video-frequency stream encoder of the present invention, owing to only one of them video flowing is carried out high-quality coding according to mpeg standard, and in another video flowing, have only a few frames to encode, all the other frames are fully " skipping " and carry out frame in decoding end and estimate that interpolation recovers then, therefore improve code efficiency greatly, saved transmission bandwidth.
The purpose of this invention is to provide a kind of three-dimensional video-frequency treatment system, it have compression rates height, decode rate fast and can with the advantage of single video coded system compatibility.
Above-mentioned purpose of the present invention is achieved through the following technical solutions:
A kind of processing system for video, comprise picked-up left road and right wing video flowing video camera, make the synchronous in time time base corrector of the video flowing of two video cameras outputs, will be through the stream of the two-path video after the time base corrector time synchronizing multiplexed frame sequential multiplexer, the computer system that comprises stereo coder as claimed in claim 1 and three-dimensional video-frequency stream decoder as claimed in claim 5 and regular display and three-dimensional display with the formation stereo video streaming
Wherein, in the time only need transmitting the single channel video flowing, video image is encoded and signal is delivered to transmission channel by the main video flowing coding unit of three-dimensional video-frequency stream encoder, when needs transmitting two paths video flowing, respectively left and right sides road video image is encoded and signal is delivered to transmission channel by main video flowing coding unit and auxilliary video flowing coding unit, when the video code flow that receives only comprises one road video flowing, encoding code stream is decoded and decoded signal is delivered to regular display by the main video code flow decoding unit of three-dimensional video-frequency stream decoder, when the code stream that receives comprises left and right sides two-path video stream signal, respectively left and right sides two-way encoding code stream is decoded and decoded signal is delivered to three-dimensional display by main video code flow decoding unit and auxilliary video code flow decoding unit.
Video system of the present invention except having the advantage that code efficiency is high and the transmission bandwidth requirement is low, also can with the serial coding standard compatibility of existing single video MPEG.This is keeping having reduced the upgrade cost of system under the compatible prerequisite of coding, and the flexible control to three-dimensional display quality is provided.
Description of drawings
By below in conjunction with the description of accompanying drawing to preferred embodiment of the present invention, can further understand purpose of the present invention, feature and advantage, wherein:
Fig. 1 is the schematic diagram according to stereo video streaming coder/decoder of the present invention.
Fig. 2 is for adopting the processing system for video schematic diagram according to stereo video streaming coder/decoder of the present invention.
Embodiment
Below in conjunction with accompanying drawing preferred embodiment of the present invention is described.
Fig. 1 is the schematic diagram according to stereo video streaming coder/decoder of the present invention.As shown in Figure 1, three-dimensional video-frequency stream encoder 1 is responsible for the left and right sides video flowing of input is encoded, below for convenience of description for the purpose of supposition left video stream be main video flowing and right video flowing is auxilliary video flowing, it is limitation of the invention that but this supposition should not be construed as, and in fact also can be opposite supposition.The video code flow channel 2 that three-dimensional video-frequency stream encoder 1 coding generates transfers to three-dimensional video-frequency stream decoder 3.
Referring to Fig. 1, three-dimensional video-frequency stream encoder 1 comprises mpeg encoder 4, multiplexer 7 as main video flowing coding unit and the auxilliary video flowing coding unit that is made of parallax/motion-compensated estimation unit 5 and compensation prediction coding unit 6.
It on the MPEG digital video coding technical spirit a kind of method for compressing image that utilizes the statistical redundancy degree of video sequence on time and direction in space to realize, it depends on the correlation of (interpel) between the pixel, comprises such hypothesis: promptly have simple correlation translational motion between each successive frame.Therefore the pixel value on special frame can adopt the intraframe coding technology to be predicted according near pixel at same frame, perhaps can adopt the interframe technology to be predicted according to the pixel near the frame.
When a video sequence shot change, the temporal correlation near each frame between the pixel is just very little, even disappears, and should adopt this moment the intraframe coding technology to come the development space correlation to compress to realize active data.In the MPEG compression algorithm, adopt discrete cosine transform (DCT) coding techniques, picture block with 8 * 8 pixels is that unit effectively develops the spatial coherence between near each picture rope of same picture, below can be called intracoded frame according to the picture frame of intraframe coding technique compresses, and brief note is I
MOr I
A, wherein subscript M and A represent main video flowing and auxilliary video flowing respectively.
If have bigger correlation near the frame between each pixel, that is to say, when the content of two successive frames is very similar or identical, just can adopt interframe DPCM coding techniques based on time prediction (motion compensated prediction of interframe), below can be called MB of prediction frame, and brief note is P according to the picture frame of inter-frame coding compression
MOr P
A, wherein subscript M and A represent main video flowing and auxilliary video flowing respectively.
Also introduce a kind of picture frame that is called bi-directional predicted frames in mpeg standard, it can adopt past frame and future frame, and reduction obtains as the reference frame, but itself can not be as the reference frame, below this class picture frame is called bi-directional predicted frames, and brief note is B
MOr B
A, wherein subscript M and A represent main video flowing and auxilliary video flowing respectively.
In the present invention, 4 pairs of left roads of mpeg encoder video flowing is encoded generating main video code flow according to mpeg standard, and this main video code flow is by according to certain tactic coding back I
M, P
MAnd B
MFrame sequence constitutes.
As shown in the figure, two-path video stream in the left and right sides all is transfused to the parallax/motion estimation unit 5 in the auxilliary video flowing coding unit, and carries out parallax and estimation in this unit.Particularly, with main video flowing and auxilliary video flowing inter-sync or corresponding intracoded frame I
MWith I
AAnd MB of prediction frame P
MWith P
ACompare to obtain picture frame I in the auxilliary video flowing
AOr P
ADisparity estimation; With the auxilliary previous intracoded frame I of video flowing
AOr MB of prediction frame P
AWith current MB of prediction frame P
ACompare to obtain estimation current MB of prediction frame.Why be every width of cloth P
AFrame provide motion estimation information and disparity estimation information be because, in the ordinary course of things, motion and parallax are carried out mixed compensation can obtain best predicting the outcome, therefore in the present invention, in order to make decoding end recover the picture frame of better quality, parallax/motion estimation unit 5 is a width of cloth P
AFrame provides motion estimation information (by with previous same video flowing internal reference frame I
AFrame or P
AFrame and current P
AFrame relatively obtains) and disparity estimation information (according to the P of correspondence
AAnd P
MFrame obtains), can effectively solve the problem of blocking the code efficiency reduction that causes with parallax barrier because of time domain like this.
The method of disparity estimation has multiple, and in the present invention, parallax/motion estimation unit 5 adopts carries out disparity estimation based on layering markov probabilistic model and multistage matching way.The advantage of this method is to obtain a level and smooth and relatively accurate optical parallax field, and this will reduce the entropy of parallax compensation residual error image greatly, thereby further improves compression ratio.For with the piece size compatibility of mpeg standard, when adopting above-mentioned layering markov probabilistic model and overlapping piece matching way, stratum level is set at two-stage, the block size is divided into 8 * 8 and 16 * 16 two kinds.
Motion-compensated estimation is a kind of time-based DPCM coded prediction technology, and it has obtained extensive use in MPEG1 and MPEG2 video encoding standard.The motion compensation notion is with the basis that is estimated as to the video interframe movement, that is to say, if all objects all spatially have a displacement in the video lens, then use limited kinematic parameter (for example for the translational motion of pixel, the available motion vector is described) to come interframe movement is described.Because the spatial coherence between some motion vectors is higher usually, sometimes can think that a motion vector represented the motion of an adjacent pixel blocks, therefore a frame picture can be divided into several pixel blocks (block of pixels is 16 * 16 pixels in MPEG1 and MPEG2 standard), and only a motion vector representing each block of pixels be estimated, encoded and transmits.Owing to only prediction error picture (difference between raw frames and the motion compensated prediction picture) is encoded, therefore reduced the temporal redundancy of interframe.
Actual observation shows that for stereo video image continuous in time, their optical parallax field has the height temporal redundancy equally, and therefore in the present invention, reasonable is the initial value that obtains disparity-compensated estimation according to following manner: at first to P
AFrame carries out motion compensated prediction to obtain motion vector, then to reference frame I previous in the same video flowing
A(or P
A) optical parallax field carry out motion compensated prediction, obtain the initial value that new optical parallax field promptly can be used as disparity estimation thus.This mode can reduce the auxilliary required time of video flowing coding greatly, has improved coding rate.
Compensation prediction coding unit 6 links to each other with parallax/motion estimation unit 5, the I that it obtains parallax/motion estimation unit 5
AFrame disparity estimation compensated information and P
AFrame disparity estimation compensated information or motion estimation and compensation information encode to generate auxilliary video code flow.I behind the coding
AFrame disparity estimation compensated information bit stream is divided into three parts: difference vector stream, parallax compensation residual error image and quad-tree structure, wherein, difference vector stream adopts differential pulse coding method (DPCM) coding, and the residual error image adopts discrete cosine transform (DCT) and mark quantization methods to encode.
Multiplexer 7 links to each other with compensation prediction coding unit 6 with mpeg encoder 4, and it generates the three-dimensional video-frequency code stream with main video code flow and auxilliary video code flow with time division multiplexing mode.In the present invention, in order to improve code efficiency, all bi-directional predicted/interpolation frame (B in the auxilliary video flowing
AFrame) do not make any encoding process, do not send into multiplexer 7 with transmission on channel 2 yet as an auxilliary video code flow part.
In above-mentioned three-dimensional video-frequency stream encoder, can be by the DCT quantization parameter of residual error image behind the above-mentioned parallax compensation of change, the additional bandwidth that changes transmission channel neatly is to satisfy the stereo display under the various bandwidth demands.
Refer again to Fig. 1, three-dimensional video-frequency stream decoder 3 comprises estimates the auxilliary decoding video stream unit that constitutes with interpolation unit 11 and auxilliary video flowing reconstruction unit 12 as the mpeg decoder 9 of main decoding video stream unit, demultiplexer 7 and by parallax/motion compensated prediction unit 10, frame.
As shown in Figure 1, demultiplexer 8 is decomposed into main video code flow and auxilliary video code flow with the three-dimensional video-frequency code stream of transmission on the channel 2 and main video flowing offered mpeg decoder 9 and will assists that video flowing offers parallax/motion compensated prediction unit 10 and frame is estimated and interpolation unit 11.
9 pairs of main video code flows of mpeg decoder are decoded generating main video flowing according to the MPEG agreement, and it is by according to certain tactic recovery back I
M, P
MAnd B
MFrame sequence constitutes.
Also estimate to link to each other with auxilliary video flowing reconstruction unit 12 with interpolation unit 11 with mpeg decoder 9, frame in parallax/motion compensated prediction unit 10, it is according to intracoded frame I in the main video flowing of mpeg decoder 9 outputs
MWith MB of prediction frame P
MAnd corresponding intracoded frame I in disparity estimation compensated information that comprises in the auxilliary video code flow of demultiplexer 8 outputs and the auxilliary video flowing of motion estimation and compensation information reconstruction
AWith MB of prediction frame P
A, the I of its reconstruction
AFrame and P
AFrame is output to frame estimation and interpolation unit 11 and auxilliary video flowing reconstruction unit 12.
Frame is estimated also to link to each other with auxilliary video flowing reconstruction unit 12 with mpeg decoder 9 with interpolation unit 11, and it is according to corresponding bi-directional predicted frames B in the main video flowing of mpeg decoder 9 outputs
M, corresponding intracoded frame I in the auxilliary video flowing
AWith MB of prediction frame P
A(this B for example
AThe I that front and back are contiguous
AFrame and P
AFrame) and the disparity estimation compensated information that comprises in the auxilliary video code flow and motion estimation and compensation information rebuild the bi-directional predicted/interpolation frame of auxilliary video flowing, the B of its reconstruction
AFrame is output to auxilliary video flowing reconstruction unit 12.
In auxilliary video flowing reconstruction unit 12, the intracoded frame I that parallax/motion compensated prediction unit 10 rebuilds
AWith MB of prediction frame P
AAnd bi-directional predicted/interpolation frame B that frame is estimated and interpolation unit is rebuild
ASuccessively sort to generate auxilliary video flowing according to acquisition time.
Because the overwhelming majority is B in the auxilliary video flowing
AFrame, so in three-dimensional encoding and decoding structure, B
AFrame is rebuild speed and image quality is crucial.For this reason, adopt a kind of frame method of estimation in the present invention, its three-dimensional frame based on Bayes's minimum cost equation is estimated and interpolating method (SFEI_BLCF).This method is utilized in motion, parallax and the pictorial information (representing with arrow shown in the dotted line in Fig. 2) of decoding end acquisition and the characteristics of stereoscopic video sequence self, can synthesize B fast
AFrame, and image reconstruction has acceptable quality on the stereoscopic vision meaning.Concrete reconstruction procedures is as follows:
(1) because B
AFrame inserts in I in being
AWith P
ABetween the frame, so to I
AWith P
AMotion vector between the frame is pressed B
AFrame is to I
AThe distance of frame is stretched to determine I
APixel in the frame is at B
APosition in the frame.
(2) for same pixel, if it is at corresponding B
M, I
AAnd P
AThe difference of the pixel value in the frame then is considered as the viewing area with it less than set point, to the weighted average of these pixel values as B
AThe value of respective pixel point in the frame, and note B
AThis pixel points to I in the frame
AAnd P
AThe motion vector of frame and sensing B
MThe difference vector of frame.
(3) for same pixel, if it is at corresponding B
M, I
AAnd P
AThe difference of the pixel value in the frame is more than or equal to set point, then this pixel is considered as blocking a little, in the viewing area of its neighborhood, select in the motion vector relevant one as match motion vector, and be mapped to corresponding picture frame to obtain the final pixel value of this point according to this motion vector with each pixel.
Therefore, in stereo video streaming encoder/decoder of the present invention, the B of main video flowing
MThe I of frame, auxilliary video flowing
AAnd P
AFrame all needs to carry out coding transmission as the reference frame of interframe compensation prediction.But, can directly utilize motion that decoding end obtains and difference vector value to auxilliary video flowing B in when decoding
AFrame recovers and rebuilds and need not to carry out match search, so the present invention has the high and fireballing characteristics of decoding of compression rates.
Fig. 2 shows processing system for video schematic diagram of the present invention.As shown in Figure 2, this processing system for video comprises two the video camera 21a and 21b, the time base corrector 22 that links to each other with video camera, the frame sequential multiplexer 23 that links to each other with time base corrector 22, computer system 24 and regular display 25 and three-dimensional display 26 that absorb left road and right wing video flowing respectively, and wherein computer system 24 comprises above-mentioned stereo coder and three-dimensional video-frequency stream decoder.
In above-mentioned processing system for video, when encoding, the left and right sides video flowing of two video camera 21a and 21b output exports frame sequential multiplexer 23 to after time base corrector 22 carries out time synchronizing, send into computer system 24 through behind the multiplexed formation stereo video streaming.When having only one road video flowing input computer system 24 or only need transmit the single channel video flowing, video image is encoded and the mpeg standard signal bit stream is delivered to transmission channel by the main video flowing coding unit of three-dimensional video-frequency stream encoder, when needs transmitting two paths video flowing, respectively left and right sides road video image is encoded and the signal that will comprise main video code flow and auxilliary video code flow is delivered to transmission channel by the main video flowing coding unit of three-dimensional video-frequency stream encoder and auxilliary video flowing coding unit.
Decoding is finished by the three-dimensional video-frequency stream decoder of computer system 24, when the video flowing that receives only comprises one road video flowing, encoding code stream is decoded and decoded signal is delivered to regular display by the main video code flow decoding unit of three-dimensional video-frequency stream decoder, when the code stream that receives comprises left and right sides two-path video stream signal, respectively left and right sides two-way encoding code stream is decoded and decoded signal is delivered to automatic stereoscopic display device by the main video code flow decoding unit of three-dimensional video-frequency stream decoder and auxilliary video code flow decoding unit.
Below with a concrete example of using effect of the present invention is described.Suppose that picture frame is CIF form (352 * 288), main video flowing is encoded according to the grammer standard of mpeg encoded, this road image quality higher relatively (average peak signal to noise ratio PSNR is about 35dB), encoding rate is 0.14MbS～2.55MbS.Only there is a few frames to carry out predictive coding and transmission in the auxilliary video flowing, all the other frames are complete " skipping " then, the frame of " being skipped " at coding side carries out real-time recovery in decoding end by frame estimation and interpolation, and the average encoding rate of this road video flowing is 14.8Kbs～108Kbs.By relatively as seen, transmit that to assist the needed additional bandwidth of video flowing extremely low, make that total bit stream of three-dimensional digit TV only is about 1.15～1.3 times of common haplopia digital television transfer bit stream.Though the image quality of auxilliary video flowing is than main video flowing low slightly (average peak signal to noise ratio PSNR is about 30dB), but this have mixed-resolution about image decoding end can utilize fully the human vision system characteristic (HumanVisualsystem, HVS) and corresponding three-dimensional display synthesize stereo image with high visual definition and enough depth perceptions.
Claims (7)
1. a three-dimensional video-frequency stream encoder is characterized in that, comprising:
Main video flowing coding unit, the road video flowing that is used for stereoscopic video stream is encoded to generate main video code flow according to the MPEG agreement;
Auxilliary video flowing coding unit, it comprises:
Parallax/motion-compensated estimation unit, another road video flowing interior corresponding intracoded frame and the MB of prediction frame of stereoscopic video stream are carried out disparity-compensated estimation respectively to be used for utilizing interior intracoded frame of main video flowing and MB of prediction frame, and utilize intracoded frame previous in the auxilliary video flowing and/or MB of prediction frame that current MB of prediction frame in the auxilliary video flowing is carried out motion-compensated estimation, wherein, the initial value of disparity-compensated estimation obtains according to following manner: utilizes by the MB of prediction frame in the auxilliary video flowing being carried out motion vector that motion-compensated estimation obtains the optical parallax field of previous intracoded frame or MB of prediction frame carried out motion-compensated estimation, and with the initial value of new optical parallax field as disparity-compensated estimation;
The compensation prediction coding unit is used for the disparity-compensated estimation information of the intracoded frame in the auxilliary video flowing and the disparity-compensated estimation information and the motion-compensated estimation information of MB of prediction frame are encoded to generate auxilliary video code flow;
Multiplexer is used for main video code flow and auxilliary video code flow are generated the three-dimensional video-frequency code stream with time division multiplexing mode.
2. three-dimensional video-frequency stream encoder as claimed in claim 1 is characterized in that, described parallax/motion estimation unit adopts carries out disparity estimation based on layering markov probabilistic model and multistage matching way.
3. three-dimensional video-frequency stream encoder as claimed in claim 2 is characterized in that, in described layering markov probabilistic model and overlapping piece matching way, stratum level is set at two-stage, and the block size is divided into 8 * 8 and 16 * 16 two kinds.
4. as any described three-dimensional video-frequency stream encoder among the claim 1-3, it is characterized in that it adjusts the transmission channel bandwidth that auxilliary video code flow takies by the residual error image DCT quantization parameter that changes in the disparity-compensated estimation information.
5. a three-dimensional video-frequency stream decoder is characterized in that, comprising:
Demultiplexer is used for the three-dimensional video-frequency code stream is decomposed into main video code flow and auxilliary video code flow;
Main video code flow decoding unit is used for main video code flow is decoded to generate main video flowing according to the MPEG agreement;
Auxilliary video code flow decoding unit, it comprises:
Parallax/motion compensated prediction unit is used for rebuilding according to the disparity-compensated estimation information that comprises in the intracoded frame of main video flowing and MB of prediction frame and the auxilliary video code flow and motion-compensated estimation information the intracoded frame and the I picture predictive frame of auxilliary video flowing;
Frame estimates and interpolation unit, be used for according to corresponding main video flowing bi-directional predicted/disparity-compensated estimation information that intracoded frame in the interpolation frame, auxilliary video flowing and MB of prediction frame and auxilliary video code flow comprise and motion-compensated estimation information rebuilds the bi-directional predicted/interpolation frame in the auxilliary video flowing;
Assist the video flowing reconstruction unit, be used for the intracoded frame of parallax/motion compensated prediction unit reconstruction and the bi-directional predicted/two-way interpolation frame of I picture predictive frame and frame estimation and the single nothing reconstruction of interpolation are sorted to generate auxilliary video flowing according to time order and function.
6. three-dimensional video-frequency stream decoder as claimed in claim 5 is characterized in that, frame estimates to estimate based on the three-dimensional frame of Bayes's minimum cost equation with the interpolation unit employing and interpolating method is rebuild bi-directional predicted/interpolation frame.
7. processing system for video, it is characterized in that, the video camera that comprises left road of picked-up and right wing video flowing, make the synchronous in time time base corrector of video flowing of two video camera outputs, will be multiplexed to form the frame sequential multiplexer of stereo video streaming through the stream of the two-path video after the time base corrector time synchronizing, the computer system and regular display and the three-dimensional display that comprise stereo coder as claimed in claim 1 and three-dimensional video-frequency stream decoder as claimed in claim 5
Wherein, in the time only need transmitting the single channel video flowing, video image is encoded and signal is delivered to transmission channel by the main video flowing coding unit of three-dimensional video-frequency stream encoder, when needs transmitting two paths video flowing, by main video flowing coding unit and auxilliary video flowing coding single do not have respectively left and right sides road video image is encoded and signal is delivered to transmission channel, when the video code flow that receives only comprises one road video flowing, encoding code stream is decoded and decoded signal is delivered to regular display by the main video code flow decoding unit of three-dimensional video-frequency stream decoder, when the code stream that receives comprises left and right sides two-path video stream signal, respectively left and right sides two-way encoding code stream is decoded and decoded signal is delivered to three-dimensional display by main video code flow decoding unit and auxilliary video code flow decoding unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 03116541 CN1204757C (en) | 2003-04-22 | 2003-04-22 | Stereo video stream coder/decoder and stereo video coding/decoding system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 03116541 CN1204757C (en) | 2003-04-22 | 2003-04-22 | Stereo video stream coder/decoder and stereo video coding/decoding system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1450816A CN1450816A (en) | 2003-10-22 |
| CN1204757C true CN1204757C (en) | 2005-06-01 |
Family
ID=28684196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 03116541 Expired - Fee Related CN1204757C (en) | 2003-04-22 | 2003-04-22 | Stereo video stream coder/decoder and stereo video coding/decoding system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN1204757C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101361371B (en) * | 2006-01-05 | 2010-11-03 | 日本电信电话株式会社 | Video encoding method, decoding method, device thereof, program thereof, and storage medium containing the program |
Families Citing this family (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100524077B1 (en) * | 2003-11-13 | 2005-10-26 | 삼성전자주식회사 | Apparatus and method of temporal smoothing for intermediate image generation |
| EP1727090A1 (en) | 2004-02-27 | 2006-11-29 | Tdvision Corporation S.A. DE C.V. | Method and system for digital decoding 3d stereoscopic video images |
| US8369406B2 (en) | 2005-07-18 | 2013-02-05 | Electronics And Telecommunications Research Institute | Apparatus of predictive coding/decoding using view-temporal reference picture buffers and method using the same |
| KR100678911B1 (en) * | 2005-07-21 | 2007-02-05 | 삼성전자주식회사 | Method and apparatus for video signal encoding and decoding with extending directional intra prediction |
| KR101245251B1 (en) * | 2006-03-09 | 2013-03-19 | 삼성전자주식회사 | Method and apparatus for encoding and decoding multi-view video to provide uniform video quality |
| WO2007110000A1 (en) * | 2006-03-29 | 2007-10-04 | Huawei Technologies Co., Ltd. | A method and device of obtaining disparity vector and its multi-view encoding-decoding |
| US8970680B2 (en) * | 2006-08-01 | 2015-03-03 | Qualcomm Incorporated | Real-time capturing and generating stereo images and videos with a monoscopic low power mobile device |
| KR101023262B1 (en) * | 2006-09-20 | 2011-03-21 | 니폰덴신뎅와 가부시키가이샤 | Image encoding method, decoding method, device thereof, program thereof, and storage medium containing the program |
| US8274551B2 (en) | 2007-06-11 | 2012-09-25 | Samsung Electronics Co., Ltd. | Method and apparatus for generating header information of stereoscopic image data |
| CN101415114B (en) * | 2007-10-17 | 2010-08-25 | 华为终端有限公司 | Method and apparatus for encoding and decoding video, and video encoder and decoder |
| CN101420609B (en) * | 2007-10-24 | 2010-08-25 | 华为终端有限公司 | Video encoding, decoding method and video encoder, decoder |
| CN101609652B (en) * | 2008-06-17 | 2012-12-19 | 联咏科技股份有限公司 | Transmission interface and method for reducing power consumption and electromagnetic interference effect |
| CN101616322A (en) * | 2008-06-24 | 2009-12-30 | 深圳华为通信技术有限公司 | Stereo video coding-decoding method, Apparatus and system |
| EP2334088A1 (en) * | 2009-12-14 | 2011-06-15 | Koninklijke Philips Electronics N.V. | Generating a 3D video signal |
| EP2355510A1 (en) * | 2009-12-21 | 2011-08-10 | Alcatel Lucent | Method and arrangement for video coding |
| CN102195894B (en) * | 2010-03-12 | 2015-11-25 | 腾讯科技（深圳）有限公司 | The system and method for three-dimensional video-frequency communication is realized in instant messaging |
| CN103190152B (en) * | 2010-10-26 | 2016-04-27 | 韩国放送公社 | For the hierarchical broadcast system and method for three-dimensional broadcast |
| US8786674B2 (en) | 2010-11-26 | 2014-07-22 | Mediatek Singapore Pte. Ltd. | Method for performing video display control within a video display system, and associated video processing circuit and video display system |
| CN104717484B (en) * | 2010-11-26 | 2017-06-09 | 联发科技（新加坡）私人有限公司 | Carry out method, video processing circuits and the video display system of video display control |
| CN102055984B (en) * | 2011-01-27 | 2012-10-03 | 山东大学 | Three-dimensional video decoding structure for smoothly switching 2D and 3D play modes and operating method |
| CN102625097B (en) * | 2011-01-31 | 2014-11-05 | 北京大学 | Method for intra-frame prediction of three-dimensional video and coding and decoding methods |
| CN102118602B (en) * | 2011-03-15 | 2013-08-21 | 深圳市捷视飞通科技有限公司 | Method and system for displaying auxiliary streaming video in multiple pictures |
| CN102196291A (en) * | 2011-05-20 | 2011-09-21 | 四川长虹电器股份有限公司 | Method for coding binocular stereo video |
| CN102244801A (en) * | 2011-07-13 | 2011-11-16 | 中国民航大学 | Digital stereoscopic television system and coding and decoding methods |
| CN102271270A (en) * | 2011-08-15 | 2011-12-07 | 清华大学 | Method and device for splicing binocular stereo video |
| CN102438147B (en) * | 2011-12-23 | 2013-08-07 | 上海交通大学 | Intra-frame synchronous stereo video multi-reference frame mode inter-view predictive coding and decoding method |
| CN103379351B (en) * | 2012-04-28 | 2016-03-02 | 中国移动通信集团山东有限公司 | A kind of method for processing video frequency and device |
| US10567765B2 (en) * | 2014-01-15 | 2020-02-18 | Avigilon Corporation | Streaming multiple encodings with virtual stream identifiers |
| US20170026659A1 (en) * | 2015-10-13 | 2017-01-26 | Mediatek Inc. | Partial Decoding For Arbitrary View Angle And Line Buffer Reduction For Virtual Reality Video |
| US20180098090A1 (en) * | 2016-10-04 | 2018-04-05 | Mediatek Inc. | Method and Apparatus for Rearranging VR Video Format and Constrained Encoding Parameters |
| CN107071385B (en) * | 2017-04-18 | 2019-01-25 | 杭州派尼澳电子科技有限公司 | A kind of method for encoding stereo video introducing parallax compensation based on H265 |
| CN107295214B (en) * | 2017-08-09 | 2019-12-03 | 湖南兴天电子科技有限公司 | Interpolated frame localization method and device |
-
2003
- 2003-04-22 CN CN 03116541 patent/CN1204757C/en not_active Expired - Fee Related
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101361371B (en) * | 2006-01-05 | 2010-11-03 | 日本电信电话株式会社 | Video encoding method, decoding method, device thereof, program thereof, and storage medium containing the program |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1450816A (en) | 2003-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1204757C (en) | Stereo video stream coder/decoder and stereo video coding/decoding system | |
| US8953898B2 (en) | Image processing apparatus and method | |
| US5619256A (en) | Digital 3D/stereoscopic video compression technique utilizing disparity and motion compensated predictions | |
| US5612735A (en) | Digital 3D/stereoscopic video compression technique utilizing two disparity estimates | |
| US8644386B2 (en) | Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method | |
| CN100473157C (en) | System and method for internet broadcasting of mpeg-4-based stereoscopic video | |
| US6055012A (en) | Digital multi-view video compression with complexity and compatibility constraints | |
| KR102343700B1 (en) | Video transmission based on independently encoded background updates | |
| US20140313291A1 (en) | Video coding method, video decoding method, video coder, and video decoder | |
| US20070041443A1 (en) | Method and apparatus for encoding multiview video | |
| US20090190662A1 (en) | Method and apparatus for encoding and decoding multiview video | |
| EP1927250A1 (en) | Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method | |
| US20110134227A1 (en) | Methods and apparatuses for encoding, decoding, and displaying a stereoscopic 3d image | |
| Lim et al. | A multiview sequence CODEC with view scalability | |
| US20100002764A1 (en) | Method For Encoding An Extended-Channel Video Data Subset Of A Stereoscopic Video Data Set, And A Stereo Video Encoding Apparatus For Implementing The Same | |
| CN101867816A (en) | Stereoscopic video asymmetric compression coding method based on human-eye visual characteristic | |
| US20150312547A1 (en) | Apparatus and method for generating and rebuilding a video stream | |
| Hewage et al. | Comparison of stereo video coding support in MPEG-4 MAC, H. 264/AVC and H. 264/SVC | |
| CN109451293B (en) | Self-adaptive stereoscopic video transmission system and method | |
| Hewage et al. | Stereoscopic tv over ip | |
| Yip et al. | Joint source and channel coding for H. 264 compliant stereoscopic video transmission | |
| KR100566100B1 (en) | Apparatus for adaptive multiplexing/demultiplexing for 3D multiview video processing and its method | |
| Willner et al. | Mobile 3D video using MVC and N800 internet tablet | |
| KR101233161B1 (en) | Method for transmission and reception of 3-dimensional moving picture in DMB mobile terminal | |
| Mallik et al. | HEVC based Stereo Video codec |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20050601 Termination date: 20150422 |
|
| EXPY | Termination of patent right or utility model |