CN1254115C - Totally embedded fgs video coding with motion compensation - Google Patents
Totally embedded fgs video coding with motion compensation Download PDFInfo
- Publication number
- CN1254115C CN1254115C CNB018038034A CN01803803A CN1254115C CN 1254115 C CN1254115 C CN 1254115C CN B018038034 A CNB018038034 A CN B018038034A CN 01803803 A CN01803803 A CN 01803803A CN 1254115 C CN1254115 C CN 1254115C
- Authority
- CN
- China
- Prior art keywords
- frame
- surplus
- difference
- coding
- enhancement layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/34—Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/31—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Abstract
A video coding technique having motion compensation within a fine granular scalable coded enhancement layer. In one embodiment, the video coding technique involves a two-loop prediction-based enhancement layer including non-motion-predicted enhancement layer I- and P-frames and motion-predicted enhancement layer B-frames. The motion-predicted enhancement layer B-frames are computed using: 1) motion-prediction from two temporally adjacent differential I- and P- or P- and P- frame residuals, and 2) the differential B-frame residuals obtained by subtracting the decoded base layer B-frame residuals from the original base layer B-frame residuals. In a second embodiment, the enhancement layer further includes motion-predicted enhancement layer P-frames. The motion-predicted enhancement layer P-frames are computed using: 1) motion-prediction from a temporally adjacent differential I- or P-frame residual, and 2) the differential P-frame residual obtained by subtracting the decoded base layer P-frame residual from the original base layer P-frame residual.
Description
Technical field
The present invention relates to video coding, more specifically, but relate to a kind of enhancement-layer video encoding scheme of scale, in enhancement layer, adopt motion compensation for bi-directional predicted frames (B-frame) and predictive frame and bi-directional predicted frames and (P-and B-frame).
Background technology
But being used to be compressed in, the enhancement-layer video of scale coding has the computer network that changes bandwidth, as the vision signal that sends on the internet.But the enhancement-layer video encoding scheme of the coding techniques of general current employing fine granular scale (being adopted by ISO MPEG-4 standard) is shown among Fig. 1.Such just as can be seen, Video Coding Scheme 10 comprises a basic unit 11 based on prediction, by bit rate R
BLCoding and a FGS enhancement layer 12 are pressed R
ELCoding.
Fig. 2 illustrates a kind of being used for the Video Coding Scheme basic unit 11 of Fig. 1 and the common FGS encoder block diagram of enhancement layer 12 codings.Such just as can be seen, the enhancement layer surplus of frame i (FGSR (i)) equals MCR (i)-MCRQ (i), and wherein MCR (i) is the motion compensation surplus of frame i, and MCRQ (i) is in quantification and goes the quantizing process motion compensation surplus of frame i later on.
Although the current FGS enhancement-layer video encoding scheme of Fig. 1 is very flexibly, it has following deficiency, but promptly it to compare with the performance of non-scale encoder under the identical traffic bit rate in the performance aspect the video image quality be lower.Decrease in image quality is not because but the fine granular scale coding of enhancement layer 12 causes, and mainly is because due to the utilization of interim redundancy reduces between the FGS residual frame in the enhancement layer 12.Particularly, the FGS enhancement layer frame of enhancement layer 12 is only from they I-of basic unit separately, and the motion compensation surplus of P-and B-frame obtains, do not have the FGS enhancement layer frame be used to predict in the enhancement layer 12 other the FGS enhancement layer frame or basic unit 11 in other frame.
Therefore, a kind of motion compensation that in enhancement layer, adopts, but need so that when keeping most of flexibility representative and attractive characteristic, improve the scale enhancement-layer video encoding scheme of image quality for current FGS Video Coding Scheme.
Summary of the invention
The present invention be directed to a kind of enhancement-layer video encoding scheme, particularly a kind of motion compensation that adopts in enhancement layer is used to predict FGS enhancement-layer video encoding scheme with bi-directional predicted frames.One aspect of the present invention relates to a kind of method that may further comprise the steps: calculate the difference frame surplus from uncoded video and basic frame, this difference frame surplus is corresponding to the quantization error in the basic frame; The difference frame surplus that will be moved predictive coding and will be by the difference frame surplus of non--moving projection coding between distinguish; Storage will be by at least a portion of each difference frame surplus of non--moving projection coding, and the difference frame surplus of described storage is used as with reference to using; Motion compensation is applied to the difference frame surplus of described storage, produces motion compensation difference frame surplus; With deduct motion compensation difference frame surplus from the difference frame surplus that will be moved predictive coding so that produce the enhancement layer frame of moving projection.
Another aspect of the present invention relates to a kind of method that may further comprise the steps: to the base layer stream decoding, produce base layer video frame; To the enhancement layer stream decoding, produce the difference frame surplus, this difference frame surplus is corresponding to the quantization error in the basic frame; Distinguish in the difference frame surplus that is moved predictive coding with between by the difference frame surplus of non--moving projection coding; Storage is by at least a portion of each difference frame surplus of non--moving projection coding, and the difference frame surplus of described storage is used as with reference to using; Motion compensation is applied to the difference frame surplus of described storage, produces motion compensation difference frame surplus; With motion compensation difference frame surplus with the difference frame surplus addition that has been moved predictive coding so that produce the enhancement layer frame of moving projection; With with the enhancement layer frame of moving projection and the combination of separately basic frame, produce the video that strengthens.
Another aspect of the present invention relates to a kind of medium that is used for video signal coding, but comprises the code that is used for the non-scale of uncoded raw video signal is encoded into basic frame; Be used for from the code of uncoded raw video signal and basic frame calculating difference frame surplus, near small part difference frame surplus is used as reference; Be used for motion compensation be applied to as with reference to use to small part difference frame surplus, to produce the code of the moving of reference-compensation difference frame surplus; Be used for from difference frame surplus separately deduct the moving of reference-compensation difference frame surplus mobile to produce-the prediction enhancement layer frame.
The present invention also relates to a kind of memory medium that is used for the compressing video frequency signal decode with base layer stream and enhancement layer stream in an aspect, comprising: be used for the base layer stream decoding to produce the code of base layer video frame; Be used for enhancement layer stream decoding producing the code of difference frame surplus, to small part difference frame surplus as with reference to using; Be used for motion compensation be applied to as with reference to use to the difference frame surplus of small part to produce the code of the moving of reference-compensation difference frame surplus; Be used for the moving of reference-compensation difference frame surplus and separately difference frame surplus produced mutually the code of mobile-enhancement layer frame of predicting; Be used for move-enhancement layer frame of prediction and the combination of basic frame separately to be to produce the code of the vision signal that strengthens.
Another aspect of the present invention relates to a kind of equipment that is used for video signal coding, and it comprises: but be used for uncoded video is carried out non--scale apparatus for encoding, in order to produce basic frame; Be used for from the device of uncoded video and basic frame calculating difference frame surplus, this difference frame surplus is corresponding to the quantization error in the basic frame; Be used for the difference frame surplus that will be moved predictive coding and will be by the difference frame surplus of non--moving projection coding between the device distinguished; Being used to store will be by the device of at least a portion of each difference frame surplus of non--moving projection coding, and the difference frame surplus of described storage is used as with reference to using; Be used for motion compensation is applied to the difference frame surplus of described storage to produce the device of motion compensation difference frame surplus; Deduct motion compensation difference frame surplus so that produce the device of the enhancement layer frame of moving projection with being used for from the difference frame surplus that will be moved predictive coding.
Another aspect of the present invention relates to a kind of equipment that is used for the compressing video frequency signal decode with base layer stream and enhancement layer stream, and it comprises: be used for the device to base layer stream decoding generation basic frame; Be used for the device to enhancement layer stream decoding generation difference frame surplus, this difference frame surplus is corresponding to the quantization error in the basic frame; Be used in the difference frame surplus that is moved predictive coding and the device of distinguishing between by the difference frame surplus of non--moving projection coding; Be used to store by the device of at least a portion of each difference frame of non--moving projection coding, the difference frame surplus of described storage is used as with reference to using; Be used for motion compensation is applied to the difference frame surplus of described storage to produce the device of motion compensation difference frame surplus; Be used for motion compensation difference frame surplus with the difference frame surplus addition that has been moved predictive coding so that produce the device of the enhancement layer frame of moving projection; With the device that is used for the enhancement layer frame of moving projection and basic frame are separately made up the video that strengthens with generation.
Description of drawings
Describe illustrative embodiment now in conjunction with the drawings in detail and will show advantage of the present invention more fully, character and various additional feature, wherein identical label is represented parts identical in the accompanying drawing:
Fig. 1 illustrates a kind of current enhancement-layer video encoding scheme;
Fig. 2 illustrates a kind of being used for the Video Coding Scheme basic unit of Fig. 1 and the common encoder block diagram of enhancement layer coding;
Fig. 3 A illustrates a kind of enhancement-layer video encoding scheme according to first exemplary embodiment of the present invention;
Fig. 3 B illustrates a kind of enhancement-layer video encoding scheme according to second exemplary embodiment of the present invention;
Fig. 4 illustrates the encoder block diagram of a kind of foundation exemplary embodiment of the present invention, can be used to produce the enhancement-layer video encoding scheme of Fig. 3 A;
Fig. 5 illustrates the encoder block diagram of a kind of foundation exemplary embodiment of the present invention, can be used to produce the enhancement-layer video encoding scheme of Fig. 3 B;
Fig. 6 illustrates the decoder side block diagram of a kind of foundation exemplary embodiment of the present invention, can be used to compressed basic unit and enhancement layer stream decoding that the encoder by Fig. 4 is produced;
Fig. 7 illustrates the decoder side block diagram of a kind of foundation exemplary embodiment of the present invention, can be used to compressed basic unit and enhancement layer stream decoding that the encoder by Fig. 5 is produced; With
Fig. 8 illustrates a kind of exemplary embodiment that is used to realize the system of principle of the present invention.
Embodiment
Fig. 3 A illustrates a kind of enhancement-layer video encoding scheme 30 according to the present invention's first exemplary embodiment.Such as can be seen, Video Coding Scheme 30 comprises one based on the basic unit 31 of prediction and the enhancement layer 32 based on prediction in one two loop.
Basic unit 31 based on prediction comprises the I frame of intraframe coding, and the bi-directional predicted B-frame of the P-frame of the prediction of interframe encode and interframe encode is as shown in the represented common enhancement-layer video scheme of Fig. 1.The I-of basic unit, but P-and B-frame can be encoded with the frame-predictive coding of common non--scale.(the I-frame of basic unit is not moved certainly-predicts).
The enhancement layer 32 based on prediction in two loops comprises the enhancement layer I-and the P-frame of non--moving projection and the enhancement layer B-frame that moves-predict.The enhancement layer I-of non-moving prediction and P-frame normally by from they separately the original I-of basic unit and P-frame surplus deduct them (decoded) I-of basic unit of reconstruction separately and P-frame surplus obtain.
According to the present invention, move-the enhancement layer B-frame of prediction each be utilize following item calculate: 1) from two interim adjacent difference I-and the moving projection (a.k.a. enhancement layer frame) and 2 of P-or P-and P-frame surplus) by deduct the difference B-frame surplus that the decoded B-of basic unit frame surplus obtains from the original B-of basic unit frame surplus.2) difference B-frame surplus and 1) difference between the B-frame moving projection that two interim adjacent motion compensation difference frame surpluses obtain provides the enhancement layer B-frame of the moving projection the enhancement layer 32.No matter be but that the enhancement layer B frame of the moving projection that obtains from this process or the enhancement layer I-of non-moving prediction and P-frame can utilize the coding decoder of any suitable scale to encode, but preferably utilize a kind of fine granular scale (FGS) coding decoder as shown in Fig. 3 A.
Video Coding Scheme 30 of the present invention has been improved the quality of video image, because it has reduced the interim redundancy in the enhancement layer B-frame of enhancement layer 32.Because enhancement layer B-frame takies 66% of the total bit rate budget of enhancement layer in the IBBP map architecture group (GOP) 32, is very limited with only enhancement layer B-frame being implemented the relevant image quality loss of motion compensation for most video sequence.(in common enhancement-layer video encoding scheme, a kind of popular rate controlled is to arrive all enhancement layer I-by the position of distributing equal number in enhancement layer mostly, and P-and B-frame are realized.)
And, point out that this point is important, rate controlled plays an important role for utilizing Video Coding Scheme of the present invention to reach good performance.Yet even the simplest a kind of method, according to Btot=bI*NO._I_ frame+bP*_P_ frame+bB*NO.B_ frame, bI＞bP＞bB wherein is total the position budget Btot that distributes for a GOP provides very good result.Further point out each enhancement layer reference frame that the enhancement layer position/plane of different numbers (not necessarily integer-bit/plane) can be considered for using in the motion compensation loop.And, if desired, in the enhancement layer reference frame, have only some part or frequency need be merged in enhancement layer motion compensation loop.
The current enhancement layer coding scheme of the strong property of the packet loss key of above scheme and Fig. 1 is similar: if mistake occurs in the enhancement layer B-frame of moving projection, this mistake will not propagate into outside the I-or P-frame that the next one receives.The sight of two kinds of packet loss may take place:
If mistake occurs in the enhancement layer B-frame of moving projection, this mistake is limited in this B-frame;
If mistake occurs in enhancement layer I-or the P-frame, mistake will shift out the enhancement layer B-frame that utilizes these enhancement layer frames (two) moving projection as a reference.Then, any one can be deleted in the enhancement layer B-frame of moving projection, and can utilize other not the enhancement layer frame of ruined reference carry out that frame repeats or mistake is offset.
Fig. 4 illustrates the block diagram of the encoder 40 of a kind of foundation exemplary embodiment of the present invention, can be used to produce the enhancement-layer video encoding scheme of Fig. 3 A.Just as can be seen, encoder 40 comprises a base layer encoder 41 and an enhancement layer encoder 42.Base layer encoder 41 is conventional and comprises a mobile estimator 43, from original video sequence be stored in reference frame suitable the memory 44 and produce mobile message (motion-vector and predictive mode).The first motion compensation device 45 in the first motion compensation loop 62 is handled mobile message and is produced basic unit's reference frame (Ref (i)) of motion compensation.First subtracter 46 deducts the reference frame Ref of basic unit (i) of motion compensation from original video sequence, produces the motion compensation surplus MCR (i) of basic frame.Cosine transform (DCT) encoder 47 that the motion compensation surplus MCR (i) of basic frame is dispersed, quantizer 48 and entropy coder 49 are treated as the part from the compressed base layer stream (basic frame) of original video sequence.The mobile message that produces by mobile estimator 43 also by by multiplexer 50 with by first subtracter 46, DCT encoder 47, the part of the base layer stream that quantizer 48 and entropy coder 49 were handled makes up.The motion compensation surplus MCR (i) of the basic frame of the quantification that produces in the output of quantizer 48 is gone to quantize by inverse quantizer 51, carries out inverse dct transform by inverse DCT unit 52 then.This process in the output of inverse DCT 52, produce basic frame the motion compensation surplus quantification and remove quantized version MCRQ (i).The motion compensation surplus MCRQ (i) that the quantification of basic frame and going quantizes and the reference frame Ref of basic unit (i) of their motion compensations are separately produced new reference frame by addition in adder 53, are deposited in first frame memory 44 and are moved estimator 43 and motion compensation device 45 is used to handle other frame.
Again with reference to figure 4, enhancement layer encoder 42, preferably comprise FGS enhancement layer encoder (as shown in Figure 4), comprise one second subtracter 54, poor between the motion compensation surplus MCRQ (i) that calculates the quantification of the motion compensation surplus MCR (i) of basic frame and basic frame and go to quantize, generation difference I-, P-and B-frame surplus FGSR (i), under the situation of I-and P-frame surplus, be enhancement layer I-and P-frame.A kind of frame flow control device 55 is provided for when difference B-frame surplus is handled with the motion compensation of foundation principle of the present invention in enhancement layer, and difference I-and P-frame surplus can be handled in the usual way.Frame flow control device 55 is finished this task, and this is to realize by data flow in the output of second subtracter 54 is flowed out in a different manner according to the type of the frame of second subtracter, 54 outputs.More specifically, difference I-that produces in the output of second subtracter 54 and P-frame surplus are sent to FGS encoder 61 (but or similar scale encoder) by frame control appliance 55, be used to utilize DCT coding and entropy coding to carry out the FGS coding, produce the part (the enhancement layer I-of non-moving prediction and P-frame) of compressed enhancement layer stream by plane DCT scanning routine subsequently.Difference I-that produces in the output of second subtracter 54 and P-frame surplus also are sent to second frame memory 58, and they are used for motion compensation later on therein.The difference B-frame surplus that produces in the output of second subtracter 54 is sent to the 3rd subtracter 60 and second frame memory 58 by frame control appliance 55.The second motion compensation device 59 in the second motion compensation loop 63, re-use mobile message (output of the mobile estimator 43 in the base layer encoder 41) and be stored in difference I-that is used as reference and P-frame surplus in second frame memory 58, produce motion compensation difference (I-and P-or P-and P-) the frame surplus MCFGSR (i) of reference from original video sequence.Note, in the difference I-of each reference and the P-frame surplus only some, for example several planes are required, although can use the difference frame surplus of whole references if desired.The 3rd subtracter 60 produces the enhancement layer B-frame MCFGS (i) of each moving projection by deduct motion compensation difference (I-and P-or P-and P-) the frame surplus MCFGSR (i) of reference from their difference B-frame surplus FGSR (i) separately.Frame flow control device 55 is sent to FGS encoder 61 with the enhancement layer B-frame MCFGS (i) of moving projection, is used to utilize DCT coding and entropy coding by plane DCT scanning routine subsequently to carry out the FGS coding, and they are added on the enhancement layer stream of compression therein.
Should be understood that now basic unit remains constant in the enhancement-layer video encoding scheme of Fig. 3 A.And, enhancement layer I-and P-frame by with the current FGS Video Coding Scheme of Fig. 1 in identical mode sequentially handle, therefore, these frames are not moved prediction in enhancement layer.Under the situation of the enhancement layer B-of moving projection frame, should be understood that now the signal MCFGS that is encoded equals in the enhancement layer of i frame:
MCFGS(i)＝FGSR(i)-MCFGSR(i)＝MCR(i)-MCRQ(i)-MCFGSR(i)
Wherein MCR (i) is in quantification and goes the quantizing process motion compensation surplus of frame i later on, FGSR (i) is equal to the current FGS Video Coding Scheme of Fig. 1 basically, just, FGSR (i) equals MCR (i)-MCRQ (i), and MCFGSR (i) is the motion compensation difference frame surplus for the reference of frame (i).Be noted that enhancement layer B-frame processing method of the present invention only needs an additional motion compensation loop in order to the enhancement layer B-frame of moving projection to be provided in enhancement layer.
Fig. 6 illustrates decoder 70 block diagrams of a kind of foundation exemplary embodiment of the present invention, can be used to the basic unit and the enhancement layer stream decoding of compression that the encoder 40 by Fig. 4 is produced.As can be seen, decoder 70 comprises a base layer decoder device 71 and an enhancement layer decoder 72.Base layer decoder device 71 comprises a contrary multiplexer 75, and it receives the base layer stream of having encoded and became first and second data flow 76 and 77 with flowing against multipath conversion.First data flow 76 that comprises mobile message (motion-vector and moving projection pattern) is applied to the first motion compensation device 78.Basic unit's reference video frame during motion compensation device 78 uses mobile messages and is stored in relevant basic frame memory 79 produces and is applied to P-of basic unit and the B-frame that first of first adder 80 is imported 81 moving projection.The variable length encoder/decoder 83 that second data flow 77 is applied to basic unit quantizes for going for decoding and being applied to inverse quantizer 84.The code that has gone to quantize is applied to inverse DCT decoder 85, and the code that goes therein to quantize is transformed into the surplus video I-of basic unit, P-and B-frame, and they are applied to second input 82 of first adder 80.The basic frame of basic unit's surplus frame of video and the moving projection that produced by motion compensation device 78 is produced the video I-of basic unit by addition in first adder 80, P-and B-frame, and their are deposited in basic frame memory 79 and are exported as the base layer video signal alternatively.
But enhancement layer decoder 72 comprises a FGS bit-plane decoder 86 or similar scale decoder, enhancement layer stream decoding to compression produces difference I-, P-in first and second outputs 73 and 74, with B-frame surplus, they are respectively applied to the first and second frame flow control device 87 and 91.The first and second frame flow control device 87 and 91 can differently be handled difference I-and P-frame surplus with difference B-frame surplus, and this is to realize by data flow in the output 73 and 74 of FGS bit-plane decoder 86 is flowed out in a different manner according to the type of the enhancement layer frame of decoder 86 output.Difference I-and P-frame surplus in first output 73 of FGS bit-plane decoder 86 are sent to enhancement layer frame memory 88 by the first frame control appliance 87, and they are stored and are used for motion compensation later on therein.Difference B-frame surplus in first output 73 of FGS bit-plane decoder 86 is sent to second adder 92 by the first frame control appliance 87 and handles by the mode that will further be explained.
The second motion compensation device 90 re-uses the mobile message that is received by base layer decoder device 71 and the difference I-and the P-frame surplus that are stored in the enhancement layer frame memory 88, produce motion compensation difference (I-and P-or P-and P-) the frame surplus of reference, they are used to predict enhancement layer B-frame.Second adder 92 produces enhancement layer B-frame with motion compensation difference frame surplus and their difference B-frame surplus additions separately of each reference.
The second frame control appliance 91 will second output of FGS bit-plane decoder 86 on 74 enhancement layer I-and the moving projection enhancement layer B-frame sequential in the output 93 of the P-frame (difference I-and P-frame surplus) and second adder 92 be sent to third phase and add device 89.Third phase adds device 89 with enhancement layer I-, P-and B-frame and their the corresponding I-of basic unit, and P-is produced the vision signal that strengthens together mutually with the B-frame.
Fig. 3 B illustrates a kind of enhancement-layer video encoding scheme 100 according to second exemplary embodiment of the present invention.Just as can be seen, be moved except the enhancement layer P-frame based in the enhancement layer of prediction in two loops resembles enhancement layer B-frame the prediction, the Video Coding Scheme 100 of second embodiment is equal to first embodiment of Fig. 3 A basically.
The enhancement layer P-frame quilt of moving projection is by calculating with the similar mode of enhancement layer B-frame, just, following of the enhancement layer P-frame utilization of each moving projection is calculated: 1) carry out moving projection and 2 from interim adjacent difference I-or P-frame surplus) by deduct the difference P-frame surplus that the decoded P-of basic unit frame surplus obtains from the original P-of basic unit frame surplus.2) difference P-frame surplus and the difference between the P-frame moving projection that interim adjacent motion compensation difference frame surplus obtains provide the enhancement layer P-frame of the moving projection the enhancement layer 132.No matter be moving projection enhancement layer P-and the B-frame that obtains from this process, still but the enhancement layer I-frame of non-moving prediction all can utilize the coding decoder of any suitable scale, but preferably a kind of fine granular scale (FGS) coding decoder as shown in Fig. 3 B is encoded.
The Video Coding Scheme 100 of Fig. 3 B is providing further improvement aspect the video image quality.This is because Video Coding Scheme 100 has reduced the P-of enhancement layer 132 and the interim redundancy in the B-frame.
Video Coding Scheme of the present invention can be used alternatingly with the current video encoding scheme of Fig. 1 for the various piece of a video sequence or for each video sequence.In addition, in all three Video Coding Scheme, just, the switching between the Video Coding Scheme of describing among the current video encoding scheme of Fig. 1 and Fig. 3 A and Fig. 3 B can be carried out according to the characteristic of channel, can implement at coding or transmission time.And Video Coding Scheme of the present invention obtains very big gain aspect code efficiency, be to increase complexity limitedly.
Fig. 5 illustrates a kind of encoder 140 block diagrams according to exemplary embodiment of the present invention, can be used to produce the enhancement-layer video encoding scheme of Fig. 3 B.Just as can be seen, except frame flow control device 55 used in encoder 40 was omitted, the encoder 140 of Fig. 5 was equal to the encoder 40 of Fig. 4 (being used to produce the enhancement-layer video encoding scheme of Fig. 3 A) basically.The frame flow control device is dispensable in this encoder 140, because difference I-frame surplus is not handled with motion compensation, does not therefore need to be transmitted difference P-and B-frame surplus respectively in enhancement layer encoder 142.
Thereby, the difference I-frame surplus that produces in the output of second subtracter 54 is sent to FGS encoder 61, is used to utilize DCT coding and entropy coding by plane DCT scanning routine subsequently to carry out the FGS coding to produce the part (the enhancement layer I-frame of non-moving prediction) of the enhancement layer stream of compressing.Difference I-frame surplus also is sent to second frame memory 58 in company with difference P-frame surplus, and they are used for motion compensation later on therein.Difference P-that produces in the output of second subtracter 54 and B-frame surplus also are sent to the 3rd subtracter 60.The second motion compensation device 59 in the second motion compensation loop 63, re-use mobile message (output of mobile estimator 43 in the base layer encoder 41) and be stored in difference I-that is used as reference and P-frame surplus in second frame memory 58, produce reference (I-and P-or P-and P-) the frame surplus MCFGSR (i) that is used for reference motion compensation difference (I or P) the frame surplus MCFGSR (i) of moving projection enhancement layer P-frame and is used for moving projection enhancement layer B-frame from original video sequence.The 3rd subtracter 60 is by from they difference P-or B-frame surplus FGSR (i) motion compensation difference (I or P) or (I-and P-or P-and P-) frame surplus MCFGSR (i) of deducting reference separately.Produce the enhancement layer P-or the B-frame MCFGS (i) of each moving projection.Then, the enhancement layer P-of moving projection and B-frame MCFGS (i) are sent to FGS encoder 61, are used to utilize DCT coding and entropy coding by plane DCT scanning routine subsequently to carry out the FGS coding, and they are added on the enhancement layer stream of compression therein.
As in the Video Coding Scheme of Fig. 3 A, basic unit remains constant in the enhancement-layer video encoding scheme of Fig. 3 B.And, be noted that enhancement layer P-of the present invention and B-frame processing method enhancement layer P-and B-frame for moving projection is provided, in enhancement layer, only need an additional motion compensation loop.
Fig. 7 illustrates a kind of decoder 170 block diagrams according to exemplary embodiment of the present invention, can be used to the basic unit and the enhancement layer stream decoding of compression that the encoder 140 by Fig. 5 is produced.Just as can be seen, except frame flow control device 87 and 91 used in decoder 70 was omitted, the decoder 170 of Fig. 7 was equal to the decoder 70 of Fig. 6 basically.The frame flow control device is not necessary in this decoder 170, because difference I-frame surplus handled with motion compensation, thereby, do not need to be transmitted respectively by difference P-decoded from enhancement layer decoder 172 and B-frame surplus.
Thereby difference I-and P-frame surplus in first output 73 of FGS bit-plane decoder 86 are sent to enhancement layer frame memory 88, and they are stored and are used for motion compensation later on therein.Difference P-and B-frame surplus in second output 74 of FGS bit-plane decoder 86 are sent to second adder 92.Difference I-frame surplus in second output 74 of FGS bit-plane decoder 86 (being enhancement layer I-frame later on) is sent to third phase and adds device 89, and its purpose will further be explained.The second motion compensation device 90 re-uses the mobile message that is received by base layer decoder device 71 and the difference I-and the P-frame surplus that are stored in the enhancement layer frame memory 88, generation 1) motion compensation difference (I-and P-or P-and P-) the frame surplus of reference, they are used to predict enhancement layer B-frame, with 2) motion compensation difference (I-or P-) the frame surplus of reference, they are used to predict enhancement layer P-frame.Second adder 92 is produced enhancement layer B-and P-frame with the motion compensation difference frame surplus of reference mutually with their difference B-frame surplus or P-frame surpluses separately.Third phase adds device 89 with enhancement layer I-, P-and B-frame and their the corresponding I-of basic unit, and P-is produced the vision signal that strengthens together mutually with the B-frame.
Fig. 8 illustrates a kind of exemplary embodiment of system 200, and it can be used to realize principle of the present invention.System 200 can represent a television set, set-top box, desktop, on knee or palmtop computer, personal digital assistant (PDA), video/Image storage equipment, as video tape recorder (VCR), digital video recorder (DVR), TiVO equipment, etc., and the part of these and other equipment or combination.System 200 comprises one or more video/eikongens 201,202, one processors 203 of one or more input-output apparatus and a memory 204.Video/eikongen 201 can represent, for example, and a television receiver, VCR or other video/Image storage equipment.Source 201 can alternately represent to be used for by, for example, a global computer communication network, as the internet, wide area network, metropolitan area network, local area network (LAN), terrestrial broadcast systems, cable system, satellite network, wireless network, or telephone network, and the part of these and other type networks or combination, from one or more networks connections of one or several server receiving video signals.
Input-output apparatus 202, processor 203 can be communicated by letter by communication medium 205 with memory 204.Communication medium 205 can represent, for example, a kind of bus, communication network, the one or more inner of circuit connects circuit card or other equipment, and the part of these and other communication mediums and combination.Inputting video data from source 201 is handled and is carried out by processor 203 according to the one or more software programs that are stored in the memory 204, supplies with display device 206 so that produce output video/image.
In a kind of preferred embodiment, adopt the Code And Decode of the principle of the invention to realize by the computer-readable code of carrying out by system.Code can be deposited in memory 204 or read/download from memory medium such as CD-ROM or floppy disk.In other embodiment, can use hardware circuit to substitute or make up and realize the present invention with software instruction.For example, also can be used as discrete hardware component at the parts shown in Fig. 4-7 implements.
Though described the present invention in the above by specific embodiment, should be appreciated that, be not with the present invention sealing or be confined in this disclosed embodiment.For example, other conversion except DCT may be utilized, and include but not limited to small echo or match tracing.In another example, though in above embodiment, realize motion compensation by re-using from the mobile data of basic unit, other embodiments of the present invention can adopt the additional mobile estimator of in the enhancement layer, and this will need to send additional motion-vector.In another example, other embodiments of the present invention can only adopt motion compensation to the P-frame in enhancement layer.These and every other such modification and change all are considered in the scope of appended claim.
Claims (20)
1. method to video coding may further comprise the steps:
But with a kind of coding decoder of non--scale to uncoded video coding (41,42) to produce basic frame;
Calculate (54) difference frame surplus from uncoded video and basic frame, this difference frame surplus is corresponding to the quantization error in the basic frame;
The difference frame surplus that will be moved predictive coding and will be by the difference frame surplus of non--moving projection coding between distinguish (55);
Storage (58) will be by at least a portion of each difference frame surplus of non--moving projection coding, and the difference frame surplus of described storage is used as with reference to using;
Motion compensation (59) is applied to the difference frame surplus of described storage, produces motion compensation difference frame surplus; With
Deduct (60) motion compensation difference frame surplus so that produce the enhancement layer frame of moving projection from the difference frame surplus that will be moved predictive coding.
2. method according to claim 1 to video coding, but comprise the coding decoder that utilizes a kind of scale step to the enhancement layer frame coding of moving projection.
3. method according to claim 1 to video coding, but comprise the coding decoder that utilizes a kind of finely particulate scale step to the enhancement layer frame coding of moving projection.
4. method according to claim 1 to video coding, the difference frame surplus that wherein will be moved predictive coding comprises the difference frame surplus of calculating from the Type B basic frame, described Type B basic frame is by two-way moving projection.
5. method according to claim 1 to video coding, the difference frame surplus that wherein will be moved predictive coding comprises the difference frame surplus of calculating from P type basic frame, described P type basic frame is by unidirectional moving projection.
6. method according to claim 1 to video coding, wherein said motion compensation (59) but re-use mobile message from the non--scale coding decoder that produces basic frame.
7. method to compressed video decoder with base layer stream and enhancement layer stream, this method may further comprise the steps:
To base layer stream decoding (71,171), produce base layer video frame;
To enhancement layer stream decoding (72,172), produce the difference frame surplus, this difference frame surplus is corresponding to the quantization error in the basic frame;
Distinguish (87) in the difference frame surplus that is moved predictive coding with between by the difference frame surplus of non--moving projection coding;
Storage (88) is by at least a portion of each difference frame surplus of non--moving projection coding, and the difference frame surplus of described storage is used as with reference to using;
Motion compensation (90) is applied to the difference frame surplus of described storage, produces motion compensation difference frame surplus;
With motion compensation difference frame surplus with the difference frame surplus addition (92) that has been moved predictive coding so that produce the enhancement layer frame of moving projection; With
With the enhancement layer frame of moving projection and the combination of basic frame separately (89), produce the video that strengthens.
One kind according to claim 7 to video frequency decoding method, the difference frame surplus that wherein has been moved predictive coding comprises the difference frame surplus of having calculated from the Type B basic frame, described Type B basic frame is by two-way moving projection.
One kind according to claim 7 to video frequency decoding method, the difference frame surplus that wherein has been moved predictive coding comprises the difference frame surplus of having calculated from P type basic frame, described P type basic frame is by unidirectional moving projection.
One kind according to claim 7 to video frequency decoding method, the step of wherein said application motion compensation (90) re-uses the mobile message from the step of decoding (71, the 171) base layer stream that produces basic frame.
11. an equipment that is used for video coding, this equipment comprises:
Be used for but uncoded video is carried out non--scale apparatus for encoding (41,141), in order to produce basic frame;
Be used for from the device (42,142) of uncoded video and basic frame calculating difference frame surplus, this difference frame surplus is corresponding to the quantization error in the basic frame;
Be used for the difference frame surplus that will be moved predictive coding and will be by the difference frame surplus of non--moving projection coding between the device (55) distinguished;
Being used to store will be by the device (58) of at least a portion of each difference frame surplus of non--moving projection coding, and the difference frame surplus of described storage is used as with reference to using;
Be used for motion compensation is applied to the difference frame surplus of described storage to produce the device (59) of motion compensation difference frame surplus; With
Be used for deducting motion compensation difference frame surplus so that produce the device (60) of the enhancement layer frame of moving projection from the difference frame surplus that will be moved predictive coding.
12., but comprise that the enhancement layer frame that is used for moving projection carries out scale apparatus for encoding (61) according to the equipment to video coding of claim 11.
13., but comprise that the enhancement layer frame that is used for moving projection carries out fine granular scale apparatus for encoding (61) according to the equipment to video coding of claim 11.
14. according to the equipment to video coding of claim 11, the difference frame surplus that wherein will be moved predictive coding comprises the difference frame surplus of calculating from the Type B basic frame, described Type B basic frame is by two-way moving projection.
15. according to the equipment to video coding of claim 11, the difference frame surplus that wherein will be moved predictive coding comprises the difference frame surplus of calculating from P type basic frame, described P type basic frame is by unidirectional moving projection.
16. the equipment according to claim 11 to video coding, wherein use the device (59) of motion compensation but re-use from the mobile message that is used for non--scale apparatus for encoding (41,141) that produces basic frame.
17. an equipment that is used for the compressed video decoder with a base layer stream and an enhancement layer stream, this equipment comprises:
Be used for device (71,171) to base layer stream decoding generation basic frame;
Be used for the device (72,172) to enhancement layer stream decoding generation difference frame surplus, this difference frame surplus is corresponding to the quantization error in the basic frame;
Be used in the difference frame surplus that is moved predictive coding and the device of distinguishing between by the difference frame surplus of non--moving projection coding (87);
Be used to store by the device (88) of at least a portion of each difference frame of non--moving projection coding, the difference frame surplus of described storage is used as with reference to using;
Be used for motion compensation is applied to the difference frame surplus of described storage to produce the device (90) of motion compensation difference frame surplus;
Be used for motion compensation difference frame surplus with the difference frame surplus addition that has been moved predictive coding so that produce the device (92) of the enhancement layer frame of moving projection; With
Be used for the enhancement layer frame of moving projection and basic frame are separately made up the device (89) of the video that strengthens with generation.
18. according to the equipment of video decode to compression of claim 17, the difference frame surplus that wherein has been moved predictive coding comprises the difference frame surplus of having calculated from the Type B basic frame, described Type B basic frame is by two-way moving projection.
19. according to the equipment of video decode to compression of claim 17, the difference frame surplus that wherein has been moved predictive coding comprises the difference frame surplus of having calculated from P type basic frame, described P type basic frame is by unidirectional moving projection.
20. according to the equipment of video decode to compression of claim 17, the device of wherein said application motion compensation (90) re-uses from what produce basic frame and is used to decode the mobile message of device (71,171) of base layer stream.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US23449900P | 2000-09-22 | 2000-09-22 | |
| US60/234,499 | 2000-09-22 | ||
| US09/887,743 | 2001-06-21 | ||
| US09/887,743 US6940905B2 (en) | 2000-09-22 | 2001-06-21 | Double-loop motion-compensation fine granular scalability |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1486574A CN1486574A (en) | 2004-03-31 |
| CN1254115C true CN1254115C (en) | 2006-04-26 |
Family
ID=26928015
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB018038034A Expired - Fee Related CN1254115C (en) | 2000-09-22 | 2001-09-18 | Totally embedded fgs video coding with motion compensation |
Country Status (7)
| Country | Link |
|---|---|
| EP (1) | EP1323316A2 (en) |
| JP (1) | JP2004509581A (en) |
| KR (1) | KR100860950B1 (en) |
| CN (1) | CN1254115C (en) |
| AU (1) | AU2002220558A1 (en) |
| MY (1) | MY126133A (en) |
| WO (1) | WO2002025954A2 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1413142A2 (en) * | 2001-01-08 | 2004-04-28 | Siemens Aktiengesellschaft | Optimal snr scalable video coding |
| KR100621581B1 (en) * | 2004-07-15 | 2006-09-13 | 삼성전자주식회사 | Method for pre-decoding, decoding bit-stream including base-layer, and apparatus thereof |
| CN101416513A (en) * | 2006-01-09 | 2009-04-22 | 诺基亚公司 | System and apparatus for low-complexity fine granularity scalable video coding with motion compensation |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB9022326D0 (en) | 1990-10-15 | 1990-11-28 | British Telecomm | Signal coding |
| JPH04177992A (en) | 1990-11-09 | 1992-06-25 | Victor Co Of Japan Ltd | Picture coder having hierarchical structure |
| GB9206860D0 (en) | 1992-03-27 | 1992-05-13 | British Telecomm | Two-layer video coder |
| CA2126467A1 (en) | 1993-07-13 | 1995-01-14 | Barin Geoffry Haskell | Scalable encoding and decoding of high-resolution progressive video |
| JP3788823B2 (en) | 1995-10-27 | 2006-06-21 | 株式会社東芝 | Moving picture encoding apparatus and moving picture decoding apparatus |
| US5852565A (en) | 1996-01-30 | 1998-12-22 | Demografx | Temporal and resolution layering in advanced television |
| US6148026A (en) | 1997-01-08 | 2000-11-14 | At&T Corp. | Mesh node coding to enable object based functionalities within a motion compensated transform video coder |
-
2001
- 2001-09-18 WO PCT/EP2001/010874 patent/WO2002025954A2/en not_active Application Discontinuation
- 2001-09-18 JP JP2002529033A patent/JP2004509581A/en active Pending
- 2001-09-18 AU AU2002220558A patent/AU2002220558A1/en not_active Abandoned
- 2001-09-18 KR KR1020027006542A patent/KR100860950B1/en not_active IP Right Cessation
- 2001-09-18 CN CNB018038034A patent/CN1254115C/en not_active Expired - Fee Related
- 2001-09-18 EP EP01985317A patent/EP1323316A2/en not_active Ceased
- 2001-09-20 MY MYPI20014417A patent/MY126133A/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| EP1323316A2 (en) | 2003-07-02 |
| KR100860950B1 (en) | 2008-09-30 |
| AU2002220558A1 (en) | 2002-04-02 |
| JP2004509581A (en) | 2004-03-25 |
| CN1486574A (en) | 2004-03-31 |
| WO2002025954A3 (en) | 2002-10-31 |
| KR20020056940A (en) | 2002-07-10 |
| WO2002025954A2 (en) | 2002-03-28 |
| MY126133A (en) | 2006-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1166200C (en) | Hybrid temporal-SNR fnie granular scalability rideo coding | |
| CN1303817C (en) | System and method for encoding and decoding residual signal for fine granular scalable video | |
| CN1636407A (en) | Totally embedded FGS video coding with motion compensation | |
| CN1196340C (en) | Efficient, flexible motion estimation architecture for real time MPEG2 compliant encoding | |
| CN1640145B (en) | Video frequency coding method and device, data stream decoding method and device | |
| US6940905B2 (en) | Double-loop motion-compensation fine granular scalability | |
| Brites et al. | Evaluating a feedback channel based transform domain Wyner–Ziv video codec | |
| KR100556838B1 (en) | Fine granularity scalability encoding and decoding apparatus and method | |
| CN1620818A (en) | Single-loop motion-compensation fine granular scalability | |
| CN1457605A (en) | Improved prediction structures for enhancement layer in fine granular scalability video coding | |
| CN1977539A (en) | Method and apparatus for generating coded picture data and for decoding coded picture data | |
| CN1240092A (en) | Video coding | |
| US7242812B2 (en) | Coding and decoding of video data | |
| WO2005032138A1 (en) | System and method for combining advanced data partitioning and fine granularity scalability for efficient spatio-temporal-snr scalability video coding and streaming | |
| US20060008002A1 (en) | Scalable video encoding | |
| EP1527607A1 (en) | Method and apparatus for performing multiple description motion compensation using hybrid predictive codes | |
| US20060093031A1 (en) | Method and apparatus for performing multiple description motion compensation using hybrid predictive codes | |
| CN1656816A (en) | Improved efficiency fgst framework employing higher quality reference frames | |
| CN1254115C (en) | Totally embedded fgs video coding with motion compensation | |
| Radhakrishnan et al. | Wavelet based video encoder using KCDS. | |
| Zheng et al. | Content-based periodic macroblock for error-resilient transmission of H. 264 video | |
| TANG et al. | Simulation of Wyner-Ziv Video Coding Algorithm Based on Transform Domain | |
| Kodavalla et al. | CHROMA CODING IN DISTRIBUTED VIDEO CODING | |
| Liu et al. | Distributed Video Coding with Novel Coding Mode Decision and Side Information Creation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20060426 Termination date: 20110918 |