Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/46—Embedding additional information in the video signal during the compression process
  • H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

• the present invention relates to the field of video coding and decoding; more particularly, the present invention relates to coding motion and prediction weighting parameters.
• Motion Compensated (Inter-frame) prediction is a vital component of video coding schemes and standards such as MPEG-1/2, and H.264 (or JVT or MPEG AVC) due to the considerable benefit it could provide in terms of coding efficiency.
• motion compensation was performed by primarily, if not only, considering temporal displacement. More specifically, standards such as MPEG- 1, 2, and MPEG-4 consider two different picture types for inter-frame prediction, predictive (P) and bi-directionally predicted (B) pictures. Such pictures are partitioned into a set of non-overlapping blocks, each one associated with a set of motion parameters.
• Figure IA illustrates an example of motion compensation in P pictures.
• Figure IB illustrates an example of motion compensation in B pictures. In the later case, essentially prediction is formed by averaging both predictions from these two references, using equal weighting factors of (V2, 1 A).
• the predicted signal is also scaled and/or adjusted using two new parameters w and o.
• the sample with brightness value J(x, y, t) at position (x, y) at time t is essentially constructed as w * I ⁇ x + dx, y + dy, f) + o, where dx and dy are the spatial displacement parameters (motion vectors).
• dx and dy are the spatial displacement parameters (motion vectors).
• these new weighting parameters could also considerably increase the overhead bits required for representing motion information, therefore potentially reducing the benefit of such strategies.
• Kamikura, et al. in "Global Brightness- Variation Compensation for Video Coding," IEEE Trans on CSVT, vol. 8, pp. 988-1000, Dec. 1998, suggests the usage of only a single set of global parameters (w, o) for every frame. The use or not of these parameters is also signaled at the block level, thereby providing some additional benefit in the presence of local brightness variations.
• the ITU-H.264 (or JVT or ISO MPEG4 AVC) video compression standard has adopted certain weighted prediction tools that could take advantage of temporal brightness variations and could improve performance.
• the H.264 video coding standard is the first video compression standard to adopt weighted prediction (WP) for motion compensated prediction.
• WP weighted prediction
• Motion compensated prediction may consider multiple reference pictures, with a reference picture index coded to indicate which of the multiple reference pictures is used.
• P pictures or P slices
• B pictures two separate reference picture lists are managed, list 0 and list 1.
• B pictures or B slices
• bi-prediction using both list 0 and list 1 is allowed.
• the list 0 and the list 1 predictors are averaged together to form a final predictor.
• the H.264 weighted prediction tool allows arbitrary multiplicative weighting factors and additive offsets to be applied to reference picture predictions in both P and B pictures. Use of weighted prediction is indicated in the sequence parameter set for P and SP slices. There are two weighted prediction modes; explicit mode, which is supported in P, SP, and B slices, and implicit mode, which is supported in B slices only.
• weighted prediction parameters are coded in the slice header.
• a multiplicative weighting factor and an additive offset for each color component may be coded for each of the allowable reference pictures in list 0 for P slices and list 0 and list 1 for B slices.
• the syntax also allows different blocks in the same picture to make use of different weighting factors even when predicted from the same reference picture store. This can be made possible by using reordering commands to associate more than one reference picture index with a particular reference picture store.
• the same weighting parameters that are used for single prediction are used in combination for bi-prediction.
• the final inter prediction is formed for the samples of each macroblock or macroblock partition, based on the prediction type used. For a single directional prediction from list 0
• SampleP Clipl(((SampleP 0 -fF 0 + 2 1 ⁇ '1 ) » LWD) + O 0 ) (1) and for a single directional prediction from list 1,
• SampleP Clipl (((SamplePr W 1 + 2 LWD - ⁇ ) » LWD) + O 7 ) (2) and for bi-prediction,
• SampleP Clipl (((SampleP 0 -JFo + SamplePr ⁇ + 2 LWD ) (3)
• Clipl() is an operator that clips the sample value within the range [0, l «SampleBitDepth - 1], with SampleBitDepth being the number of bits associated with the current sample, WQ and Oo are the weighting factor and offset associated with the current reference in list 0, Wi and Oi are the weighting factor and offset associated with the current reference in list 1, and LWD is the log weight denominator rounding factor, which essentially plays the role of a weighting factor quantizer.
• SamplePo and SaHIpIeP 1 are the list 0 and list 1 initial predictor samples, while SampleP is the final weight predicted sample.
• weighting factors are not explicitly transmitted in the slice header, but instead are derived based on relative distances between the current picture and its reference pictures. This mode is used only for bi-predictively coded macroblocks and macroblock partitions in B slices, including those using direct mode.
• the same formula for bi-prediction as given in the preceding explicit mode section is used, except that the offset values Oo and Oi are equal to zero, and the weighting factors Wo and Wi are derived using the formulas:
• Wi (64 * TD D ) I TD B
• TD D and TD B are the clipped within the range [-128,127] temporal distances between the list 1 reference and the current picture versus the list O reference picture respectively.
• the H.264 video coding standard enables the use of multiple weights for motion compensation, such use is considerably limited by the fact that the standard only allows signaling of up to 16 possible references for each list at the slice level. This implies that only a limited number of weighting factors could be considered. Even if this restriction did not apply, it could be potentially inefficient if not difficult to signal all possible weighting parameters that may be necessary for encoding a picture. Note that in H.264, weighting parameters for each reference are independently coded without the consideration of a prediction mechanism, while the additional overhead for signaling the reference indices may also be significant.
• the encoding method comprises generating weighting parameters for multi-hypothesis partitions, transforming the weighting parameters and coding transformed weighting parameters.
• Figure IA illustrates an example of motion compensation in P pictures.
• Figure IB illustrates an example of motion compensation in B pictures.
• Figure 2 is a flow diagram of one embodiment of an encoding process.
• Figure 3 is a flow diagram of one embodiment of a decoding process.
• Figure 4 illustrates a tree with parent-child relationships for encoding of bi- prediction motion information.
• Figure 5 illustrates another tree with node information that may be encoded.
• Figure 6 is a block diagram of one embodiment of an encoder.
• Figure 7 is a block diagram of one embodiment of a decoder.
• Figure 8 is an example of the relationship between the representation and the true value of the weighting parameter.
• Figure 9 illustrates one embodiment of a flow diagram for a process of using global and local weighting jointly.
• Figure 10 illustrates a transform process of the weighting parameters for coding frames using biprediction.
• Figure 11 illustrates the process for weighting parameters for the multi- hypothesis case.
• Figure 12 illustrates the transform process of the weighting parameters for biprediction with controlled fidelity.
• Figure 13 is a block diagram of an exemplary computer system that may perform one or more of the operations described herein.
• An efficient coding scheme for coding weighting parameters within bi- predicted (or multi-predicted) partitions of a video coding architecture is disclosed. This scheme improves performance of the video coding system and can be used to handle local brightness variations.
• the coding scheme includes a transform process between pairs of weighting parameters associated with each reference for each block that is being subjected to motion compensated prediction. The transform process causes a transform to be applied to the weighting parameters.
• the transformed weighting parameters are signaled for every block using a prediction method and are coded using a zero tree coding structure. This reduces the overhead required for coding these parameters. While the methods described herein are primarily aimed at bi-predictive partitions. Some of the concepts could also apply to uni-predicted partitions.
• the interactions between uni-predicted and bi-predicted partitions are used to further improve efficiency. Special consideration is also made for bi- predictive pictures or slices, where a block may be predicted from a different list or both lists.
• the coding scheme is further extended by combining and considering both global and local weighting parameters. Additional weighting parameters transmitted at the sequence, picture, or slice level are also optionally considered.
• variable granularity dynamic range of such parameters is also taken into account.
• weighting parameters are finely adjustable, which provides additional benefits in terms of coding efficiency.
• This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer.
• a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.
• a machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer).
• a machine-readable medium includes read only memory (“ROM”); random access memory (“RAM”); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.); etc.
• ROM read only memory
• RAM random access memory
• magnetic disk storage media includes magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.); etc.
• Embodiments of the present invention include a coding scheme that uses motion compensation that includes weighted prediction.
• the weighting parameters associated with a prediction are transformed and then coded with the offset associated with the prediction.
• the decoding process is a reverse of the encoding process.
• FIG. 2 is a flow diagram of one embodiment of an encoding process.
• the process is performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is ran on a general purpose computer system or a dedicated machine), or a combination of both.
• the process begins by processing logic generating weighting parameters for multi-hypothesis partitions (e.g., bi-predictive partitions) as part of performing motion estimation (processing block 201).
• the weighting parameters comprise at least one pair of a set of weighting parameters for each partition (block) of a frame being encoded.
• the weights for multiple pairs of transformed weighting parameters sum to a constant.
• different weighting factors for at least two partitions of the frame are used.
• the weighting parameters compensate for local brightness variations.
• the weighted prediction parameters have variable fidelity levels. The variable fidelity levels are either predefined or signaled as part of a bitstream that includes an encoded representation of the transformed weighting parameters.
• one group of weighting parameters have a higher fidelity than those of another group of weighting parameters.
• processing logic applies a transform to the weighting parameters to create transformed weighting parameters (processing block 202).
• transforming the weighting parameters comprises applying a transform to at least two weighting parameters associated with each reference.
• processing logic codes transformed weighting parameters and an offset (processing block 203).
• the transformed weighting parameters are coded with at least one offset using variable length encoding.
• the variable length encoding may use a zero tree coding structure.
• Variable length encoding may comprise Huffman coding or arithmetic coding.
• coding the transformed weighting parameters comprises differentially coding using predictions based on one of a group consisting of transformed coefficients of neighboring partitions or predetermined weighting parameters.
• the predetermined weighting parameter are used when no neighboring partition exists for a prediction.
• the predetermined weighting parameters may be default weighting parameters or globally transmitted weighting parameters.
• FIG. 3 is a flow diagram of one embodiment of a decoding process.
• a decoding process is used to decode information encoded as part of the encoding process of Figure 2.
• the process is performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.
• processing logic may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.
• processing logic decoding a bitstream using variable length decoding to obtain decoded data that includes transformed weighting parameters for multi-hypothesis partitions (e.g., bi-predictive partitions) (processing block 301).
• Variable length decoding may comprise Huffman coding or arithmetic coding.
• processing logic applies an inverse transform to the transformed weighting parameters (processing block 302).
• the inverse transformed weighting parameters comprise at least two weighting parameters associated with a reference.
• processing logic reconstructs frame data using motion compensation in conjunction with inverse transformed weighting parameters (processing block 303).
• Weighted prediction is useful for encoding video data that comprises fades, cross-fades/dissolves, and simple brightness variations such as flashes, shading changes etc.
• Bi-prediction and multi-hypothesis prediction in general
• Conventional bi- prediction tends to use equal weights (i.e. 1 A) for all predictions, hi one embodiment, one signal is more correlated than another, for example due to temporal distance or quantization or other noise introduced/existing within each signal.
• I(x, y, t) weight 0 ⁇ I(x + dx 0 , y + dy 0 , t 0 ) + weight ⁇ • I(x + dx ⁇ , y + dy x , t l ) + offset (5)
• weighty, dx ⁇ , dy & and t k are the weights, horizontal and vertical displacement, and time corresponding to prediction from list k, and offset is the offset parameter.
• the constant c is equal to 1. This could basically also be seen as the computation of the weighted average between all prediction samples.
• N-bit quantized parameters wQo and WQ 1 are used. In one embodiment, N is 6 bits. Using substitution, equation 5 above then becomes equal to:
• I(x,y,t) ⁇ (wQ ti -I(x+ ⁇ x o ,y + dy o ,t o ) + wQ i - I(x + dx 1 ,y + dy ⁇ ,t ⁇ ) + (l « (N-l)j)» N)
• Equation 6 Considering that the relation in equation 6 may hold for a majority of blocks/partitions, in one embodiment, B_wQo may be further modified to « N) finally resulting in equations:
• parameters B_wQ 0 and B wQ 1 are coded as is.
• the parameters B_wQ 0 and B-WQ 1 are differentially coded by considering the values of B_wQo and B-WQ 1 from neighboring bi-predicted partitions. If no such partitions exist, then these parameters could be predicted using default values dB_wQo and dB_wQi (e.g., both values could be set to 0).
• FIG 10 illustrates a transform process of the weighting parameters for coding frames using bi-prediction.
• transform 1001 receives the quantized weighting parameters wQo and WQ 1 and produces transformed weighting parameters B_wQo and B-WQ 1 , respectively.
• These transformed weighting parameters are variable length encoded using variable length encoder 1002 along with the offset. Thereafter, the encoded weighting parameters are sent as part of the bitstream to a decoder.
• the variable length decoder 1103 decodes the bitstream and produces the transformed weighting parameters B_wQ 0 and B_wQ l5 along with the offset.
• Inverse transform 1004 inverse transforms the transformed weighting parameters to produce the quantized version of the weighting parameters wQo and WQ 1 .
• Weighting parameter wQ 0 is multiplied by a motion -compensated sample xo by multiplier 1006 to produce an output which is input into adder 1007.
• Weighting parameter WQ 1 is multiplied by a motion-compensated sample X 1 using multiplier 1005.
• the result from multiplier 1005 is added to the output of multiplier 1006 and to 2 N"J using adder 1007.
• the output of adder 1007 is divided by 2 N by divider 1008. The result of the division is added to the offset using adder 1009 to produce a predicted sample x'.
• the present invention is not limited to the bi-prediction case; it is equally applicable to the multi-hypothesis case where M different hypothesis are used. By again making the assumption that the most likely relationship between the M hypothesis
• FIG 11 illustrates the process for weighting parameters for the multi- hypothesis case.
• the same arrangement as shown in Figure 10 is shown in Figure 11 with the difference that sets of two or more weighting parameters are transformed by transform 1101 and variable length encoder 1102 encodes sets of quantized weighting parameters along with one offset for each set
• variable length decoder 1103 decodes the bitstream to produce sets of transformed weighting parameters.
• Each set of transformed weighting parameters is inverse transformed by an inverse transform 1104, the output of which are a combined to produce a predicted sample x' in the same way as is done in Figure 10.
• Adder 1107 adds the results of all multiplications for all of the quantized weighting parameters.
• the same picture or slice may have a combination of both bi- predicted and uni-predicted partitions.
• a bi-predicted partition it is possible for a bi-predicted partition to have a uni-predicted neighbor using either listO or listl prediction, or even a uni-predicted partition using listX (where X could be 0 or 1) having neighbors using a different list.
• listX where X could be 0 or 1
• motion vectors within a bi-predicted partition could still be correlated with motion vectors of a uni-predicted partition, this is less likely to be the case for motion vectors of different lists, but also of weighting parameters using different number of hypotheses (i.e.
• weights from bi-predicted partitions may have little relationship with weights from uni-predicted partitions, while a weight and motion vector from a uni- predicted partition using list 0, may have little relationship with these parameters from another partition using list 1).
• all weights from a given prediction mode (listO, listl, and bi-predicted) are restricted to be only predicted from adjacent partitions in the same mode, hi one embodiment, if the current partition is a bi-predicted partition, and all R neighbors (NeighcNeighi,... Neighs) are also bi-predicted, then a weighting parameter B wQ k is predicted as:
• B_wQ k f(Neigh o ,...,Neigh R ,k) + dB_wQ k (13)
• /( ) corresponds to the relationship used to determine the prediction for the k-th weighting parameter, hi one embodiment, a function used for/ for example, is the median value of all predictors.
• a function such as the mean, consideration of only the top or left neighbors based on the reference frame they use or/and a predefined cost, etc. However, if Neighj uses a different prediction type, then it may be excluded from this prediction process, hi one embodiment, if no predictions are available, then default values, as previously discussed could be used as a predictor. [0062] hi one embodiment, after prediction, zero tree coding is employed to further efficiently encode the weighting parameters, and in general the motion parameters for a block. Figures 4 and 5 represent two possible trees with the node information that may be encoded .
• FIG. 4 an example parent-child relationship for encoding of bi- prediction motion information is shown, hi this example, nodes are segmented based on lists, with the horizontal and vertical motion vector displacement difference for ListO having a common parent node and the horizontal and vertical motion vector displacement difference for Listl having a common parent node.
• motion is segmented based on motion vector component. In this case, the horizontal motion vector displacement difference for both ListO and Listl have a common parent node, while the vertical motion vector displacement difference for both ListO and Listl have a common parent mode.
• the trees described in Figure 4 and 5 containing both node type data and leaf indices data may be encoded (and subsequently decoded) using tree coding (e.g., zero tree coding) as described in U.S. Patent Application No. 11/172,052, entitled “Method and Apparatus for Coding Positions of Coefficients," filed June 29, 2005.
• tree coding e.g., zero tree coding
• Default values for a given prediction type do not have to be fixed, hi one embodiment, a set of global/default parameters are used and coded such as the ones used by the H.264 explicit mode, or alternatively derive these parameters using a mechanism similar to that of the H.264 implicit mode.
• the method or parameters could be signaled at the picture or slice level.
• FIG. 9 is a flow diagram of one embodiment for a process of using global and local weighting jointly.
• the process is performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.
• the process begins by setting default weighting parameters (processing block 901). hi such a case, there are default scaling and offset parameters for ListO, Listl, and Bipred.
• the default weighting parameters are ⁇ DScale ⁇ ), DOffsetLo ⁇ .
• the weighting parameters are ⁇ DScale ⁇ , DOffsetu ⁇ .
• the default weighting parameters are ⁇ DScal ⁇ Bi LO, DScale B i_ L1 ,DOffset ⁇ .
• processing logic tests whether the block is a bi-prediction block (processing block 903). If so, processing logic transitions to processing block 904 where the processing logic tests whether there are any available bi-prediction predictors. If there are, processing logic computes the weighting parameter predictors using neighbors using a function f b i pred (Neigho. . . . Neigh] . ) (processing block 905) and processing continues to processing block 914.
• processing logic sets the weighting parameters to the default (DScale B i j L O , DScale ⁇ i Lh DOffset ⁇ (processing block 906) and the processing transitions to processing block 914.
• processing block 907 processing logic tests whether the block list is ListO. If it is, processing logic transitions to processing block 908 where the processing logic tests whether there are any available ListO prediction predictors. If there are, processing logic computes the weighting parameter predictors using neighbors f L o(Neigh o . . . . NeighO (processing block 909) and processing continues to processing block 914. If there are no available ListO predictors, processing logic sets the weighting parameters to the default values ⁇ DScale L0 , DOffsetLo ⁇ (processing block 910) and the processing transitions to processing block 914.
• processing logic tests whether there are any available Listl prediction predictors. If there are, processing logic computes the weighting parameter predictors using neighbors fLo(Neigho. . . . Neighi) (processing block 912) and processing continues to processing block 914. If there are no available Listl predictors, processing logic sets the weighting parameters to the default values ⁇ DScale ⁇ , DOffsetu ⁇ (processing block 913) and the processing transitions to processing block 914. [0070] At processing block 914, processing logic decodes the weighting parameters. Thereafter, the weighting parameters predictors are added to the weighting parameters (processing block 915).
• bi-predicted weights could have different dynamic range, while the parameters luma_bipred_weight_type and chroma_bipred_weight_type are used to control the method that is to be used for deriving the weights. More specifically for luma, if luma_bipred_weight__type is set to 0, then the default 1 A weight is used. If 1, then implicit mode is used, while if 2, the uniprediction weights are used instead as is currently done in H.264 explicit mode. Finally, if 3, weighting parameters are transmitted explicitly within this header.
• a weighting parameter is uniformly quantized to N bits
• non uniform quantization may instead be more efficient and appropriate to better handle different variations within a frame.
• a finer fidelity is used for smaller weighting or offset parameters, whereas fidelity is increased for larger parameters.
• encoding a parameter qw ⁇ that provides a mapping to the weighting parameters B_wQk,wQb or even possibly to the differential value dB_wQk is contemplated, using an equation of the form: -(1 «(N-1)) ,..., (1 «(N-1))-1 (14)
• Equation 17 (or the more generic equation 18) allows for increased flexibility in terms of representing the fidelity of the weighting parameters. Note also that the operation a ⁇ x could indicate not only a multiply, but also an integer or floating point divide (i.e. assume that a is less than 1). Figure 8 is an example of the relationship between the representation and the true value of the weighting parameter.
• the parameters are determined by performing a preanalysis of the frame or sequence, generating the distribution of the weighting parameters, and approximating the distribution with g(x) (i.e. using a polynomial approximation).
• Figure 12 illustrates the transform process of the weighting parameters for bi-prediction with controlled fidelity. Referring to Figure 12, the process is the same as in Figure 10, with the exception that each of the transformed weighting parameters output from transform 1001 isn't input directly into variable-length encoder 1002, along with the offset. Instead, they are input into functions, the output of which are input to variable length encoder 1002. More specifically, transform weighting parameter B_wQ 0 is input into function g o (x) to produce XQ. More specifically, transform weighting parameter B wQ 1 is input into function gi(x) to produce X 1 (1202). The offset is also subjected to function g o (x) to produce x 2 (1203).
• decoder 1003 which are xo, X 1 and x 2 , are input into functions g 0 ⁇ '(x) (1204), g/ ⁇ (x) (1205), and g 0 ⁇ '(x) (1206), which produce the transformed weighting parameters in the offset. Thereafter, these are handled in the same manner as shown in Figure 10.
• Figure 6 is a block diagram of one embodiment of an encoder.
• the encoder first divides the incoming bitstream into rectangular arrays, referred to as macroblocks. For each macroblock, the encoder then chooses whether to use intra-frame or inter-frame coding.
• Intra-frame coding uses only the information contained in the current video frame and produces a compressed result referred to as an I-frame.
• Intra-frame coding can use information of one or more other frames, occurring before or after the current frame. Compressed results that only use data from previous frames are called P-frames, while those that use data from both before and after the current frame are called B-frames.
• video 601 is input into the encoder.
• frames of video are input into DCT 603.
• DCT 603 performs a 2D discrete cosign transform (DCT) to create DCT coefficients.
• the coefficients are quantized by quantizer 604.
• the quantization performed by quantizer 604 is weighted by sealer.
• the quantizer sealer parameter QP takes values from 1 to 31. The QP value can be modified in both picture and macroblock levels.
• VLC 605 which generates bitstream 620.
• VLC 605 performs entropy coding by using Huffman coding or arithmetic coding.
• reordering may be performed in which quantized DCT coefficients are zigzag scanned so that a 2D array of coefficients is converted into a ID array of coefficients in a manner well-known in the art. This may be followed by run length encoding in which the array of reordered quantized coefficients corresponding to each block is encoded to better represent zero coefficients.
• each nonzero coefficient is encoded as a triplet (last, run, level), where "last" indicates whether this is the final nonzero coefficient in the block, "ran” signals the preceeding 0 coefficients and "level” indicates the coefficients sign and magnitude.
• a copy of the frames may be saved for use as a reference frame. This is particularly the case of I or P frames.
• the quantized coefficients output from quantizer 604 are inverse quantized by inverse quantizer 606.
• An inverse DCT transform is applied to the inverse quantized coefficients using IDCT 607.
• the resulting frame data is added to a motion compensated predication from motion compensation (MC) unit 609 in the case of a P frame and then the resulting frame is filtered using loop filter 612 and stored in frame buffer 611 for use as a reference frame, hi the case of I frames, the data output from IDCT 607 is not added to a motion compensation prediction from MC unit 609 and is filtered using loop filter 612 and stored in frame buffer 611.
• the P frame is coded with interprediction from previous a I or P frame, which is referred to commonly in the art as the reference frame, hi this case, the interprediction is performed by motion estimation (ME) block 610 and motion compensation unit 609.
• ME motion estimation
• motion estimation unit 610 uses the reference frame from frame store 611 and the input video 601, motion estimation unit 610 searches for a location of a region in the reference frame that best matched the current macro block in the current frame. As discussed above, this includes not only determining a displacement, but also determining weighting parameters (WP) and an offset. The displacement between the current macroblock and the compensation region in the reference frame, along with the weighting parameters and the offset is called the motion vector.
• the motion vectors for motion estimation unit 610 are sent to motion compensation unit 609. At motion compensation unit 609, the prediction is subtracted from the current macroblock to produce a residue macroblock using subtractor 602. The residue is then encoded using DCT 603, quantizer 604 and VLC 605 as described above.
• Motion estimation unit 610 outputs the weighting parameters to VLC 605 for variable length encoding 605.
• the output of VLC 605 is bitstream 620.
• Figure 7 is a block diagram of one embodiment of a decoder. Referring to
• bitstream 701 is received by variable length decoder 702 which performs variable length decoding.
• the output of variable length decoding is sent to inverse quantizer 703 which performs an inverse quantization operation that is the opposite of the quantization performed by quantizer 604.
• the output of inverse quantizer 703 comprise coefficients that are inverse DCT transformed by IDCT 704 to produce image data, hi the case of I frames, the output of IDCT 704 is simply filtered by loop filter 721, stored in frame buffer 722 and eventually output as output 760. hi the case of P frames, the image data output from IDCT 704 is added to the prediction from motion compensation unit 710 using adder 705.
• Motion compensation unit 710 uses the output from variable length decoder 722, which includes the weighting parameters discussed above, as well as reference frames from frame buffer 722.
• the resulting image data output from adder 705 is filtered using loop filter 721 and stored in frame buffer 722 for eventual output as part of output 760.
• Figure 13 is a block diagram of an exemplary computer system that may perform one or more of the operations described herein.
• computer system 1300 may comprise an exemplary client or server computer system.
• Computer system 1300 comprises a communication mechanism or bus 1311 for communicating information, and a processor 1312 coupled with bus 1311 for processing information.
• Processor 1312 includes a microprocessor, but is not limited to a microprocessor, such as, for example, PentiumTM, PowerPCTM, AlphaTM, etc.
• System 1300 further comprises a random access memory (RAM), or other dynamic storage device 1304 (referred to as main memory) coupled to bus 1311 for storing information and instructions to be executed by processor 1312.
• main memory dynamic storage device 1304
• Main memory 1304 also may be used for storing temporary variables or other intermediate information during execution of instructions by processor 1312.
• Computer system 1300 also comprises a read only memory (ROM) and/or other static storage device 1306 coupled to bus 1311 for storing static information and instructions for processor 1312, and a data storage device 1307, such as a magnetic disk or optical disk and its corresponding disk drive.
• ROM read only memory
• data storage device 1307 such as a magnetic disk or optical disk and its corresponding disk drive.
• Data storage device 1307 is coupled to bus 1311 for storing information and instructions.
• Computer system 1300 may further be coupled to a display device 1321, such as a cathode ray tube (CRT) or liquid crystal display (LCD), coupled to bus 1311 for displaying information to a computer user.
• a display device 1321 such as a cathode ray tube (CRT) or liquid crystal display (LCD)
• An alphanumeric input device 1322 may also be coupled to bus 1311 for communicating information and command selections to processor 1312.
• cursor control 1323 such as a mouse, trackball, trackpad, stylus, or cursor direction keys, coupled to bus 1311 for communicating direction information and command selections to processor 1312, and for controlling cursor movement on display 1321.
• bus 1311 Another device that may be coupled to bus 1311 is hard copy device 1324, which may be used for marking information on a medium such as paper, film, or similar types of media.
• hard copy device 1324 Another device that may be coupled to bus 1311 is a wired/wireless communication capability 1325 to communication to a phone or handheld palm device.
• wired/wireless communication capability 1325 to communication to a phone or handheld palm device.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)

Priority Applications (3)

Applications Claiming Priority (3)

Publications (2)

Family

ID=37462886

Family Applications (1)

Country Status (5)

Cited By (22)

Families Citing this family (54)

Family Cites Families (13)

• 2006
  • 2006-05-24 US US11/440,604 patent/US8457203B2/en active Active
  • 2006-05-25 CN CN200680016666XA patent/CN101176350B/zh active Active
  • 2006-05-25 EP EP06771414.7A patent/EP1891810B1/en active Active
  • 2006-05-25 JP JP2008508011A patent/JP2008541502A/ja active Pending
  • 2006-05-25 WO PCT/US2006/020633 patent/WO2006128072A2/en not_active Ceased
• 2010
  • 2010-12-06 JP JP2010271444A patent/JP5086422B2/ja active Active

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events