KR101037834B1 - Coding and decoding for interlaced video - Google Patents

Abstract

See (1) hybrid motion vector prediction for interlaced forward prediction field, (2) using motion vector block pattern, (3) choosing between dominant and non-dominant polarity for motion vector predictor, (4) Joint coding and decoding of field selection information and differential motion vector information, (5) co-coding / decoding of macroblock mode information for macroblocks of interlaced forward prediction fields, (6) interlaced forward prediction Various techniques for coding and decoding interlaced video, including using a signal of the number of reference fields available for the field, and (7) deriving chroma motion vectors for macroblocks of the interlaced forward prediction field. And tools are described. These various techniques and tools can be used in combination or independently.

Interlaced video, encoding, decoding, motion vectors, motion compensation, macroblocks

Description

Coding and decoding of interlaced video {CODING AND DECODING FOR INTERLACED VIDEO}

Copyright permission

Portions of the disclosure of this patent document contain copyrighted material. The copyright owner does not dispute the facsimile reproduction of this patent disclosure by anyone, as the disclosure appears in a patent office file or a record, but otherwise reserves all copyrights.

The present invention relates to techniques and tools for interlaced video coding and decoding.

Digital video consumes large amounts of storage and transmission capacity. Typical raw digital video sequences include 15 or 30 frames per second. Each frame may contain tens or hundreds of thousands of pixels (also called pels), each pixel representing a small element of a picture. In the raw form, a computer typically represents a pixel as a series of three samples (24 bits total). For example, a pixel is a luma sample because an 8-bit luminance sample that defines the grayscale component of the pixel (as the terms "luminance" and "luma" are used interchangeably herein). And two 8-bit chrominance samples (also referred to herein) as defining the color components of a pixel (as the terms "chrominance" and "chroma" are used interchangeably herein), (also called a choroma sample). Thus, the number of bits per second, ie, the bit rate, of a typical raw digital video sequence may be at least 5 million bits per second.

Many computers and computer networks lack the resources to process raw digital video. For this reason, engineers use compression (also known as coding or encoding) to reduce the bit rate of digital video. Compression reduces the cost of storing and transmitting video by converting the video to a lower bit rate form. Decompression (also known as decoding) reconstructs the version of the original video from the compressed form. "Codec" refers to an encoder / decoder system. Compression can be lossless where the quality of the video does not degrade, but the reduction in bit rate is limited by the inherent degree of variability (sometimes called entropy) of the video data. Alternatively, compression may be lossy with poor quality of video, but the reduction in achievable bit rate is more dramatic. Lossy compression is often used in combination with lossless compression-lossy compression achieves an approximation of information and lossless compression is applied to express that approximation.

In general, video compression techniques include "intra-picture" compression and "inter-picture" compression, where a picture is, for example, a progressively scanned video frame, Interlaced video frames (with alternating lines to video fields), or interlaced video fields. In the case of progressive frames, intra-picture compression techniques compress individual frames (commonly referred to as I-frames or key frames), and inter-picture compression techniques involve previous and / or next frames (usually Compress a frame (generally called a predictive frame, P-frame, or B-frame) with reference to a reference frame or anchor frame or frames (for B-frames) do.

Inter-picture compression techniques often use motion estimation and motion compensation. For example, for motion estimation, the encoder splits the current prediction frame in units of 8x8 or 16x16 pixels. For the units of the current frame, similar units within the reference frame are searched for use as predictors. The motion vector indicates the position of the predictor in the reference frame. In other words, the motion vector for the unit of the current frame indicates the displacement between the spatial position of the unit in the current frame and the predictor's spatial position in the reference frame. The encoder calculates the sample-by-sample difference between the current unit and the predictor to determine a residual (also called an error signal). If the current unit size is 16x16, the residual is divided into four 8x8 blocks. For each 8x8 residual, the encoder applies a reversible frequency conversion operation that generates a series of frequency domain (ie, spectral) coefficients. Discrete cosine transform ["DCT"] is a type of frequency transform. The resulting block of spectral coefficients is quantized and entropy encoded. If the predictive frame is used as a reference for subsequent motion compensation, the encoder reconstructs the predictive frame. When reconstructing the residuals, the encoder reconstructs the quantized transform coefficients (eg, DCT coefficients) and performs a frequency inverse transform, such as inverse DCT [“IDCT”]. The encoder performs motion compensation to calculate the predictor and combines the predictor with the residual. During decoding, decoders generally perform similar operations to entropy decode information, reconstruct residuals, perform motion compensation, and synthesize predictors with residuals.

I. Windows Media Video  In versions 8 and 9 Inter  compression( Inter Compression )

Microsoft's Windows Media Video, version 8 ["WMV8"] includes a video encoder and a video decoder. WMV8 encoders use intra and inter compression, and WMV8 decoders use intra and inter decompression. Windows Media Video, version 9 ["WMV9"], uses a similar architecture for many operations.

Inter compression in the WMV8 encoder uses block-based motion-compensated predictive coding followed by transform coding of residual error. 1 and 2 show block-based inter compression for predictive frames in a WMV8 encoder. Specifically, FIG. 1 shows the motion estimation for the prediction frame 110, and FIG. 2 shows the compression of the prediction residual for the motion-compensated block of the prediction frame.

For example, in FIG. 1, the WMV8 encoder calculates a motion vector for macroblock 115 in prediction frame 110. To calculate the motion vector, the encoder searches in the search region 135 of the reference frame 130. Within the search region 135, the encoder compares the macroblock 115 from the prediction frame 110 with various candidate macroblocks to find candidate macroblocks that are well matched. The encoder outputs information defining a motion vector (entropy coded) for the matching macroblock.

Since the motion vector value is often correlated with the values of the spatially surrounding motion vectors, the compression of the data used to transmit the motion vector information determines or selects the motion vector predictor from neighboring macroblocks and Can be achieved by predicting the motion vector for the current macroblock. The encoder can encode the difference between the motion vector and the motion vector predictor. For example, the encoder calculates the difference between the horizontal component of the motion vector and the horizontal component of the motion vector predictor, calculates the difference between the vertical component of the motion vector and the vertical component of the motion vector predictor, and encodes the differences.

After reconstructing the motion vector by adding this difference to the motion vector predictor, the decoder predicts for macroblock 115 using information from reference frame 130, which is a previously reconstructed frame available at the encoder and decoder. Use motion vectors to compute macroblocks. Prediction is rarely perfect, so the encoder usually encodes a block of pixel differences (also called an error or residual block) between the prediction macroblock and the macroblock 115 itself.

2 shows an example of the calculation and encoding of error block 235 in a WMV8 encoder. Error block 235 is the difference between prediction block 215 and original current block 225. The encoder applies a discrete cosine transform ["DCT"] 240 to the error block 235, resulting in an 8x8 block 245 of coefficients. The encoder then quantizes the DCT coefficients 250, resulting in an 8 × 8 block 255 of quantized DCT coefficients. The encoder scans 260 an 8x8 block 255 to produce a one-dimensional array 265 with coefficients generally ordered from lowest frequency to highest frequency. The encoder uses a variant of run length coding 270 to entropy encode the scanned coefficients. The encoder selects an entropy code from one or more run / level / last tables 275 and outputs this entropy code.

3 shows an example of a corresponding decoding process 300 for an inter-coded block. 3, the decoder decodes 310 the entropy-coded information representing the prediction residual using variable length decoding 310 and run length decoding 320 with one or more run / level / last table 315. , 320). The decoder inversely scans 330 a one-dimensional array 325 that stores entropy-decoded information into a two-dimensional block 335. The decoder dequantizes this data and inverse discrete cosine transform (both represented by 340), resulting in a reconstructed error block 345. In a separate motion compensation path, the decoder calculates the prediction block 365 using the motion vector information 355 for the displacement from the reference frame. The decoder combines 370 the prediction block 365 with the reconstructed error block 345 to form a reconstructed block 375.

II . Interlaced  Video and Sequential Video

The video frame includes lines of spatial information of the video signal. In the case of sequential video, these lines contain samples starting from one moment and continuing through successive lines in a raster scan to the bottom of the frame. Sequential I-frames are intra-coded sequential video frames. Sequential P-frames are sequential video frames coded using forward prediction, and sequential B-frames are sequential video frames coded using bidirectional prediction.

The main aspect of interlaced video is that raster scans of the entire video frame are performed in two passes by scanning alternating lines in each pass. For example, the first scan consists of even lines of the frame and the second scan consists of odd lines of the scan. As a result, each frame contains two fields representing two different time epochs. 4 shows an interlaced video frame 400 that includes an upper field 410 and a lower field 420. In frame 400, the even numbered lines (upper field) are scanned starting at one time (e.g., time t), and the odd numbered lines (lower field) are scanned at another (typically, later) time (e.g., For example, scanning starts at time t + 1). This timing can produce a sawtooth feature in the regions of the interlaced video frame, where there is motion when the two fields start and scan at different times. Because of this, interlaced video frames can be rearranged according to the field structure, with odd lines grouped together in one field and even lines grouped together in another field. This configuration, called field coding, is useful for high-motion pictures for reducing such saw blade-like artifacts. In the static domain, on the other hand, image details in interlaced video frames can be preserved more efficiently without such a configuration. Thus, frame coding is often used in static or low-motion interlaced video frames, where the original alternating field line configuration is preserved.

A typical sequential video frame consists of one frame of content with non-alternating lines. In contrast to interlaced video, sequential video does not divide the video frame into separate fields, and the entire frame is scanned from left to right, top to bottom, starting at a single time.

III . WMV  Previous coding and decoding at encoders and decoders

Previous software in WMV encoders and decoders released in executable form used coding and decoding of sequential and interlaced P-frames. Although encoders and decoders are efficient for many different encoding / decoding scenarios and many different types of content, there are some room for improvement.

A. Reference for Motion Compensation Picture

The encoder and decoder use motion compensation for the sequential forward prediction field and the interlaced forward prediction field. For sequential P-frames, motion compensation is for a single reference frame that is a previously reconstructed I-frame or P-frame immediately preceding the current P-frame. Since we know the reference frame for the current P-frame and only one reference frame is possible, there is no need for information used to select between multiple reference frames.

Macroblocks for interlaced P-frames may be field-coded or frame-coded. In a field-coded macroblock, up to two motion vectors are associated with the macroblock, one for the upper field and the other for the lower field. In a frame coded macroblock, at most one motion vector is associated with the macroblock. For frame coded macroblocks in interlaced P-frames, motion compensation is for a single reference frame that is a previously reconstructed I-frame or P-frame immediately preceding the current P-frame. For field-coded macroblocks in interlaced P-frames, motion compensation is still for a single reference frame, but only the lines of the upper field of the reference frame are motion vectors for the upper field of the field-coded macroblock. Is considered, and only the lines of the lower field of the reference frame are considered for the motion vector for the lower field of the field-coded macroblock. In other words, since the reference frame is known and only one reference frame is possible, there is no need for information used to select between multiple reference frames.

In some encoding / decoding scenarios (eg, high bit rate interlaced video with many motions), limiting motion compensation for forward prediction to a single reference can hurt overall compression efficiency.

B. Macroblock Information Signaling

The encoder and decoder use signaling of macroblock information for sequential or interlaced P-frames.

1. Macroblock Information for Sequential P-frames Signaling

Sequential P-frames may be 1MV or mixed 4MV frames. A 1MV sequential P-frame includes a 1MV macroblock. The 1MV macroblock has one motion vector to indicate the displacement of the prediction block for all six blocks in the macroblock. Mixed-MV sequential P-frames include 1MV and / or 4MV macroblocks. A 4MV macroblock has zero to four motion vectors, each motion vector for one of up to four luminance blocks of the macroblock. Macroblocks in a sequential P-frame may be one of three possible types, 1MV, 4MV and skipped. In addition, the 1MV and 4MV macroblocks can be intra coded. The macroblock type is represented by a combination of a picture and a macroblock layer element.

Thus, 1MV macroblocks can occur in 1MV and mixed-MV sequential P-frames. A single motion vector data MVDATA element is associated with every block in the 1MV macroblock. MVDATA signals whether blocks are coded as intra type or inter type. If the blocks are coded as inter type, MVDATA also indicates motion vector difference.

If the sequential P-frame is 1MV, all macroblocks within that frame are 1MV macroblocks, and thus there is no need to signal the macroblock type separately. If the sequential P-frame is mixed-MV, the macroblocks in that frame may be 1MV or 4MV. In this case, for each macroblock in the frame, the macroblock type (1MV or 4MV) is signaled by the bitplane in the picture layer in the bitstream. The decoded bitplane represents the 1MV / 4MV state for the macroblock as the 1-bit value plane from the upper left to the lower right in raster scan order. A value of 0 indicates that the corresponding macroblock is coded in 1MV mode. A value of 1 indicates that the corresponding macroblock is coded in 4MV mode. In one coding mode, 1MV / 4MV state information is signaled per macroblock in the macroblock layer of the bitstream (not as a plane for sequential P-frames).

4MV macroblocks occur in mixed-MV sequential P-frames. Individual blocks within a 4MV macroblock may be coded as intra blocks. For each of the four luminance blocks of the 4MV macroblock, the intra / inter status is signaled by that block and the associated block motion vector data BLKMVDATA element. For 4MV macroblocks, the coded block pattern CBPCY element indicates which block has the BLKMVDATA element present in the bitstream. The inter / intra state for the chroma block is derived from the luminance inter / intra state. If two or more of the luminance blocks are coded as intra, then the chroma block is also coded as intra.

In addition, the skipped / not skipped status of each macroblock in a frame is also signaled by the bitplane for the sequential P-frame. The skipped macroblock may still have relevant information for hybrid motion vector prediction.

CBCPY is a variable length code ["VLC"] that is decoded into a 6-bit field. CBPCY appears at different positions in the bitstream for 1MV and 4MV macroblocks and has different semantics for 1MV and 4MV macroblocks.

CBPCY indicates that (1) MVDATA indicates that macroblocks are inter-coded, and (2) MVDATA contains coefficient information (indicated by the "last" value decoded from MVDATA) at least one block of the 1MV macroblock. If present, it exists in the 1MV macroblock layer. If CBPCY is present, it is decoded into a 6-bit field indicating which of the corresponding six blocks contains at least one non-zero coefficient.

CBPCY is always present in the 4MV macroblock layer. The CBPCY bit position (bits 0 to 3) for the luminance block has a slightly different meaning than the bit position (bits 4 and 5) for the chroma block. For the bit position for the luminance block, 0 indicates that the corresponding block does not contain motion vector information or any non-zero coefficient. For such a block, BLKMVDATA does not exist, the predicted motion vector is used as the motion vector, and there is no residual data. If the motion vector predictor indicates that hybrid motion vector prediction is used, there is a single bit indicating the motion vector predictor candidate to use. One in the bit position for the luminance block indicates that BLKMVDATA is present for that block. BLKMVDATA indicates whether the block is inter or intra, and if the block is inter, it indicates a motion vector difference. BLKMVDATA also indicates whether there is coefficient data (with the "last" value decoded from BLKMVDATA) for that block. For bit positions for chroma blocks, 0 or 1 indicates whether the corresponding block contains non-zero coefficient information.

The encoder and decoder use code table selection for the VLC table of MVDATA, BLKMVDATA and CBPCY, respectively.

2. Interlaced  Of macroblock information for a P-frame Signaling

Interlaced P-frames may have a mixture of frame-coded macroblocks and field-coded macroblocks. In a field-coded macroblock, up to two motion vectors are associated with the macroblock. In a frame-coded macroblock, at most one motion vector is associated with the macroblock. If the sequence layer element INTERLACE is 1, the picture layer element INTRLCF is present in the bitstream. INTRLCF is a 1-bit element indicating the mode used to code the macroblocks within that frame. If INTRLCF = 0, all macroblocks in the frame are coded in frame mode. If INTRLCF = 1, the macroblock can be coded in field mode or frame mode, and the bitplane INTRLCMB present in the picture layer indicates the field / frame coding state for each macroblock in the interlaced P-frame.

Macroblocks in interlaced P-frames can be one of three possible types: frame-coded, field-coded and skipped. The macroblock type is represented by a combination of picture and macroblock layer elements.

A single MVDATA is associated with every block in the frame-coded macroblock. MVDATA indicates whether the blocks are coded as an intra type or an inter type. If the blocks are coded as inter, MVDATA also indicates motion vector difference.

In the field-coded macroblock, the upper field motion vector data TOPMVDATA element is associated with the upper field block and the lower field motion vector data BOTMVDATA element is associated with the lower field block. These elements are signaled in the first block of each field. More specifically, TOPMVDATA is signaled with the left upper field block and BOTMVDATA is signaled with the left lower field block. TOPMVDATA indicates whether the upper field block is intra or inter. If the block is inter, TOPMVDATA also indicates a motion vector difference for the upper field block. Similarly, BOTMVDATA signals the inter / intra state for the lower field block, and potential motion vector difference information for the lower field block. CBPCY indicates which field has a motion vector data element present in the bitstream.

The skipped macroblock is signaled by the SKIPMB bitplane in the picture layer. CBPCY and motion vector data elements are used to specify whether the block has an AC coefficient. If the "last" value decoded from MVDATA indicates that there is data to decode after the motion vector, then CBPCY is present for the frame-coded macroblock of the interlaced P-frame. If CBPCY is present, it is decoded into 6-bit fields, i.e. 1 bit for each of the 4 Y blocks, 1 bit for both U blocks (upper field and lower field), and both V blocks (upper field and lower). Field is 1 bit.

CBPCY is always present for field-coded macroblocks. The CBPCY and two field motion vector data elements are used to determine if an AC coefficient is present in the blocks of the macroblock. The meaning of CBPCY is the same as for frame-coded macroblocks for bits 1, 3, 4 and 5. That is, they indicate whether an AC coefficient exists in the upper right field Y block, the lower right field Y block, the upper / lower U block, and the upper / lower V block, respectively. For bit positions 0 and 2, the meaning is slightly different. 0 at bit position 0 indicates that there is no TOPMVDATA and the motion vector predictor is used as the motion vector for the upper field block. This also indicates that the left upper field block does not contain nonzero coefficients. A 1 in bit 0 indicates that TOPMVDATA is present. TOPMVDATA indicates whether the upper field block is inter or intra and also indicates motion vector difference when the block is inter. When the "last" value decoded from TOPMVDATA is decoded to 1, there is no AC coefficient for the left upper field block, otherwise there is a non-zero AC coefficient for the left upper field block. Similarly, the rule applies to bit position 2 for BOTMVDATA and the left lower field block.

The encoder and decoder use code table selection for the VLC table of MVDATA, TOPMVDATA, BOTMVDATA and CBPCY, respectively.

3. Transfer of Macroblock Information In signaling  Problems

In summary, various information about macroblocks of sequential P-frames and interlaced P-frames is signaled as separate codes (or a combination of codes) in the frame and macroblock layers. This individually signaled information includes the number of motion vectors, macroblock intra / inter status, whether CBPCY is present (eg, has a "last" value for 1MV and frame-coded macroblocks), and motion vectors Whether data is present (eg, having 4MV and CBPCY for field-coded macroblocks). While this signaling provides good overall performance in many cases, it does not properly exploit the statistical dependencies between different signaled information in various common cases. In addition, this does not enable or mention various useful configurations, such as the presence of CBPCY for 4MV macroblocks or the presence of motion vector data for 1MV macroblocks.

In addition, to the extent that the presence or absence of motion vector data is signaled, this signaling requires a confusing redefinition of the conventional role of the CBPCY element. This in turn requires signaling of conventional CBPCY information with different elements (eg BLKMVDATA, TOPMVDATA, BOTMVDATA) not conventionally used for that purpose. And this signaling does not enable or mention various useful configurations such as the presence of coefficient information when there is no motion vector data.

C. Motion Vector Prediction

For motion vectors for macroblocks (or blocks, or fields of macroblocks, etc.) in interlaced or sequential P-frames, the encoder calculates a motion vector predictor based on neighboring motion vectors and moves motion vectors and motions. The motion vector is encoded by calculating the difference between the vector predictors and encoding the difference. The decoder reconstructs the motion vector by calculating the motion vector predictor (ie, based on the neighboring motion vector), decoding the motion vector difference and adding the motion vector difference to the motion vector predictor.

5A and 5B show the positions of macroblocks considered for candidate motion vector predictors for 1MV macroblocks in a 1MV sequential P-frame. Candidate predictors are taken from left, top, and right-top macroblocks except where the macroblock is the last macroblock in the row. In this case, predictor B is taken from the left-top macroblock rather than the right-top. In the special case where a frame is one macroblock wide, the predictor is always predictor A (upper predictor). If predictor A is out of bounds because the macroblock is in the upper row, the predictor is predictor C. Various other rules address other special cases, such as intra-coded predictors.

6A to 10 illustrate the positions of blocks or macroblocks considered for up to three candidate motion vectors relative to the motion vectors of the 1MV or 4MV macroblocks in a mixed-MV sequential P-frame. In the figures, the large square is the macroblock boundary and the small square is the block boundary. In the special case where a frame is one macroblock wide, the predictor is always predictor A (upper predictor). Various other rules address other special cases, such as the top row block, the top row 1MV macroblock, and the intra-coded predictor for the top row 4MV macroblock.

Specifically, FIGS. 6A and 6B show the positions of blocks considered for the candidate motion vector predictor for 1 MV current macroblock in a mixed-MV sequential P-frame. The neighboring macroblocks may be 1MV or 4MV macroblocks. 6A and 6B show the positions of candidate motion vectors assuming neighbors are 4MV (ie, predictor A is the motion vector for block 2 in the macroblock above the current macroblock, and predictor C is Motion vector for block 1 in the macroblock immediately to the left of the current macroblock). If any of the neighbors is a 1MV macroblock, the motion vector predictor shown in Figs. 5A and 5B is taken as the motion vector predictor for the entire macroblock. As FIG. 6B shows, if the macroblock is the last macroblock in the row, predictor B is not the block of the left-top macroblock, but from block 2 in the right-top macroblock as otherwise. Is from 3.

7A-10 illustrate the positions of blocks considered for the candidate motion vector predictor for each of four luminance blocks in a 4MV macroblock of a mixed-MV sequential P-frame. 7A and 7B show the positions of the blocks considered for the candidate motion vector predictor for the block at position 0, and FIGS. 8A and 8B illustrate the candidate motion vector predictors for the block at position 1 9 shows the positions of the blocks considered for the candidate motion vector predictor for the block at position 2, and FIG. 10 shows the positions for the candidate motion vector predictor for the block at position 3 Indicates the location of the blocks. In other words, if the neighbor is a 1MV macroblock, the motion vector predictor for that macroblock is used for the blocks of that macroblock.

If the macroblock is the first macroblock in that row, predictor B for block 0 is treated differently from block 0 for the remaining macroblocks in that row (see FIGS. 7A and 7B). In this case, predictor B is taken from block 3 in the macroblock immediately above the current macroblock, rather than from block 3 in the macroblock at the top left of the current macroblock as otherwise. Similarly, if the macroblock is the last macroblock in the row, the predictor B for block 1 is treated differently (see Figures 8A and 8B). In this case, the predictor is taken from block 2 in the macroblock immediately above the current macroblock and not from block 2 in the macroblock at the top right of the current macroblock as otherwise. In general, if the macroblock is in the first macroblock column, the predictor C for blocks 0 and 2 is set to zero.

If a macroblock of a sequential P-frame is coded as skipped, the motion vector predictor for it is used as the motion vector for the macroblock (or the predictor for that block is used for that block). There may still be a single bit to indicate which predictor to use in hybrid motion vector prediction.

11, 12A and 12B show examples of candidate predictors for motion vector prediction for frame-coded macroblocks and field-coded macroblocks in interlaced P-frames, respectively. 11 shows candidate predictors A, B, and C for the current frame-coded macroblock at an internal position in an interlaced P-frame (not the first or last macroblock in the macroblock row, top). Not in a row). Predictors may be obtained from candidate directions other than those indicated by A, B, and C (eg, in special cases, such as when the current macroblock is the first macroblock or the last macroblock in a row or in an upper row, etc. Self is not available in this case). For the current frame-coded macroblock, the predictor candidate is calculated differently depending on whether the neighboring macroblock is field-coded or frame-coded. For neighboring frame-coded macroblocks, the motion vector for it is simply taken as the predictor candidate. For neighboring field-coded macroblocks, the candidate motion vectors are determined by averaging the upper and lower field motion vectors.

12A and 12B show candidate predictors A, B, and C for the current field in the field-coded macroblock at an internal position in that field. In FIG. 12A, the current field is a lower field, and the lower field motion vector in the neighboring macroblock is used as the candidate predictor. In FIG. 12B, the current field is the upper field, and the upper field motion vector in the neighboring macroblock is used as the candidate predictor. For each field in the current field-coded macroblock, the number of motion vector predictor candidates for each field is up to three, with each candidate having the same field type (eg, top or bottom) as the current field. From) If a neighboring macroblock is frame-coded, the motion vector for it is used as its upper field predictor and lower field predictor. In other words, various special cases (not shown) apply because no predictor is available in this case if the current macroblock is the first macroblock or the last macroblock in the row or in the upper row. If the frame is one macroblock wide, the motion vector predictor is predictor A. If the neighboring macroblock is intra, then the motion vector predictor for it is zero.

13A and 13B show pseudo codes for calculating a motion vector predictor given a series of predictors A, B and C. To select predictors from a series of predictor candidates, the encoder and decoder use a selection algorithm such as a median-of-three algorithm shown in FIG. 13C.

D. For Sequential P-Frames hybrid  Motion vector prediction

Hybrid motion vector prediction is possible for motion vectors of sequential P-frames. For macroblocks or block motion vectors, the motion vector computed in the previous section to determine whether the predictor selection is explicitly coded in the bitstream, regardless of whether the sequential P-frame is 1MV or mixed-MV. Predictors are tested against A and C predictors. If so, the bits indicating whether to use predictor A or predictor C as the motion vector predictor for the motion vector (without using the motion vector example calculated in section C above) are decoded. Hybrid motion vector prediction is not used in motion vector prediction for interlaced P-frames or any interlaced video representation.

14A and 14B illustrate hybrid motion vector prediction for motion vectors of sequential P-frames. In the pseudo code, the variables predictor_pre_x and predictor_pre_y are the horizontal and vertical motion vector predictors respectively calculated in the previous section. The variables predictor_post_x and predictor_post_y are the horizontal and vertical motion vector predictors after the test for hybrid motion vector prediction, respectively.

E. Decoding Motion Vector Difference

For macroblocks or blocks of sequential P-frames, the MVDATA or BLKMVDATA elements signal motion vector difference information. A 1 MV macroblock has a single MVDATA. The 4MV macroblock has 0 to 4 BLKMVDATA elements, the presence of which is indicated by CBPCY.

MVDATA or BLKMVDATA jointly encode three things: (1) the horizontal motion vector differential component, (2) the vertical motion vector differential component, and (3) a binary "last" flag that generally indicates whether a transform coefficient is present. do. Whether the macroblock (or block in case of 4MV) is intra-coded or inter-coded is signaled as one of the motion vector difference possible values. The pseudo code of FIGS. 15A and 15B shows how motion vector difference information, inter / intra type, and last flag information are decoded in case of MVDATA or BLKMVDATA. In this pseudo code, the variable last_flag is a binary flag, the use of which is described in the section on the signaling of macroblock information. The variable intra_flag is a binary flag indicating whether a block or macroblock is intra. The variables dmv_x and dmv_y are the differential horizontal and vertical motion vector components, respectively. The variables k_x and k_y are fixed lengths for the extended range motion vector whose values vary as shown in the table of FIG. 15C. The variable halfpel_flag is a binary value indicating whether 1/2 pixel of 1/4 pixel precision is used for the motion vector, the value of which is set based on the picture layer syntax element. Finally, tables size_table and offset_table are arrays defined as follows:

size_table [6] = {0, 2, 3, 4, 5, 8},

offset_table [6] = {0, 1, 3, 7, 15, 31}.

For frame-coded or field-coded macroblocks of interlaced P-frames, the MVDATA, TOPMVDATA and BOTMVDATA elements are decoded in the same way.

F. Reconstruction and Derivation of Motion Vectors

The luminance motion vector is reconstructed from the encoded motion vector difference information and the motion vector predictor, and the chrominance motion vector is derived from the reconstructed luminance motion vector.

For 1MV and 4MV macroblocks of sequential P-frames, the luminance motion vector is reconstructed by adding the difference to the motion vector predictor as follows.

mv_x = (dmv_x + predictor_x) smod range_x,

mv_y = (dmv_y + predictor_y) smod range_y,

Where smod is a signed modulus operation defined as

A smod b = ((A + b)% 2b)-b,

This ensures that the reconstructed vector is valid.

In a 1MV macroblock, there is a single motion vector for the four blocks that make up the luminance component of the macroblock. If the macroblock is intra, there is no motion vector associated with the macroblock. If the macroblock is skipped, then dmv_x = 0 and dmv_y = 0, so mv_x = predictor_x and mv_y = predictor_y.

Each inter luminance block in a 4MV macroblock has its own motion vector. Thus, there are zero to four luminance motion vectors in the 4MV macroblock. If the 4MV macroblock is skipped or if the CBPCY for the 4MV macroblock indicates that the block is uncoded, there may be a non-coded block in the 4MV macroblock. If the block is not coded, then dmv_x = 0 and dmv_y = 0, so mv_x = predictor_x and mv_y = predictor_y.

For sequential P-frames, the chroma motion vector is derived from the luminance motion vector. In addition, in the case of the 4MV macroblock, the determination of whether to code the chroma block as inter or intra is made based on the state of the luminance block. Chroma vectors are composed of two steps.

In a first step, a nominal chroma motion vector is obtained by properly combining and scaling luminance motion vectors. This scaling is performed so that the half pixel offset takes precedence over the quarter pixel offset. FIG. 16A illustrates a pseudo code for scaling when deriving a chroma motion vector from a luminance motion vector for a 1MV macroblock. 16B illustrates pseudo code for synthesizing and scaling up to four luminance motion vectors when deriving chroma motion vectors for a 4MV macroblock. 13C shows a pseudo code for the median3 () function, and FIG. 16C shows a pseudo code for the median4 () function.

In the second step, sequence level 1-bit elements are used to determine if further rounding of the chroma motion vector is needed. If so, the chroma motion vector at the quarter-pixel offset is rounded up to the nearest full-pixel position.

For frame-coded and field-coded macroblocks of interlaced P-frames, the luminance motion vectors are reconstructed as if they were done for sequential P-frames. In a frame-coded macroblock, there is a single motion vector for the four blocks that make up the luminance component of the macroblock. If a macroblock is intra, there is no motion vector associated with that macroblock. If the macroblock is skipped, then dmv_x = 0 and dmv_y = 0, so mv_x = predictor_x and mv_y = predictor_y. In a field-coded macroblock, each field may have its own motion vector. Thus, there will be zero to two luminance motion vectors in the field-coded macroblock. If the field-coded macroblock is skipped or if the CBPCY for the field-coded macroblock indicates that the field is uncoded, there may be a non-coded field in the field-coded macroblock. If the field is not coded, then dvm_x = 0 and dvm_y = 0, so mv_x = predictor_x and mv_y = predictor_y.

For an interlaced P-frame, the chroma motion vector is derived from the luminance motion vector. In the case of a frame-coded macroblock, there is one chrominance motion vector corresponding to a single luminance motion vector. For field-coded macroblocks, there are two chrominance motion vectors. One for the upper field and one for the lower field, corresponding to the upper and lower field luminance motion vectors. The rules for deriving chroma motion vectors are the same for both field-coded macroblocks and frame-coded macroblocks. This rule depends not on the type of macroblock but on the luminance motion vector. 17 shows a pseudo code for deriving a chroma motion vector from a luminance motion vector for a frame-coded or field-coded macroblock of an interlaced P-frame. Basically, the x component of the chrominance motion vector is scaled 4 times while the y component of the chrominance motion vector remains the same (due to 4: 1: 1 macroblock chroma subsampling). The scaled x component of the chrominance motion vector is also rounded up to the neighboring quarter pixel position. If cmv_x or cmv_y is out of range, it is dragged back into the valid range.

G. Century Reward

For sequential P-frames, the picture layer includes a syntax element that controls the motion compensation mode and intensity compensation for the frame. If intensity compensation is signaled, this is followed by the LUMSCALE and LUMSHIFT elements in the picture layer. LUMSCALE and LUMSHIFT are 6-bit values that specify the parameters used in the intensity compensation process.

When intensity compensation is used for sequential P-frames, the pixels in the reference frame are remapped prior to using them in the motion-compensated prediction for the P-frame. The pseudo code of FIG. 18 shows how the LUMSCALE and LUMSHIFT elements are used to build a lookup table used to remap reference frame pixels. As follows, the Y component of the reference frame is remapped using the LUTY [] table, and the U and V components are remapped using the LUTUV [] table.

, 및

, And

,

,

는 참조 프레임에서의 원래의 루미넌스 픽셀 값이고,

는 참조 프레임에서의 재매핑된 루미넌스 픽셀 값이며,

는 참조 프레임에서의 원래의 U 또는 V 픽셀 값이고,

는 참조 프레임에서의 재매핑된 U 또는 V 픽셀 값이다.here,

Is the original luminance pixel value in the reference frame,

Is the remapped luminance pixel value in the reference frame,

Is the original U or V pixel value in the reference frame,

Is the remapped U or V pixel value in the reference frame.

For interlaced P-frames, the 1-bit picture-layer INTCOMP value signals whether intensity compensation is used for the frame. When intensity compensation is used, it is followed by LUMSCALE and LUMSHIFT elements in the picture layer, where LUMSCALE and LUMSHIFT are 6-bit values that specify the parameters used in the intensity compensation process for the entire interlaced P-frame. The intensity compensation itself is the same as for sequential P-frames.

VI . Standard for Video Compression and Decompression

Apart from previous WMV encoders and decoders, several international standards relate to video compression and decompression. These standards include the Motion Picture Experts Group (MPEG) 1, 2, and 4 standards and H.261, H.262 (another name for MPEG2), H.263, and H.264 standards from the International Telecommunication Union (ITU). do. Encoders and decoders that follow one of these standards generally use motion estimation and compensation to reduce temporal redundancy between pictures.

A. Reference for Motion Compensation Picture

For some standards, motion compensation for forward prediction frames is associated with a single reference frame, which is a previously reconstructed I-frame or P-frame immediately preceding the current forward prediction frame. Since the reference frame for the current forward prediction frame is known and only one reference frame is possible, the information used to select between multiple reference frames is not necessary. See, for example, H.261 and MPEG 1 standards. In some encoding / decoding scenarios (eg, high bit rate interlaced video with many motions), limiting motion compensation for forward prediction to be related to a single reference can impair overall compression efficiency.

The H.262 standard allows interlaced video frames to be encoded as a single frame or as two fields, where frame encoding or field encoding can be adaptively selected frame by frame. For field-based prediction of the current field, motion compensation uses previously reconstructed upper or lower fields [see H262 standard, section 7.6.1 and 7.6.2.1]. The H.262 standard describes the choice of two reference fields for use in motion compensation with the motion vectors for the current field [H.262 standard, section 6.2.5.2, 6.3.17.2 and 7.6.4. Reference]. For a given motion vector (or upper 16x8 half of a macroblock or lower 16x8 half of a macroblock) for a 16x16 macroblock, a single bit is signaled to indicate whether to apply the motion vector to the upper reference field or the lower reference field [ Id . ]. See the H.262 standard for additional details.

In some cases this reference field selection provides some flexibility and prediction improvement in motion compensation, but this has some disadvantages with regard to bit rate. Reference field selection signaling for a motion vector may consume many bits. For example, for a single 720x288 field with 810 macroblocks (each macroblock has 0, 1, or 2 motion vectors), the reference field select bit for the motion vector consumes at most 1620 bits. . No attempt is made to reduce the bit rate of the reference field selection information by predicting which reference field is selected for the individual motion vector. The signal of the reference field selection information is inefficient in terms of pure coding efficiency. In addition, for some scenarios, the information is encoded, but the reference field selection information may consume so many bits that the benefit of prediction improvement from having multiple available references in motion compensation is overshadowed. There is no option to disable the reference field selection to cope with this scenario.

The H.262 standard also describes dual-prime prediction, a prediction mode in which two forward field-based predictions are averaged over 16x16 blocks in an interlaced P-picture [H.262 standard, See section 7.6.3.6].

Macroblocks of interlaced video frames can be frame-coded or field-coded by the MPEG-4 standard [see MPEG-4 standard, section 6.1.3.8]. For field-based prediction of the upper or lower field lines of field-coded sheets, motion compensation uses previously reconstructed upper or lower fields [see MPEG-4 standard, sections 6.3.7.3 and 7.6.2]. . The MPEG-4 standard describes the selection of two reference fields for use in motion compensation [see MPEG-4 standard, section 6.3.7.3]. For a given motion vector for the upper field line or lower field line of the macroblock, a single bit is signaled to indicate whether to apply the motion vector to the upper reference field or the lower reference field [ Id . ]. See the MPEG-4 standard for additional details. This signaling of reference field selection information has a similar problem as described above for H.262.

The H.263 standard describes motion compensation for sequential P-frames, including an optional reference picture selection mode [see H.263 standard, section 3.4.12, Appendix N]. Typically, the most recent previous anchor picture is used for motion compensation. However, when the reference picture selection mode is used, time prediction can be made from pictures other than the most recent reference picture [ Id . ]. This allows the encoder to optimize its video encoding for the conditions of the channel (e.g., stop error propagation due to loss of information needed for reference in inter-frame coding). prone) can improve the performance of real-time video communications over channels [ Id . ]. If used, for a block or slice of a given group within a picture, the 10-bit value indicates the reference used for prediction of the block or slice of that group [ Id . ]. The reference picture selection mechanism described in H.263 is for sequential video and is not intended to improve compression efficiency on its own but is configured to solve the problem of error propagation in error prone channels.

In draft JVT-D157 of the H.264 standard, the inter prediction process for motion-compensated prediction of a block may involve selecting a reference picture from a number of stored previously decoded pictures [JVT- D157, section 0.4.3]. At the picture level, one or more parameters specify the number of reference pictures used to decode the picture [JVT-D157, section 7.3.2.2 and 7.4.2.2]. At the slice level, the number of available reference pictures can vary, and additional parameters can be received to reorder and manage the reference pictures in the list [JVT-D157, section 7.3.3 and 7.4.3]. For a given motion vector (for the macroblock or sub-macroblock portion), if a reference index is present it indicates the reference picture to be used for prediction [JVT-D157, section 7.3.5.1 and 7.4.5.1]. The reference index points to the first, second, third, or other frame or field in the list [ Id . ]. If there is only one active reference picture in the list, there is no reference index [ Id . ]. If there are only two active reference pictures in the list, a single encoded bit is used to indicate the reference index [Id.]. See draft JVT-D157 in the H.264 standard for additional details.

The reference picture selection of the JVT-D157 provides flexibility, thereby improving the prediction for motion compensation. However, the process of managing the reference picture list and signaling the reference picture selection is complex and in some scenarios consumes an inefficient number of bits.

B. Macroblocks Mod  signal

Various standards use different mechanisms for signaling macroblock information. For example, in the H.261 standard, the macroblock header of a macroblock includes a macroblock type MTYPE element signaled as VLC [H.261 standard, section 4.2.3]. The MTYPE element is a prediction mode (intra, inter, inter + MC, inter + MC + loop filtering), whether a quantizer MQANT element exists for the macroblock, whether the motion vector data MVD element exists for the macroblock, the macro Indicates whether a coded block pattern CBP element exists for the block and whether a transform coefficient TCOEFF element exists for the blocks of the macroblock [Id.]. There is an MVD element for every motion-compensated macroblock [Id.].

In the MPEG-1 standard, macroblocks have a macroblock_type element signaled as VLC [MPEG-1 standard, section 2.4.3.6, table B.2a, table B.2a to B.2d, D.6.4.2]. For macroblocks in the forward prediction picture, the macroblock_type element is used to determine whether there is a quantizer scale element for the macroblock, whether forward motion vector data exists for the macroblock, and if there are coded block pattern elements for the macroblock. And whether the macroblock is intra [Id.]. If the macroblock uses forward motion compensation, forward motion vector data is always present [Id.].

In the H.262 standard, macroblocks have a macroblock_type element that is signaled as a VLC [H.261 standard, section 6.2.5.1, 6.3.17.1, and tables B.2 to B.8]. For macroblocks in a forward prediction picture, the macroblock_type element is used to determine whether a quantizer_scale_code element exists for the macroblock, whether forward motion vector data exists for the macroblock, and whether a coded block pattern element exists for the macroblock. , Whether the macroblock is intra, and the scalability option for the macroblock [Id.]. If the macroblock uses forward motion compensation, forward motion vector data is always present [Id.]. A separate code (frame_motion_type or field_motion_type) may further indicate the macroblock prediction type, including the number of motion vectors for the macroblock and the motion vector format [Id.].

In the H.263 standard, macroblocks have a macroblock type and a coded block pattern for chrominance MCBPC elements signaled as VLC [H.263 standard, section 5.3.2, tables 8 and 9, and F.2 ]. The macroblock type provides information about the macroblock (eg inter, inter4V, intra) [Id.]. For coded macroblocks in inter-coded pictures, there is always an MCBPC and coded block pattern for luminance, and the macroblock type indicates whether a quantizer information element is present for the macroblock. The forward motion-compensated macroblock always has motion vector data for the macroblock (or block in case of inter4V type) [Id.]. The MPEG-4 standard similarly defines an MCBPC element signaled as VLC [MPEG-4 standard, section 6.2.7, 6.3.7, 11.1.1].

In JVT-D157, the mb_type element is part of the macroblock layer [JVT-D157, section 7.3.5 and 7.4.5]. mb_type indicates a macroblock type and various related information [Id.]. For example, for a P-slice, the mb_type element may be the type of prediction (intra or forward), the various intra mode coding parameters if the macroblock is intra coded, the macroblock partition (e.g. 16x16, 16x8, 8x16, Or 8x8) and thus the number of motion vectors when the macroblock is predicted forward, and whether the reference picture selection information is present (if the partition is 8x8) [Id.]. The type of prediction and mb_type also together indicate whether a block pattern element coded for the macroblock exists [Id.]. For each 16x16, 16x8, 8x16 partition in the forward motion-compensated macroblock, motion vector data is signaled [Id.]. For a forward prediction macroblock with 8x8 partitions, the sub_mb_type element for every 8x8 partition indicates the type of prediction (intra or forward) for it [Id.]. If an 8x8 partition is predicted forward, sub_mb_type indicates the number of motion vectors for the sub-partition (eg 8x8, 8x4, 4x8 or 4x4) and therefore 8x8 partition [Id.]. For each sub-partition in the forward motion-compensated 8x8 partition, motion vector data is signaled [Id.].

Different standards use various signaling mechanisms for macroblock information. Whatever advantages these signaling mechanisms may have, they also have the following disadvantages. First, they sometimes do not effectively signal the macroblock type, the presence of coded block pattern information, and the presence of motion vector difference information for motion-compensated macroblocks. In practice, these standards generally do not signal the presence or absence of motion vector difference information for motion-compensated macroblocks (or blocks or fields thereof); instead, motion vector difference information is signaled when motion compensation is used. Assume that Finally, these standards are inflexible in determining which code table to use for macroblock mode information.

C. Motion Vector Prediction

H.261, H.262, H.263, MPEG-1, MPEG-4, and JVT-D157 define some form of motion vector prediction, but the details of motion vector prediction vary greatly between standards. Motion vector prediction is the simplest in the H.261 standard, for example, where the motion vector predictor for the motion vector of the current macroblock is the motion vector of the previously coded / decoded macroblock [H.261 standard, section 4.2.3.4]. In various special cases (eg, when the current macroblock is first in a row), the motion vector predictor is zero. Motion vector prediction in the MPEG-1 standard is similar [MPEG-1 standard, section 2.4.4.2 and D.6.2.3].

Other standards (H.262, etc.) prescribe much more complex motion vector predictions, but still generally determine motion vector predictors from a single neighbor [H.262 standard, section 7.6.3]. It is sufficient to determine the motion vector predictor from a single neighbor if the motion is uniform, but in many other cases it is inefficient.

Thus, different standards (H.263, MPEG-4, JVT-D157) determine motion vector predictors from a number of different neighbors with different candidate motion vector predictors [H.263 standard, section 6.1.1; MPEG-4 standard, sections 7.5.5 and 7.6.2, F.2; JVT-D157, section 8.4.1]. They are efficient for more types of motion, but still do not adequately cope with scenarios that show a large variation between different candidate motion vector predictors, indicating discontinuities in the motion pattern.

Please refer to your standards for additional details.

D. Decoding Motion Vector Difference

H.261, H.262, H.263, MPEG-1, MPEG-4 and JVT-D157 prescribe some form of differential motion vector coding and decoding, but the details of this coding and decoding vary greatly between standards. Motion vector coding and decoding is, for example, the simplest in the H.261 standard, where one VLC represents a horizontal difference component and another VLC represents a vertical difference component [H.261 standard, section 4.2.3.4]. Other standards specify more complex coding and decoding for motion vector difference information. Please refer to your standards for additional details.

E. Reconstruction and Derivation of Motion Vectors

In general, motion vectors in H.261, H.262, H.263, MPEG-1, MPEG-4, or JVT-D157 are reconstructed by synthesizing motion vector predictors and motion vector differences. In other words, the details of the reconstruction depend on the standard.

The chrominance motion vector (which is not signaled) is generally derived from the luminance motion vector (which is signaled). For example, in the H.261 standard, luminance motion vectors are bisected and truncated to zero to derive the chrominance motion vectors [H.261 standard, section 3.2.2]. Similarly, luminance motion vectors are bisected to derive chrominance motion vectors in the MPEG-1 standard and JVT-D157 [MPEG-1 standard, section 2.4.4.2; JVT-D157, section 8.4.1.4].

In the H.262 standard, the luminance motion vector is scaled down to the chrominance motion vector by a factor according to the chrominance sub-sampling mode [H.262 standard, section 7.6.3.7].

In the H.263 standard, for a macroblock having a single luminance motion vector for all four luminance blocks, the chrominance motion vector is derived by dividing the luminance motion vector by two and rounding it to the half pixel position [H. 263 standard, section 6.1.1]. For macroblocks with four luminance motion vectors (one per block), the chrominance motion vectors are derived by summing four luminance motion vectors, dividing by eight and rounding them to half pixel positions [H.263 standard] , Section F.2]. Chrominance motion vectors are similarly derived in the MPEG-4 standard [MPEG-4 standard, section 7.5.5 and 7.6.2].

F. Weighted Forecast

Draft JVT-D157 of the H.264 standard describes weighted prediction. The weighted prediction flag for a picture indicates whether weighted prediction is used for the prediction slice in the picture [JVT-D157, section 7.3.2.2 and 7.4.2.2]. When weighted prediction is used for a picture, each prediction slice in the picture has a table of prediction weights [JVT-D157, section 7.3.3, 7.3.3.2, 7.4.3.3 and 10.4.1]. For a table, the denominator of the luma weight parameter and the denominator of the chroma weight parameter are signaled [Id.]. Then, for each reference picture available for the slice, the luma weight flag indicates whether the luma weight and luma offset molecular parameters are signaled for the picture (if signaled, followed by the parameter if signaled), and the chroma weight flag. Indicates whether the chroma weight and chroma offset molecular parameters are signaled for the picture (if signaled, the parameter follows the picture) [Id.]. Molecular weight parameters that are not signaled are given default values associated with the signaled denominator values [Id.]. While JVT-D157 offers some flexibility in signaling weighted prediction parameters, in many scenarios this signaling mechanism is inefficient.

Given the importance of video compression and decompression for digital video, it is not surprising that video compression and decompression is a well-developed field. However, none of the advantages of previous video compression and decompression techniques have the advantages of the following techniques and tools.

In summary, the detailed description is directed to various techniques and tools for coding and decoding interlaced video. These various techniques and tools can be used in combination or independently.

Some of the detailed description is directed to various techniques and tools for hybrid motion vector prediction for interlaced forward prediction fields. The described techniques and tools include, but are not limited to the following.

A tool such as a video encoder or decoder examines the hybrid motion vector prediction condition based at least in part on the predictor polarity signal applicable to the motion vector predictor. For example, the predictor polarity signal is for selecting a dominant polarity or a non-dominant polarity for the motion vector predictor. The tool then determines the motion vector predictor.

Or, a tool such as a video encoder or decoder determines the first derived motion vector predictor for the motion vector of the interlaced forward prediction field. The tool then checks for variation conditions based at least in part on the original derived motion vector predictor and one or more neighboring motion vectors. If this variation condition is met, the tool uses one of the one or more neighboring motion vectors as the final motion vector predictor for the motion vector. Otherwise, the tool uses the first derived motion vector predictor as the final motion vector predictor.

Some of the detailed descriptions relate to various techniques and tools that use motion vector block patterns to signal the presence or absence of motion vector data for macroblocks having multiple motion vectors. The described techniques and tools include, but are not limited to the following.

A tool such as a video encoder or decoder processes the first variable length code representing the first information for the macroblock having a plurality of luminance motion vectors. This first information includes one motion vector data present indicator for each luminance motion vector of the macroblock. The tool also processes a second variable length code representing the second information for the macroblock. This second information includes a plurality of transform coefficient data present indicators for the plurality of blocks of the macroblock.

Alternatively, a tool such as a video encoder or decoder may be used for macroblocks having a luminance motion vector of a first number (where the first number is greater than one), wherein the second number is a first number. Motion vector block pattern consisting of bits). Each of these bits indicates whether the corresponding one of the luminance motion vectors has associated motion vector data signaled in the bitstream. The tool also processes the relevant motion vector data for each of the luminance motion vectors for which the relevant motion vector data is indicated to be signaled in the bitstream.

Some of the details relate to various techniques and tools for choosing between dominant and non-dominant polarities for motion vector predictors. The described techniques and tools include, but are not limited to the following.

Tools such as a video encoder or decoder determine the dominant polarity for the motion vector predictor. The tool processes the motion vector predictor based at least in part on the dominant polarity and processes the motion vector based at least in part on the motion vector predictor. For example, the motion vector is for the current block or macroblock of the interlaced forward prediction field, and the dominant polarity is based at least in part on the polarity of each of the plurality of previous motion vectors for the neighboring block or macroblock.

Or, a tool such as a video encoder or decoder processes information indicating a choice between a predominant polarity and a non predominant polarity for the motion vector predictor and processes the motion vector based at least in part on the motion vector predictor. For example, the decoder determines the dominant polarity and the non-dominant polarity, and then determines the motion vector predictor based at least in part on the information indicating the dominant polarity and the non-dominant polarity and the selection therebetween.

Some of the detailed description is directed to various techniques and tools for joint coding and decoding of reference field selection information and differential motion vector information. The described techniques and tools include, but are not limited to the following.

Tools such as a video encoder or decoder decode variable length codes that jointly represent differential motion vector information and motion vector predictor selection for the motion vector. The decoder then reconstructs the motion vector based at least in part on differential motion vector information and motion vector predictor selection.

Or a tool such as a video encoder or decoder determines the dominant / non-dominant predictor selection for the motion vector. The encoder determines differential motion vector information for the motion vector and jointly codes the dominant / non-dominant predictor selection into the differential motion vector information.

Some of the description is directed to various techniques and tools for co-coding / decoding of macroblock mode information for macroblocks for interlaced forward prediction fields and code table selection. The described techniques and tools include, but are not limited to the following.

Tools such as video encoders or decoders process variable length codes that jointly signal macroblock mode information for macroblocks. The macroblock is motion compensated, and the jointly signaled macroblock mode information includes: (1) macroblock type, (2) whether a coded block pattern exists, and (3) motion vector for the motion compensated macroblock. Contains whether data exists.

Or, a tool such as a video encoder or decoder selects one code table from among a number of available code tables for macroblock mode information for the interlaced forward prediction field. This tool uses the selected code table to process variable length codes that point to macroblock mode information for macroblocks. The macroblock mode information includes (1) macroblock type, (2) whether a coded block pattern exists, and (3) whether motion vector data is present, if applicable for the macroblock type.

Some of the detailed description is directed to various techniques and tools that use signals of the number of reference fields available for interlaced forward prediction fields. The described techniques and tools include, but are not limited to the following.

A tool such as a video encoder or decoder processes the first signal indicating whether the interlaced forward prediction field has one reference field or two possible reference fields for motion compensation. If the first signal indicates that the interlaced forward prediction field has one reference field, the tool processes the second signal that identifies one of the two possible reference fields. On the other hand, if the first signal indicates that the interlaced forward prediction field has two possible reference fields, then for each of the multiple motion vectors for the block and / or macroblock of the interlaced forward prediction field, The third signal can be processed to select between two possible reference fields. The tool then performs motion compensation on the interlaced forward prediction field.

Or, a tool such as a video encoder or decoder processes a signal indicating whether the interlaced forward prediction field has one reference field or two possible reference fields for motion compensation. This tool performs motion compensation on the interlaced forward prediction field. The tool also updates the reference field buffer for subsequent motion compensation without processing additional signals to manage the reference field buffer.

Some of the detailed description is directed to various techniques and tools for deriving chroma motion vectors for macroblocks of interlaced forward prediction fields. The described techniques and tools include, but are not limited to the following.

 A tool, such as a video encoder or decoder, derives a chroma motion vector for a macroblock having one or more luma motion vectors based at least in part on the polarity evaluation of the one or more luma motion vectors. For example, each of the one or more luma motion vectors is an odd or even polarity, and the polarity assessment includes determining which polarity is more common among the one or more luma motion vectors.

Or, a tool such as a video encoder or decoder determines the dominant polarity among the plurality of luma motion vectors for the macroblock. The tool then derives a chroma motion vector for the macroblock based at least in part on one or more of the plurality of luma motion vectors having the dominant polarity.

Additional features and advantages will be apparent from the following detailed description of various embodiments described with reference to the accompanying drawings.

The present application relates to techniques and tools for efficient compression and decompression of interlaced video. Compression and decompression of interlaced video content is enhanced with various techniques and tools specifically designed to address certain characteristics of interlaced video representations. In various described embodiments, the video encoder and decoder may comprise a bitstream comprising different layers or levels (eg, sequence level, frame level, field level, slice level, macroblock level and / or block level). Techniques for encoding and decoding interlaced forward prediction fields along with corresponding signaling techniques for use with format or syntax.

Interlaced video content is commonly used in digital video broadcasting systems via cable, satellite or DSL, and the like. Efficient techniques and tools for compressing and decompressing interlaced video content are an important part of the video codec.

Various alternatives to the implementations described herein are possible. For example, the techniques described with reference to a flowchart diagram may be varied by changing the order of the steps shown in the flowchart, by repeating or omitting certain steps, and the like. As another example, although some implementations are described with respect to a particular macroblock format, other formats may be used. In addition, the techniques and tools described with respect to interlaced forward prediction fields may also be applicable to other types of pictures.

In various embodiments, the encoder and decoder use flags and / or signals in the bitstream. Although specific flags and signals are described, it will be appreciated that this description encompasses other conventions for flags and signals (eg, 0 rather than 1).

Various techniques and tools may be used in combination or independently. Different embodiments implement one or more of the described techniques and tools. Any of the techniques and tools described herein may be used in a video encoder or decoder or in some other systems are not particularly limited to video encoding or decoding.

I. Computing Environment

19 illustrates a generalized example of a suitable computing environment 1900 in which some of the described embodiments may be implemented. Because these techniques and tools can be implemented in a variety of general purpose or dedicated computing environments, computing environment 1900 is not intended to suggest any limitation as to the scope of use or functionality.

Referring to FIG. 19, the computing environment 1900 includes at least one processing unit 1910 and a memory 1920. In Fig. 19, this most basic configuration 1930 is contained within a dashed line. The processing unit 1910 executes computer executable instructions and may be a real or virtual processor. In a multi-processing system, multiple processing units execute computer executable instructions to improve processing power. Memory 1920 may be volatile memory (eg, registers, cache, RAM), nonvolatile memory (eg, ROM, EEPROM, flash memory, etc.), or some combination of the two. Memory 1920 stores software 1980 that implements a video encoder or decoder.

The computing environment may have additional features. For example, computing environment 1900 includes storage 1940, one or more input devices 1950, one or more output devices 1960, and one or more communication connections 1970. Interconnection mechanisms (not shown), such as a bus, controller, or network, interconnect the components of computing environment 1900. In general, operating system software (not shown) provides an operating environment for other software running in computing environment 1900 and coordinates the activities of components of computing environment 1900.

Storage 1940 may be removable or non-removable, and may be used to store magnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or information and any other that may be accessed within computing environment 1900. Media. Storage 1940 stores instructions for software 1980 implementing a video encoder or decoder.

The input device (s) 1950 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing environment 1900. In the case of audio or video encoding, the input device (s) 1950 may include a sound card, video card, TV tuner, or similar device that accepts audio or video input in analog or digital form, or an audio or video sample in a computing environment ( 1900) can be a CD-ROM or CD-RW. Output device (s) 1960 may be a display, printer, speaker, CD-writer, or other device that provides output from computing environment 1900.

Communication connection (s) 1970 enables communication to other computing entities via communication media. The communication medium carries information such as computer executable instructions, audio or video inputs or outputs, or other data in a modulated data signal. A modulated data signal is a signal that is set or changed in such a way that one or more of its characteristics encode information in the signal. By way of example, and not limitation, communication media includes wired or wireless technology implemented with electrical, optical, RF, infrared, acoustic, or other carrier waves.

These techniques and tools may be described in the general context of computer readable media. Computer-readable media can be any available media that can be accessed within a computing environment. By way of example, and not limitation, in computing environment 1900, computer readable media includes memory 1920, storage 1940, communication media, and any combination of the above.

These techniques and tools may be described in the general context of computer-executable instructions, such as those included in program modules, being executed on a target real or virtual processor in a computing environment. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or separated between the program modules as desired in various environments. The computer executable instructions of the program module may be executed within a local or distributed computing environment.

For purposes of explanation, the detailed description uses terms such as "estimation", "compensation", "prediction", and "apply" to describe computer operations in the computing environment. These terms are highly abstraction of the operations performed by the computer and should not be confused with the actions performed by humans. Actual computer operation corresponding to these terms will vary from implementation to implementation.

II . Generalized Video Encoder and Decoder

20 is a block diagram of a generalized video encoder system 2000, and FIG. 21 is a block diagram of a video decoder system 2100, in which various described embodiments may be implemented.

The relationship shown between the modules in the encoder and decoder indicates the main flow of information at the encoder and decoder, and for simplicity no other relationship is shown. In particular, FIGS. 20 and 21 do not represent incidental information indicating encoder settings, modes, tables, etc., which are commonly used for video sequences, frames, macroblocks, blocks, and the like. This incidental information is generally sent to the output bitstream after entropy encoding of the incidental information. The format of the output bitstream may be Windows Media Video version 9 or other format.

Encoder 2000 and decoder 2100 process video pictures, which may be video frames, video fields, or a combination of frames and fields. Bitstream syntax and semantics at the picture and macroblock level may depend on whether a frame or field is used. In addition, there may be changes to the macroblock configuration and overall timing. Encoder 2000 and decoder 2100 are block-based and use a 4: 2: 0 macroblock format for frames, with each macroblock being treated as four 8x8 luminance blocks (sometimes as one 16x16 macroblock). ) And two 8x8 chrominance blocks. For fields, the same or different macroblock configurations and formats may be used. The 8x8 block may be further subdivided at other stages, for example at the frequency transform and entropy encoding stages. Example video frame configurations are described in the following sections.

Depending on the desired implementation and type of compression, the modules of the encoder or decoder may be added, omitted, divided into multiple modules, combined with other modules, and / or replaced with similar modules. In alternative embodiments, encoders or decoders having other modules and / or modules of other configurations perform one or more of the desired techniques.

A. Video Frame Composition

In some implementations, encoder 2000 and decoder 2100 process video frames configured as follows. The frame includes lines of spatial information of the video signal. In the case of sequential video, these lines contain samples starting at one moment and continuing through successive lines to the bottom of the frame. The sequential video frame is divided into macroblocks such as the macroblock 2200 shown in FIG. The macroblock 2200 is in the same place as the four 8x8 luminance blocks Y1 to Y4 and four luminance blocks, but two 8x8 horizontal and vertical half resolutions according to the conventional 4: 2: 0 macroblock format. It contains a chrominance block. The 8x8 block may be further subdivided in other steps, for example in the frequency conversion (eg 8x4, 4x8, or 4x4 DCT) and entropy encoding steps. Sequential I-frames refer to intra-coded sequential video frames. Sequential P-frames refer to sequential video frames coded using forward prediction, and sequential B-frames refer to sequential video frames coded using bidirectional prediction. Sequential P-frames and B-frames may include intra-coded macroblocks 2200 as well as other types of predictive macroblocks.

An interlaced video frame consists of two scans of the frame-one constitutes an even line (upper field) of the frame and the other constitutes an odd line (lower field) of the frame. These two fields may represent two different time periods or they may come from the same time period. FIG. 23A illustrates a portion of an interlaced video frame 2300 that includes alternating lines of the upper field and lower field in the upper left portion of interlaced video frame 2300.

FIG. 23B illustrates the interlaced video frame 2300 of FIG. 23A configured for encoding / decoding as frame 2330. The interlaced video frame 2300 is divided into macroblocks such as macroblocks 2331 and 2332 using a 4: 2: 0 format as shown in FIG. In the luminance plane, each macroblock 2331 and 2332 includes 8 lines from the top field alternating with 8 lines from the bottom field for a total of 16 lines, each line being 16 pixels long. (The actual configuration and placement of the luminance block and chrominance block within macroblocks 2331 and 2332 are not shown and may actually differ for other encoding decisions.) Within a given macroblock, the upper field information and The bottom field information can be coded jointly or separately at any of a variety of steps. An interlaced I-frame is two intra-coded fields of an interlaced video frame, where the macroblock contains information about these two fields. Interlaced P-frames are two fields of an interlaced video frame coded using forward prediction, and interlaced B-frames are two fields of an interlaced video frame coded using bidirectional prediction, where macroblocks Contains information about these two fields. Interlaced P-frames and B-frames may include intra-coded macroblocks as well as other types of predictive macroblocks.

FIG. 23C illustrates the interlaced video frame 2300 of FIG. 23A configured for encoding / decoding as field 2360. Each of the two fields of interlaced video frame 2300 is divided into macroblocks. The upper field is divided into macroblocks, such as macroblock 2361, and the lower field is divided into macroblocks, such as macroblock 2362. (In other words, the macroblock uses the 4: 2: 0 format as shown in FIG. 22, and the configuration and arrangement of the luminance block and the chrominance block within the macroblock are not shown.) In the luminance plane, Macroblock 2361 includes 16 lines from the top field, macroblock 2362 includes 16 lines from the bottom field, and each line is 16 pixels long. An interlaced I-field is a single separately expressed field of interlaced video frames. An interlaced P-field is a single separately expressed field of interlaced video frames coded using forward prediction, and an interlaced B-field is a single separately expressed field of interlaced video frames coded using bidirectional prediction. to be. Interlaced P-fields and B-fields may include intra-coded macroblocks as well as other types of predictive macroblocks.

The term picture generally refers to source, coded or reconstructed image data. For sequential video, a picture is a sequential video frame. In the case of interlaced video, a picture refers to an interlaced video frame, an upper field of a frame, or a lower field of a frame, depending on the context.

As another alternative, encoder 2000 and decoder 2100 are object-based, use different macroblocks or block formats, or perform operations on pixel sets of sizes or configurations different from 8x8 blocks and 16x16 macroblocks. .

B. Video Encoder

20 is a block diagram of a generalized video encoder system 2000. Encoder system 2000 receives a video picture sequence that includes a current picture 2005 (eg, a sequence of sequential video frames, interlaced video frames, or fields of interlaced video frames), and compressed video information 2095. Produces as output. Certain embodiments of the video encoder generally use a modified or supplemented version of the generalized encoder 2000.

Encoder system 2000 compresses the predictive picture and the key picture. 20 illustrates a path of a key picture and a path of a forward predicted picture through the encoder system 2000. Many of the components of the encoder system 2000 are used to compress both key pictures and predictive pictures. The exact actions performed by these components may vary depending on the type of information being compressed.

A predictive picture (also called a p-picture, b-picture for bidirectional prediction, or inter-coded picture) is represented in terms of prediction (or difference) from one or more other pictures. The prediction residual refers to the difference between the predicted and the original picture. In contrast, a key picture (also called an I-picture or intra-coded picture) is compressed without reference to another picture.

If the current picture 2005 is a forward predictive picture, the motion estimator 2010 may be a macroblock or other pixel of the current picture 2005 for the reference picture, which is a reconstructed previous picture 2025 buffered in the picture store 2020. Estimate the movement of the set. In an alternate embodiment, the reference picture is a later picture or the current picture is bi-predicted. The motion estimator 2010 may estimate motion by pixel, by 1/2 pixel, by 1/4 pixel, or in other increments, and may switch the precision of motion estimation by picture or in other ways. The precision of motion estimation may be the same or different in horizontal and vertical. The motion estimator 2010 outputs motion information 2015 such as a motion vector as additional information. The motion compensator 2030 applies the motion information 2015 to the reconstructed previous picture 2025 to form a motion-compensated current picture 2035. However, the prediction is rarely perfect, and the difference between the motion compensated current picture 2035 and the original current picture 2005 is the prediction residual 2045. As another alternative, the motion estimator and the motion compensator apply other types of motion estimation / compensation.

The frequency converter 2060 converts spatial domain video information into frequency domain (ie, spectral) data. For block-based video pictures, frequency converter 2060 applies a transformation of DCT or DCT to a block of pixel data or prediction residual data to produce a block of DCT coefficients. As another alternative, the frequency converter 2060 may apply other conventional frequency transforms, such as Fourier transforms or use wavelet or subband analysis. The frequency converter 2060 applies 8x8, 8x4, 4x8, or other magnitude frequency transform (eg, DCT) to the prediction residual for the predictive picture.

Subsequently, quantizer 2070 quantizes the block of spectral data coefficients. The quantizer applies uniform scalar quantization to spectral data having step-sizes that vary from picture to picture or in other ways. As another alternative, the quantizer is an encoder that does not apply other types of quantizations, such as non-uniform, vector or non-adaptive quantization to spectral data coefficients or uses frequency conversion. Quantize spatial domain data directly in the system. In addition to adaptive quantization, encoder 2000 may use frame dropping, adaptive filtering, or other techniques for rate control.

If a given macroblock in a predictive picture does not have any type of information (eg, does not have motion information and residual information for the macroblock), the encoder 2000 skipped the macroblock. macroblock). If so, the encoder can signal the skipped macroblock to the output bitstream of the compressed video information 2095.

If the reconstructed current picture is needed for subsequent motion estimation / compensation, inverse quantizer 2076 performs inverse quantization on the quantized spectral data coefficients. The frequency inverse transformer 2066 then performs the inverse of the operation of the frequency converter 2060 to generate a reconstructed prediction residual (in the case of a predictive picture) or a reconstructed sample (in the case of an intra-coded picture). If the picture being encoded 2005 is an intra-coded picture, the reconstructed sample forms the reconstructed current picture (not shown). If the picture being encoded 2005 is a predictive picture, the reconstructed prediction residual is added to the motion compensated prediction 2035 to form the reconstructed current picture. The picture store 2020 buffers the reconstructed current picture for use in predicting the next picture. In some embodiments, the encoder applies a deblocking filter to the reconstructed frame to adaptively smooth the discontinuities between the blocks of the frame.

Entropy coder 2080 compresses the output of quantizer 2070 as well as some additional information (eg, motion information 2015, quantization step size). Common entropy coding techniques include arithmetic coding, differential coding, Huffman coding, run length coding, LZ coding, dictionary coding, and combinations thereof. Entropy coder 2080 generally uses different coding techniques for different kinds of information (eg, DC coefficients, AC coefficients, other kinds of incidental information), and multiple code tables within a particular coding technique. You can choose from.

Entropy coder 2080 puts compressed video information 2095 into buffer 2090. The buffer level indicator is fed back to the bit rate adaptive module. Compressed video information 2095 is exhausted from buffer 2090 at a constant or relatively constant bit rate and stored for later streaming at that bit rate. Thus, the level of buffer 2090 is primarily a function of the entropy of filtered and quantized video information, which affects the efficiency of entropy coding. As another alternative, encoder system 2000 streams the compressed video information immediately after compression, and the level of buffer 2090 also depends on the rate at which information is exhausted from buffer 2090 for transmission.

Before or after the buffer 2090, the compressed video information 2095 may be channel coded for transmission over the network. Channel coding may apply error detection and correction data to the compressed video information 2905.

C. Video Decoder

21 is a block diagram of a general video decoder system 2100. Decoder system 2100 receives information 2195 about a compressed video picture sequence and includes a reconstructed picture 2105 (eg, a sequential video frame, an interlaced video frame, or a field of interlaced video frames). Produces output Certain embodiments of the video decoder generally use a modified or supplemented version of the generalized decoder 2100.

Decoder system 2100 decompresses the predictive picture and the key picture. 21 illustrates a path of a key picture and a path of a forward predicted picture through the decoder system 2100. Many of the components of the decoder system 2100 are used to decompress both the key picture and the predictive picture. The exact actions performed by these components may vary depending on the type of information being decompressed.

The buffer 2190 receives information 2195 about the compressed video sequence and makes the received information available to the entropy decoder 2180. Buffer 2190 generally receives information at a fairly constant rate over time, and includes a jitter buffer to smooth out short-term variations in bandwidth or transmission. Buffer 2190 may also include a playback buffer and other buffers. As another alternative, buffer 2190 receives information at a variable rate. Before or after buffer 2190, compressed video information may be channel decoded and processed for error detection and correction.

Entropy decoder 2180 entropy decodes entropy-coded and quantized data as well as entropy-coded incidental information (e.g., motion information 2115, quantization step size), i.e., entropy generally performed at an encoder. Apply the inverse of the encoding. Entropy decoding techniques include arithmetic decoding, differential decoding, Huffman decoding, run length decoding, LZ decoding, pre decoding and combinations thereof. Entropy decoder 2180 frequently uses different decoding techniques for different kinds of information (e.g., DC coefficients, AC coefficients, and other kinds of incidental information), and selects among multiple code tables within a particular decoding technique. Can be.

If the picture 2105 to be reconstructed is a forward predictive picture, the motion compensator 2130 applies the motion information 2115 to the reference picture 2125 to form a prediction 2135 of the picture 2105 being reconstructed. For example, the motion compensator 2130 uses the macroblock motion vector to find the macroblock in the reference picture 2125. Picture buffer 2120 stores a previous reconstructed picture for use as a reference picture. The motion compensator 2130 may compensate for motion in pixels, 1/2 pixels, 1/4 pixels, or other increments, and may switch the precision of motion compensation per picture or in other ways. The precision of motion compensation may be the same or different in horizontal and vertical. As another alternative, the motion compensator applies other types of motion compensation. The prediction by the motion compensator is rarely perfect, so the decoder 2100 also reconstructs the prediction residual.

If the decoder needs a reconstructed picture for later motion compensation, the picture store 2120 buffers the reconstructed picture for use in predicting the next picture. In some embodiments, the encoder applies a deblocking filter to the reconstructed frame to adaptively smooth the discontinuities between the blocks of the frame.

Inverse quantizer 2170 dequantizes the entropy-decoded data. In general, inverse quantizers apply uniform scalar inverse quantization to entropy-decoded data that varies from picture to picture or in other ways. As another alternative, the inverse quantizer applies other types of inverse quantization to the data, for example heterogeneous, vector or non-adaptive inverse quantization, or directly dequantizes the spatial domain data in a decoder system that does not use frequency inverse transform.

The frequency inverse transformer 2160 converts the quantized frequency domain data into spatial domain video information. For block-based video pictures, frequency inverse transformer 2160 applies IDCT or a transform of IDCT to a block of DCD coefficients to generate pixel data or prediction residual data for a key picture or a predictive picture, respectively. Alternatively, frequency inverse transformer 2160 may apply other conventional frequency inverse transforms, such as Fourier inverse transforms or use wavelet or subband synthesis. Frequency inverse transformer 2160 applies an 8x8, 8x4, 4x8, or other magnitude frequency inverse transform (eg, IDCT) to the prediction residual for the predictive picture.

III . Interlaced  P-field and Interlaced  P-frame

A typical interlaced video frame consists of two fields (e.g., top field and bottom field) scanned at different times. In general, stationary regions of interlaced video frames are more efficient to encode by coding the fields together (“frame mode” coding). On the other hand, often a moving region of an interlaced video frame is more efficient to code by coding the fields separately (“field mode” coding), because these two fields tend to have different movements. to be. The forward prediction interlaced video frame may be coded as two separate forward prediction fields (interlaced P-fields). For example, if there is a lot of motion throughout an interlaced video frame and therefore there are many differences between the fields, it may be efficient to code the fields separately for the forward predicted interlaced video frame.

Alternatively, the forward predictive interlaced video frame may be coded as an interlaced P-frame using a mixture of field coding and frame coding. In the case of a macroblock of an interlaced P-frame, this macroblock contains pixel lines for the upper and lower fields, which can be coded together in the frame-coding mode or separately in the field-coding mode.

An interlaced P-field refers to one or more previously decoded fields. For example, in some implementations, an interlaced P-field refers to one or two previously decoded fields, while an interlaced B-field refers to up to two previous reference fields and two future reference fields (ie , Up to a total of four reference fields). (Encoding and decoding techniques for interlaced P-fields are described in detail below.) Or, more specifically, for more information about interlaced P-fields and 2-referenced interlaced P-fields, according to some embodiments. , US Patent Application No. 10, entitled "Predicting Motion Vectors for Fields of Forward-predicted Interlaced Video Frames," filed May 27, 2004. See / 857,473.

IV . Interlaced  Number of reference fields in the P-field

In some embodiments, two previously coded / decoded fields may be used as reference fields when performing motion compensated prediction of a single current interlaced P-field. In general, the availability of two reference fields results in better compression efficiency than when motion compensated prediction is limited to one reference field. However, the signaling overhead is higher when two reference fields are available, because extra information is needed to indicate which of the two fields provides a reference to each macroblock or block with a motion vector. Because it is sent.

In some situations, the benefit of having more potential motion compensation predictors per motion vector (two reference fields versus one reference field) is no greater than the overhead required to signal reference field selection. For example, if all of the best references come from one of two possible reference fields, it may be advantageous to choose to use a single reference field instead of two. This is usually due to a scene change such that only one of the two reference fields is from the same scene as the current field. Or, only one reference field may be available, such as at the beginning of a sequence. In these cases, signaling that only one reference field is used at the field level for the current P-field and what that one reference field is, and applying the decision to macroblocks and blocks in the current P-field More efficient. The reference field selection information then no longer needs to be sent with every macroblock or block with a motion vector.

A. Number of reference fields in different ways

In one scheme, two previous coded / decoded fields may be used as reference fields for the current P-field. The reference field used by the motion vector (for the macroblock or block) is signaled for the motion vector, like other information about the motion vector. For example, in the case of a motion vector, the signaled information indicates (1) a reference field and (2) a position in the reference field of the block or macroblock predictor for the current block or macroblock associated with the motion vector. Or, the reference field information and the motion vector information are signaled as described in one of the joint implementations in section XII.

In another way, only one previously coded / decoded field is used as a reference field for the current P-field. In the case of a motion vector, it is not necessary to point to a reference field to which the motion vector refers. For example, in the case of a motion vector, the signaled information indicates only the location in the reference field of the predictor for the current block or macroblock associated with the motion vector. Or, the motion vector information is signaled as described in one of the joint implementations in section XII. The motion vector in the one reference field scheme is generally coded with fewer bits than the same motion vector in the two reference field scheme.

Either way, updating the buffer or picture store for the reference field is simple for future motion compensation. The reference field or fields for the current P-field are one or both of the most recent and second most recent I-field or P-field before the current P-field. Since the position of the candidate reference field is known, the encoder and decoder can automatically update the reference picture store for motion compensation of the next P-field, again without a buffer management signal.

As another alternative, the encoder and decoder use one or more additional schemes for multiple reference fields for interlaced P-fields.

B. Signaling  Yes

Specific examples of signaling described in this section and in the joint implementation in section XII include how many reference fields are currently used for P-fields, and which candidate reference fields are used when one reference field is used. Signaling. For example, a 1-bit field (called NUMREF) in the P-field header indicates whether the P-field uses one or two previous fields as references. If NUMREF = 0, only one reference field is used. If NUMREF = 1, two reference fields are used. If NUMREF = 0, there is another 1-bit field (referred to as REFFIELD), which indicates which of the two fields is used as a reference. If REFFIELD = 0, the field closer in time is used as the reference field. If REFFIELD = 1, the farther in time of the two candidate reference fields is used as the reference field for the current P-field. As another alternative, the encoder and decoder use other and / or additional signals for reference field selection.

C. Location of Reference Field

24A-F illustrate the positions of reference fields available for use in motion compensated prediction for interlaced P-fields. The P-field may use one or two previously coded / decoded fields as reference. Specifically, FIGS. 24A to 24F show examples of reference fields for NUMREF = 0 and NUMREF = 1.

24A and 24B show an example in which two reference fields are used for the current P-field (NUMREF = 1). In FIG. 24A, the current field refers to the top field and bottom field in an interlaced video frame previously in time. The intermediate interlaced B-field is not used as a reference field. In FIG. 24B, the current field refers to the upper field and lower field in the interlaced video frame immediately preceding the interlaced video frame containing the current field.

24C and 24D show an example in which one reference field is used for the current P-field (NUMREF = 0) and another reference field is the most recent reference field in time (REFFIELD = 0). The polarity of the reference field is opposite to the polarity of the current P-field, which means, for example, if the current P-field is from an even line, the reference field is from an odd line. In FIG. 24C, the current field refers to a lower field in a previous interlaced video frame in time and does not refer to a less recent upper field in an interlaced video frame. In other words, the interlaced B-field in the middle is not an allowed reference field. In FIG. 24D, the current field refers to the lower field in the interlaced video frame immediately preceding the interlaced video frame that includes the current field rather than the less recent upper field.

24E and 24F show an example in which one reference field is used for the current P-field (NUMREF = 0) and this one reference field is the second most recent reference field in time (REFFIELD = 1). . The polarity of the reference field is the same as the polarity of the current field, which means that, for example, if the current field is from an even line, the reference field is also from an even line. In FIG. 24E, the current field refers to the upper field in a previous interlaced video frame in time, and not to the more recent lower field. In other words, the interlaced B-field in the middle is not an allowed reference field. In FIG. 24F, the current field refers to the upper field rather than the more recent lower field.

As another alternative, the encoder and decoder use a reference field at other and / or additional position or timing for motion compensated prediction for the interlaced P-field. For example, a reference field in the same frame as the current P-field is allowed. Alternatively, the upper field or lower field of the frame may be coded / decoded first.

D. Encoding Technology

An encoder such as encoder 2000 of FIG. 20 signals which of a number of reference field schemes is used to code the interlaced P-field. For example, the encoder performs the technique 2500 shown in FIG. 25A.

For a given interlaced P-field, the encoder signals the number of reference fields used in motion compensated prediction for the interlaced P-field (step 2510). For example, the encoder uses a single bit to indicate whether one or two reference fields are used. As another alternative, the encoder uses another signaling / encoding mechanism for the number of reference fields.

The encoder determines whether one or two reference fields are used (step 2520). If one reference field is used, the encoder signals a reference field selection for the interlaced P-field (step 2530). For example, the encoder uses a single bit to indicate whether the most recent reference field (previous I-field or P-field) in time is used. As another alternative, the encoder uses other signaling / encoding mechanisms for selecting the reference field for the P-field.

If two reference fields are used, the encoder signals a reference field selection for a motion vector of a block, macroblock, or other portion of the interlaced P-field (step 2540). For example, the encoder jointly encodes the reference field selection for the motion vector along with the differential motion vector information for the motion vector. As another alternative, the encoder uses a different signaling / encoding mechanism for selecting the reference field for the motion vector. The encoder repeats signaling for the next motion vector until there are no more motion vectors to signal for the P-field (steps 2545 and 2540). (For simplicity, FIG. 25A does not show the various stages of macroblock and block encoding and corresponding signaling that may occur after or near the signaling of reference field selection (step 2540.) Instead, FIG. 25A shows P Focused on repeated signaling of reference field selection for multiple motion vectors in the field.)

As another alternative, the encoder performs another technique to indicate which of the multiple reference field schemes is used to code the interlaced P-field. For example, the encoder has more and / or different options for the number of reference fields.

For simplicity, FIG. 25A does not represent various ways in which the technique 2500 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

E. Decoding Technology

A decoder, such as decoder 2100 of FIG. 21, receives and decodes a signal indicating which of a number of schemes to use to decode the interlaced P-field. For example, the decoder performs the technique 2550 shown in FIG. 25B.

For a given interlaced P-field, the decoder receives and decodes a signal for the number of reference fields used for motion compensated prediction for the interlaced P-field (step 2560). For example, the decoder receives and decodes a single bit to indicate whether one or two reference fields are used. As another alternative, the decoder uses a different decoding mechanism for the number of reference fields.

The decoder determines whether one or two reference fields are used (step 2570). If one reference field is used, the decoder receives and decodes a signal for reference field selection for the interlaced P-field (step 2580). For example, the decoder receives and processes a single bit to indicate whether the most recent reference field (previous I-field or P-field) in time is used. As another alternative, the decoder uses another decoding mechanism for selecting the reference field for the P-field.

If two reference fields are used, the decoder receives and decodes a signal for reference field selection for a motion vector of a block, macroblock, or other portion of the interlaced P-field (step 2590). For example, the decoder decodes the reference field selection for the jointly coded motion vector along with the differential motion vector information for the motion vector. As another alternative, the decoder uses another decoding mechanism for selecting the reference field for the motion vector. The decoder repeats receiving and decoding for the next motion vector until there are no more motion vectors signaled for the P-field (steps 2595, 2590). (For simplicity, FIG. 25B does not show the various steps of macroblock and block decoding that may occur after or near the reception and decoding of reference field selection (step 2590.) Instead, FIG. 25B shows a P-field. We have focused on repeated reception / decoding of reference field selection for multiple motion vectors in.

As another alternative, the decoder performs another technique to indicate which of the multiple reference field schemes is used to decode the interlaced P-field. For example, the decoder has more and / or different options for the number of reference fields.

For simplicity, FIG. 25B does not show the various ways in which this technique 2550 can be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

V. Interlaced  Macroblocks for P-Fields mode  Information Signaling

In some embodiments, various macroblock mode information for macroblocks of interlaced P-fields are jointly grouped for signaling. Macroblocks of interlaced P-fields may be encoded in many different modes, with or without any of several different syntax elements. Specifically, the type of motion compensation (1MV, 4MV, or intra), whether the coded block pattern is in the bitstream for the macroblock, and (if 1MV) the motion vector data is in the bitstream for the macroblock. Whether or not is jointly coded. Different code tables can be used for different scenarios for macroblock mode information, resulting in a more efficient overall compression of the information.

Specific examples of signaling described in this section and in the joint implementation in section XII signal macroblock mode information with variable length coded MBMODE syntax elements. Table selection for MBMODE is signaled via the field level element MBMODETAB, which is fixed length coded. As another alternative, the encoder and decoder use other and / or additional signals to signal macroblock mode information.

A. Different Types Interlaced  Macroblocks for P-Fields mode

In general, macroblock mode indicates the macroblock type (1MV, 4MV or intra), the presence of a coded block pattern for the macroblock, and the presence of motion vector data for the macroblock. The information represented by the macroblock mode syntax element is either as an interlaced P-field with a 1MV field (with intra and / or 1MV macroblocks) or as a mixed-MV field (with intra, 1MV and / or 4MV macroblocks). Depends on whether or not it is encoded.

In a 1MV interlaced P-field, the macroblock mode element for a macroblock may be the macroblock type (intra or 1MV), the presence of a coded block pattern element for the macroblock, and the presence of motion vector data (macroblock type). Is 1 MV but not intra). The table in FIG. 26 shows the complete event space for macroblock information signaled by MBMODE in the 1MV interlaced P-field.

In a mixed-MV interlaced P-field, the macroblock mode element for a macroblock is determined by the macroblock type (intra, 1MV or 4MV), the presence of a coded block pattern for the macroblock, and the presence of motion vector data ( The macroblock type is 1MV but not intra or 4MV). The table in FIG. 27 shows the complete event space for macroblock information signaled by MBMODE in mixed-MV interlaced P-field.

When the macroblock mode indicates that motion vector data exists, the motion vector data is present in the macroblock layer and signals the motion vector difference synthesized with the motion vector predictor to reconstruct the motion vector. If the macroblock mode element indicates that no motion vector exists, the motion vector difference is assumed to be zero, so the motion vector is equal to the motion vector predictor. The macroblock mode element thus signals efficiently when motion compensation (not modified with motion vector differences), which consists only of motion vector predictors, is used.

One of a number of different VLC tables is used to signal the macroblock mode element for the interlaced P-field. For example, eight different code tables for MBMODE of macroblocks of mixed-MV interlaced P-field are shown in FIG. 47H, and eight different codes for MBMODE of macroblocks of 1MV interlaced P-field. The code table is shown in FIG. 47I. Table selection is indicated by the MBMODETAB element signaled in the field hierarchy. As another alternative, the encoder and decoder use other and / or additional code to signal macroblock mode information and table selection.

B. Encoding Technology

An encoder such as encoder 2000 of FIG. 20 encodes macroblock mode information for a macroblock of an interlaced P-field. For example, the encoder performs the technique 2800 shown in FIG. 28A.

For a given P-field, the encoder selects a code table to be used to encode macroblock mode information for the macroblock of the interlaced P-field (step 2810). For example, the encoder selects one of the VLC tables for MBMODE shown in FIG. 47H or 47I. As another alternative, the encoder selects from other and / or additional tables.

The encoder signals the selected code table to the bitstream (step 2820). For example, given the type of interlaced P-field, the encoder signals the FLC pointing to the selected code table. As another alternative, the encoder uses another signaling mechanism for code table selection, for example using VLC for code table selection.

The encoder selects a macroblock mode for the macroblock from the plurality of available macroblock modes (step 2830). For example, the encoder selects a macroblock mode that indicates the macroblock type, whether a coded block pattern is present, and whether motion vector data is present (if applicable for the macroblock type). Various combinations of options for MBMODE are listed in FIGS. 26 and 27. As another alternative, the encoder selects between different and / or additional macroblock modes for other and / or additional combinations of macroblock options.

The encoder signals the selected macroblock mode using the selected code table (step 2840). In general, the encoder signals the macroblock mode as VLC using the selected VLC table. The encoder repeats the selection and signaling of the macroblock mode until there are no more macroblock modes to signal for the P-field (steps 2845, 2830, 2840). (For simplicity, FIG. 28A does not show the various stages of macroblock and block encoding and corresponding signaling that may occur after or near the signaling of the selected macroblock (step 2840.) Instead, FIG. 28A shows P The selected code table for the field was used to focus on repeated signaling of macroblock mode for macroblocks in P-fields.)

As another alternative, the encoder performs other techniques to encode macroblock mode information for interlaced P-fields. For example, although FIG. 28A illustrates code table selection prior to mode selection, in many common encoding scenarios, the encoder first selects a macroblock mode for a macroblock, and then efficiently signals every selected mode. Selects a code table for which to signal the code table selection and mode. In addition, while FIG. 28A shows that code table selection is done for each interlaced P-field, as an alternative, the code table is selected more frequently, or less frequently, or aperiodically, or the encoder makes code table selection. Skip completely (always use the same code table). Or, the encoder can select a code table from the context information (which makes signaling of code table selection unnecessary).

For simplicity, FIG. 28A does not represent various ways in which the technique 2800 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

C. Decoding Technology

A decoder such as decoder 2100 of FIG. 21 receives and decodes macroblock mode information for a macroblock of an interlaced P-field. For example, the decoder performs the technique 2850 shown in FIG. 28B.

For a given interlaced P-field, the decoder receives and decodes a code table selection for the code table to be used to decode macroblock mode information for the macroblocks of the interlaced P-field (step 2860). For example, given the type of interlaced P-field, the decoder receives and decodes the FLC pointing to the selected code table. As another alternative, the decoder operates with other signaling mechanisms for code table selection, for example using VLC for code table selection.

The decoder selects a code table based on the decoded code table selection (and potentially other information) (step 2870). For example, the decoder selects one of the VLC tables for the MBMODE shown in FIG. 47H or 47I. As another alternative, the decoder selects from other and / or additional tables.

The decoder receives and decodes the macroblock mode selection for the macroblock (step 2880). For example, macroblock mode selection indicates whether a macroblock type, a coded block pattern is present, and whether motion vector data is present (if applicable for the macroblock type). Various combinations of these options for MBMODE are listed in FIGS. 26 and 27. As another alternative, the macroblock mode is one of the other and / or additional macroblock modes for other and / or additional combinations of macroblock options. The decoder then repeats receiving and decoding for the macroblock mode of the macroblock until there are no more macroblock modes to receive and decode for the P-field (steps 2885 and 2880). (For simplicity, FIG. 28B does not show the various stages of macroblock and block decoding that may occur after or near the reception and decoding of macroblock mode selection (step 2880.) Instead, FIG. The selected code table for the field was used to focus on repeated reception / decoding of macroblock mode selection for macroblocks in P-fields.)

As another alternative, the decoder performs other techniques to decode macroblock mode information for macroblocks of interlaced P-fields. For example, while FIG. 28B shows that code table selection is done per interlaced P-field, as another alternative, the code table is selected more frequently, less frequently, or aperiodically, or the decoder is coded. Skip table selection completely (always use the same code table). Alternatively, the decoder can select the code table from the context information (which makes the reception and decoding of the code table selection unnecessary).

For simplicity, FIG. 28B does not represent the various ways in which this technique 2850 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

VI . 2 reference field Interlaced  Reference field selection in P-fields

In some embodiments, when performing motion compensated prediction for a single current interlaced P-field, two previously coded / decoded fields are used as reference fields. (See, eg, section IV.) The signaled information indicates which of the two fields provides a reference to each macroblock (or block) with a motion vector.

In this section, various techniques and tools are provided for efficiently signaling which of a plurality of previously coded / decoded reference fields are used to provide motion compensated prediction information when coding or decoding a current macroblock or block. Described. For example, the encoder and decoder implicitly derive (or correspondingly) the dominant and non-dominant reference fields for the current macroblock or block based on previously coded motion vectors in the interlaced P-field. The encoder and decoder derive the dominant and non-dominant motion vector predictor polarity). The signaled information then indicates whether the dominant or non-dominant reference field is currently used for motion compensation of the macroblock or block.

A. Prevailing and non-dominant reference fields and Predictor

Interlaced fields can be coded without using motion compensation (I-field), with forward motion compensation (P-field), or with forward and backward motion compensation (B-field). An interlaced P-field may refer to two reference fields that are previously coded / decoded I-fields or P-fields. 24A and 24B show an example in which two reference fields are used for the current P-field. These two reference fields have opposite polarities. One reference field represents an odd line of a video frame, and the other reference field represents an even line of a video frame (the frame need not necessarily be the same frame including an odd line reference field). The P-field currently being coded or decoded may use one or both of the two previously coded / decoded fields as a reference in motion compensation. Thus, motion vector data for a macroblock or block of P-fields generally refers to (1) which field to use as a reference field in motion compensation, and (2) its reference of sample values to use in motion compensation. Indicates displacement / position in the field.

Signaling of the reference field selection information may consume an inefficient number of bits. However, by predicting which reference field will be used for that motion vector for a given motion vector and then signaling whether the predicted reference field is actually used as a reference field for the motion vector, the number of bits can be reduced.

For example, for each macroblock or block using motion compensation in an interlaced P-field, the encoder or decoder analyzes up to three previously coded / decoded motion vectors from neighboring macroblocks or blocks. From these, the encoder or decoder derives a dominant and non-dominant reference field. In essence, the encoder or decoder determines which of the two possible reference fields are used by most of the motion vectors of the neighboring macroblock or block. The field referenced by more motion vectors of neighbors is the dominant reference field, and the other reference field is the non-dominant reference field. Similarly, the polarity of the dominant reference field is the dominant motion vector predictor polarity, and the polarity of the non-dominant reference field is the dominant motion vector predictor polarity.

The pseudo code of FIG. 29 illustrates one technique by which an encode or decoder determines the dominant and non-dominant reference fields. In the pseudo code, the terms "same field" and "opposite field" refer to the current interlaced P-field. For example, if the current P-field is an even field, the "same field" is an even line reference field and the "opposite field" is an odd line reference field. 5A-10 illustrate the locations of neighboring macroblocks and blocks for which predictors A, B, and C are taken. In the pseudo code of FIG. 29, the dominant field is the field referenced by most motion vector predictor candidates. In the case of a tie, the motion vector derived from the opposite field is considered as the dominant motion vector predictor. Intra-coded macroblocks are not considered in the calculation of dominant / non-dominant predictors. If all candidate predictor macroblocks are intra-coded, the dominant and non-dominant motion vector predictors are set to zero, and the dominant predictor is assumed to be from the opposite field.

As another alternative, the encoder and decoder apply other decision logic to analyze other and / or additional motion vectors from neighboring macroblocks or blocks and / or to determine the dominant and non-dominant reference fields. Or, the encoder and decoder use different mechanisms to predict which reference field will be selected for a given motion vector in the interlaced P-field.

In any case, one bit of information indicating whether the dominant or non-dominant field is used is jointly coded together with the differential motion vector information. Thus, the bits / symbols for this one bit of information can more accurately match true symbol entropy. For example, a dominant / non-dominant selector is signaled as part of the vertical component of the motion vector difference, as shown in the pseudo code of FIG. Here, MVY is the vertical component of the motion vector, and PMVY is the vertical component of the motion vector predictor. Basically, the vertical motion vector difference jointly codes the reference field selector and the vertical offset difference as follows.

DMVY = (MVY-PMVY) * 2 + p

Here, p = 0 when the dominant reference field is used and p = 1 when the dominant reference field is used. As a numerical example, assuming that the current block is even polarity, the actual reference field for the motion vector is even polarity, and the dominant predictor is the oppfield (in other words, the dominant reference field is an odd polarity reference field). In addition, it is assumed that the vertical displacement of the motion vector is 7 units (MVY = 7) and the vertical component of the motion vector predictor is 4 units (PMVY = 4). DMVY = (7-4) * 2 + 1 = 7 because the current reference field and the dominant predictor have opposite polarities.

As another alternative, the dominant / non-dominant selector is coded jointly with the motion vector difference information in some other way. Alternatively, the dominant / non-dominant selector is signaled to another mechanism.

B. Encoding Technology

An encoder such as encoder 2000 of FIG. 20 determines the dominant and non-dominant reference field polarity for the motion vector predictor candidate during encoding of the motion vector for the two reference field interlaced P-field. For example, the encoder performs the technique 3100 shown in FIG. 31A on the current macroblock or motion vector of the block. In general, the encoder performs some form of motion estimation on two reference fields to obtain a motion vector and a reference field. The motion vector is then coded according to the technique 3100.

The encoder determines a motion vector predictor of the same reference field polarity as the motion vector (step 3110). For example, the encoder determines the motion vector predictor as described in section VII for the motion vector and associated reference field. As another alternative, the encoder determines the motion vector predictor by another mechanism.

The encoder determines the dominant and non-dominant reference field polarity of the motion vector (step 3120). For example, the encoder follows the pseudo code shown in FIG. As another alternative, the encoder uses other techniques to determine the dominant and non-dominant polarities.

The encoder signals in the bitstream a dominant / non-dominant polarity selector that indicates whether dominant or non-dominant polarity should be used for the motion vector and associated motion vector predictor and reference field (step 3125). For example, the encoder uses joint VLC to jointly encode the dominant / non-dominant polarity selectors with other information. As another alternative, the encoder uses another mechanism to signal the selector using, for example, arithmetic coding of bits indicating the selector. Prediction of the reference field polarity for the motion vector predictor degrades the entropy of the selector information, which allows for more efficient encoding of the selector information.

The encoder calculates a motion vector difference from the motion vector predictor and the motion vector (step 3130), and signals information about the motion vector difference information (step 3140).

In another alternative, the encoder performs other techniques to determine the dominant and non-dominant polarity for motion vector prediction during encoding of the motion vector for the two reference field interlaced P-field. In addition, although FIG. 31A shows separately signaling the dominant / non-dominant selector and motion vector difference information, in various embodiments, this information is jointly signaled. Various other reordering is possible, including determining the motion vector after determining the dominant / non-dominant polarity (to put the cost of the selector signaling overhead into the motion vector selection process).

For simplicity, FIG. 31A does not represent various ways in which the technique 3100 may be integrated with other aspects of encoding and decoding. Various binding implementations are described in detail in section VII.

C. Decoding Technology

A decoder such as decoder 2100 of FIG. 21 determines the dominant and non-dominant reference field polarity for the motion vector predictor candidate during decoding of the motion vector for the two reference field interlaced P-field. For example, the decoder performs the technique 3150 shown in FIG. 31B.

The decoder determines 3160 the dominant and non-dominant reference field polarity of the motion vector of the current macroblock or block. For example, the decoder follows the pseudo code shown in FIG. As another alternative, the decoder uses other techniques to determine the dominant and non-dominant polarities.

The decoder receives and decodes a dominant / non-dominant polarity selector in the bitstream indicating whether dominant or non-dominant polarity should be used for the motion vector and associated motion vector predictor and reference field (step 3165). For example, the decoder uses joint VLC to receive and decode predominant / non-dominant polarity selectors jointly coded with other information. As another alternative, the decoder uses other mechanisms to receive and decode the signaled selector, e.g. perform arithmetic decoding of bits indicating the selector.

The decoder determines a motion vector predictor for the reference field to be used with the motion vector (step 3170). For example, the decoder determines the motion vector predictor as described in section VII for the signaled polarity. As another alternative, the decoder determines the motion vector predictor by another mechanism.

The decoder receives and decodes information about the motion vector difference (step 3180) and reconstructs the motion vector from the motion vector difference and the motion vector predictor (step 3190).

As another alternative, the decoder performs another technique to determine the dominant and non-dominant polarity for motion vector prediction during the decoding of the motion vector for the two reference field interlaced P-field. For example, while FIG. 31B shows separate signaling of dominant / non-dominant selectors and motion vector differential information, this information is jointly signaled as another alternative. Many other reordering is possible.

For simplicity, FIG. 31B does not represent the various ways in which this technique 3150 may be integrated with other aspects of encoding and decoding. Various binding implementations are described in detail in section VII.

VII . Interlaced  For P-field hybrid  Motion vector prediction

In some embodiments, the motion vector is signaled as a difference to the motion vector predictor to reduce the bit rate associated with signaling the motion vector. The performance of motion vector differential signaling depends in part on the quality of motion vector prediction, which is usually improved when multiple candidate motion vector predictors are considered from the region around the current macroblock, block, etc. However, in some cases, using multiple candidate predictors impairs the quality of motion vector prediction. This happens, for example, when the motion vector predictor is calculated as the median of a series of candidate predictors of various (e.g., fluctuations between the motion vector predictors).

Thus, in some embodiments, the encoder and decoder perform hybrid motion vector prediction on the motion vectors of the interlaced P-field. If the vectors constituting the current macroblock or the causal neighborhood of the blocks vary according to some criteria, the hybrid motion vector prediction mode is used. In this mode, the median of the candidate predictor set is not used as a motion vector predictor, and a specific motion vector (eg, upper predictor, left predictor) in that set is used as a selector bit or codeword. Signaled as (codeword). This helps to improve motion vector prediction in motion discontinuities in interlaced P-fields. In the case of two reference field interlaced P-fields, the predominant polarity is also taken into account when examining the hybrid motion vector prediction conditions.

A. Interlaced  Motion vector prediction for P-field

Hybrid motion vector prediction is a special case of conventional motion vector prediction for interlaced P-fields. As described above, the motion vector is reconstructed by adding the motion vector difference (signaled in the bitstream) to the motion vector predictor. The predictor is calculated from up to three neighboring motion vectors. 5A to 10 illustrate the positions of neighboring macroblocks and blocks for which predictors A, B, and C are taken for motion vector prediction. (These figures show macroblocks and blocks of sequential P-frames, but they also apply to macroblocks and blocks of interlaced P-fields, as described in section VI.)

If an interlaced P-field refers to only one previous field, a single motion vector predictor is calculated for each motion vector of the P-field. For example, the pseudo code of FIGS. 51A and 51B (alternatively, FIGS. 60A and 60B) shows how the motion vector predictor is calculated for the motion vector of the 1 reference field interlaced P-field, This is described in detail in section VII.

If two reference fields are used for the interlaced P-field, there may be two motion vector predictors for each motion vector of the P-field. One may be selected after both motion vector predictors are calculated, or only one motion vector predictor may be calculated by first determining the predictor selection. One potential motion vector predictor is from a dominant reference field and another potential motion vector predictor is from a nondominant chama, where the terms 'dominance' and 'nondominance' are described, for example, in section VI. As shown. The dominant and non-dominant reference fields have opposite polarities, so one motion vector predictor is from a reference field of the same polarity as the current P-field and the other motion vector predictor is from a reference field with the opposite polarity. For example, the pseudo codes and tables of FIGS. 52A-52N illustrate a process for calculating a motion vector predictor for a motion vector of a 2-reference field P-field, which is described in detail in section VII. The variables samefieldpred_x and samefieldpred_y represent the horizontal and vertical components of the motion vector predictor from the same field, respectively, and the variables oppositefieldpred_x and the oppositefieldpred_y represent the horizontal and vertical components of the motion vector predictor from the opposite fields, respectively. The variable dominantpredictor indicates which field contains the dominant predictor. predictor_flag indicates whether a dominant or non-dominant predictor is used for the motion vector. As another alternative, the pseudo code of FIGS. 61A-61F is used.

B. Interlaced  For P-field hybrid  Motion vector prediction

In the case of hybrid motion vector prediction for a motion vector, the encoder and decoder examine the hybrid motion vector prediction condition for the motion vector. In general, this condition is related to the degree of variation of the motion vector predictor. The estimated predictor may be a candidate motion vector predictor and / or the motion vector predictor may be calculated using conventional motion vector prediction. If the condition is met (e.g., the degree of variation is high), one of the original candidate motion vector predictors is generally used in place of the conventional motion vector predictor. The encoder signals which hybrid motion vector predictor to use, and the decoder receives and decodes the signal. The hybrid motion vector predictor is not used when the inter-predictor variation is low (this is the usual case).

The encoder and decoder check the hybrid motion vector condition for each motion vector of the interlaced P-field regardless of whether the motion vector is for a macroblock, block, or the like. In other words, the encoder and decoder determine if a condition is triggered for each motion vector and thus predictor selection signal is expected. As another alternative, the encoder and decoder check the hybrid motion vector condition only for certain motion vectors of the interlaced P-field.

An advantage of hybrid motion vector prediction for interlaced P-fields is that this prediction uses calculated predictors and dominant polarity to select good motion vector predictors. From extensive experimental results, the hybrid motion vector prediction described below provides a significant compression / quality improvement over motion vector prediction without it and also over previous implementations of hybrid motion vector prediction. In addition, the additional calculations for hybrid vector prediction checks are not very expensive.

In some embodiments, the encoder or decoder tests a conventional motion vector predictor (which is determined by the technique described in section VII.A) against a series of original candidate motion vector predictors. A typical motion vector predictor is the component-specific median of predictors A, B, and / or C, and the encoder or decoder tests it with respect to predictor A and predictor C. This test checks whether the variation between normal motion vector predictors and candidates is high. If high, the exact motion vector may be closer to one of these candidate predictors (A, B or C) than the predictor derived from the median operation. If the candidate predictors are far apart, their component-specific median does not provide good prediction, and it is more efficient to send additional signals indicating whether the correct motion vector is close to A or close to B. If predictor A is closer, it is used as the motion vector predictor for the current motion vector, and if predictor C is closer, it is used as the motion vector predictor for the current motion vector.

The pseudo code of FIG. 32 illustrates this hybrid motion vector prediction during decoding. The variables predictor_pre_x and predictor_pre_y are the horizontal and vertical motion vector predictors, respectively, calculated using conventional hybrid motion vector prediction. The variables predictor_post_x and predictor_post_y are the horizontal and vertical motion vector predictors after hybrid motion vector prediction, respectively. In this pseudo code, a typical motion vector predictor is tested against predictors A and C to see if the motion vector predictor selection is explicitly coded in the bitstream. When explicitly coded, there is a single bit in the bitstream indicating whether to use predictor A or predictor B as a motion vector predictor. Otherwise, a normal motion vector predictor is used. Various other conditions (eg, the magnitude of a typical motion vector when A or C are intra) can also be checked. If A or C is intra, the motion corresponding to A or C is considered to be zero, respectively.

In the case of a motion vector of two reference field P-fields, all of the predictors have the same polarity. In some embodiments, the reference field polarity is determined by the dominant / non-dominant predictor polarity and selector signal obtained in the differential motion vector decoding process. For example, when the opposite field predictor is used, predictor_pre_x = oppositefieldpred_x, predictor_pre_x = oppositefieldpred_y, predictorA_x = oppositefieldpredA_x, predictorA_y = oppositefieldpredA_y, predictorC_x = oppositefieldpredC_x, and predictorC_y = oppositefieldpredC_y. When the same field predictor is used, predictor_pre_x = samefieldpred_x, predictor_pre_x = samefieldpred_y, predictorA_x = samefieldpredA_x, predictorA_y = samefieldpredA_y, predictorC_x = samefieldpredC_x, and predictorC_y = samefieldpredC_y. The values of oppositefieldpred and samefieldpred are calculated as in the pseudo code of FIGS. 52A-52J or 61A-61F, for example. 53 illustrates alternative pseudo code for hybrid motion vector prediction in a joint implementation (see section XII).

As another alternative, the encoder and the decoder may use different hybrid motion vector prediction conditions, e.g., conditions that consider other and / or additional predictors, conditions that use different decision logic to detect motion vector discontinuity, and / or ( Test conditions that use different thresholds for variation (not 32).

The simple signal to choose between two candidate predictors (e.g., A and C) is a single bit per motion vector. As another alternative, the encoder and decoder use different signaling mechanisms, for example jointly signaling selector bits with other information such as motion vector data.

C. Encoding Technology

An encoder such as encoder 2000 of FIG. 20 performs hybrid motion vector prediction during encoding of motion vectors for interlaced P-fields. For example, the encoder performs the technique 3300 shown in FIG. 33A on the current macroblock or motion vector of the block.

The encoder determines a motion vector predictor for the motion vector (step 3310). For example, the encoder uses the technique described in section VII.A to determine the motion vector predictor. As another alternative, the encoder uses other techniques to determine the motion vector predictor.

The encoder then checks the hybrid motion vector prediction condition for the motion vector predictor (step 3320). For example, the encoder uses a technique very similar to the decoder side pseudo code shown in FIG. As another alternative, the encoder checks other hybrid motion vector prediction conditions. (Whatever the condition, the corresponding decoder checks the same hybrid motion vector prediction condition as the encoder, since the presence of predictor signal information is implicitly derived by the encoder and the corresponding decoder.)

If the hybrid motion vector condition is not triggered ('no' path from decision step 3325), the encoder uses the initially determined motion vector predictor.

On the other hand, if the hybrid motion vector condition is triggered (the 'yes' path from decision step 3325), the encoder selects the hybrid motion vector predictor to use (step 3330). For example, the encoder selects between the top candidate predictor and the left candidate predictor, which are neighboring motion vectors. As another alternative, the encoder selects from other and / or additional predictors.

The encoder then signals the selected hybrid motion vector predictor (step 3340). For example, the encoder transmits a single bit indicating whether the top candidate predictor or the left candidate predictor is used as the motion vector predictor. As another alternative, the encoder uses other signaling mechanisms.

The encoder performs the description 3300 on all motion vectors of the interlaced P-field (eg, depending on the macroblock type) or only on certain motion vectors of the interlaced P-field. For simplicity, FIG. 33A does not represent various ways in which the technique 3300 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

D. Decoding Technology

A decoder such as decoder 2100 of FIG. 21 performs hybrid motion vector prediction during decoding of motion vectors for interlaced P-fields. For example, the decoder performs the technique 3350 shown in FIG. 33B for the current macroblock or motion vector of the block.

The decoder determines a motion vector predictor for the motion vector (step 3360). For example, the decoder uses the technique described in section VII.A to determine the motion vector predictor. As another alternative, the decoder uses other techniques to determine the motion vector predictor.

The decoder then examines the hybrid motion vector prediction condition for the motion vector predictor (step 3370). For example, the decoder follows the pseudo code shown in FIG. As another alternative, the decoder checks for other hybrid motion vector prediction conditions. (Whatever the condition, the decoder checks the same hybrid motion vector prediction condition as the corresponding encoder.)

If the hybrid motion vector condition is not triggered (the 'no' path from decision step 3375), the decoder uses the initially determined motion vector predictor.

On the other hand, when the hybrid motion vector condition is triggered (Yes path from decision 3375), the decoder receives and decodes a signal indicating the selected hybrid motion vector predictor (step 3380). For example, the decoder gets a single bit indicating whether the top candidate predictor or the left candidate predictor will be used as the motion vector predictor. As another alternative, the decoder operates in conjunction with other signaling mechanisms.

The decoder then selects a hybrid motion vector predictor to use (step 3390). For example, the decoder selects between the top candidate predictor and the left candidate predictor, which are neighboring motion vectors. As another alternative, the decoder selects other and / or additional predictors.

The decoder performs the description 3350 on all motion vectors of the interlaced P-field (eg, depending on the macroblock type) or only for certain motion vectors of the interlaced P-field. For simplicity, FIG. 33B does not represent various ways in which the technique 3350 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

VIII . Movement vector block pattern

In some embodiments, a macroblock can have multiple motion vectors. For example, a macroblock of a mixed-MV interlaced P-field may have one motion vector, four motion vectors (one for each luminance block of the macroblock) or may be intra coded (no motion vector). Similarly, a field-coded macroblock of an interlaced P-frame may have two motion vectors (one per field) or four motion vectors (two per field) and the frame- of an interlaced P-frame. A coded macroblock may have one motion vector or four motion vectors (one per luminance block).

A 2MV or 4MV macroblock may be signaled as “skipped” if the macroblock does not have associated motion vector data (eg, differential) to signal. If so, the motion vector predictor is generally used as the motion vector of the macroblock. Or, the macroblock may have non-zero motion vector data to signal for one motion vector, but not for other motion vectors (with (0, 0) motion vector difference). For 2MV or 4MV macroblocks with (0, 0) difference for at least one motion vector but not all, signaling motion vector data can consume an inefficient number of bits.

Thus, in some embodiments, the encoder and decoder use a signaling mechanism that efficiently signals the presence or absence of motion vector data for macroblocks having multiple motion vectors. The motion vector coded block pattern (or simply referred to as the "motion vector block pattern") for a macroblock is the motion vector per block, which block, field, half of the field, etc. has motion vector data signaled in the bitstream and which Indicates whether it is not. The motion vector block pattern jointly signals the pattern of motion vector data for the macroblock, which allows the encoder and decoder to take advantage of the spatial correlation that exists between the blocks. In addition, signaling the presence or absence of motion vector data in a motion vector block pattern provides a simple way to signal this information, i.e., separate from signaling about presence or transform coefficient data (as in a CBPCY element, etc.). .

Specific examples of signaling described in this section and in the joint implementation in section XII signal a motion vector block pattern with variable length coded 2MVBP and 4MVBP syntax elements. Table selection for 2MVBP and 4MVBP is signaled via fixed length coded 2MVBPTAB and 4MVBPTAB elements, respectively. As another alternative, the encoder and decoder use other and / or additional signals to signal the motion vector block pattern.

A. Motion Vector Block Pattern

The motion vector block pattern indicates which motion vectors are "coded" and which are "uncoded" for macroblocks having multiple motion vectors. If the differential motion vector for the motion vector is not zero (ie, the motion vector to be signaled is different from its motion vector predictor), the motion vector is coded. Otherwise, the motion vector is not coded.

If the macroblock has four motion vectors, the motion vector block pattern has four bits, one for each of the four motion vectors. The order of the bits in the motion vector block pattern follows the block order shown in FIG. 34 for 4MV macroblocks in interlaced P-fields or 4MV frame-coded macroblocks in interlaced P-frames. For 4MV field-coded macroblocks in interlaced P-frames, the bit order of the motion vector block pattern is a left upper field motion vector, a right upper field motion vector, a left lower field motion vector, and a right lower field motion vector.

If the macroblock has two motion vectors, the motion vector block pattern has two bits, one for each of the two motion vectors. For 2MV field-coded macroblocks of interlaced P-frames, the bit order of the motion vector block pattern is simply the upper field motion vector, followed by the lower field motion vector.

One of a number of different VLC tables may be used to signal the motion vector block pattern element. For example, four different code tables for 4MVBP are shown in FIG. 47J, and four different code tables for 2MVBP are shown in FIG. 47K. Table selection is indicated by 4MVBPTAB or 2MVBPTAB elements signaled at the picture level. As another alternative, the encoder and decoder use other and / or additional code to signal motion vector block pattern information and table selection.

An additional rule is applied to determine which motion vector is coded for the macroblock of the two reference field interlaced P-field. As described in section VI, the "uncoded" motion vector has a dominant predictor. A "coded" motion vector may have a non-zero motion vector difference but signal a non-dominant predictor. Or, a "coded" motion vector may be nonzero and have a differential motion vector and signal a dominant or non-dominant predictor.

As another alternative, the encoder and decoder may be used for other and / or additional kinds of pictures, for other and / or additional kinds of macroblocks, for other and / or additional numbers of motion vectors, and / or other A motion vector block pattern with a bit pattern is used.

B. Encoding Technology

An encoder, such as encoder 2000 of FIG. 20, encodes motion vector data for a macroblock using a motion vector block pattern. For example, the encoder performs the technique 3500 shown in FIG. 35A.

For a given macroblock with multiple motion vectors, the encoder determines a motion vector block pattern for the macroblock (step 3510). For example, the encoder determines a four motion vector block pattern for a 4MV macroblock in an interlaced P-field or for a 4MV field-coded or frame-coded macroblock in an interlaced P-frame. Or, the encoder determines two motion vector block patterns for 2MV field-coded macroblocks in interlaced P-frames. As another alternative, the encoder determines the motion vector block pattern for different kinds of macroblocks and / or other numbers of motion vectors.

The encoder then signals a motion vector block pattern (step 3520). In general, the encoder signals the VLC for the motion vector block pattern using a code table such as those shown in FIGS. 47J and 47K. As another alternative, the encoder uses another mechanism to signal the motion vector block pattern.

If there is at least one motion vector to which the motion vector data should be signaled (“Yes” path from decision 3525), the encoder signals the motion vector data for the motion vector (step 3530). For example, the encoder uses the technique described in section IX to encode motion vector data as BLKMVDATA, TOPMVDATA or BOTMVDATA elements. As another alternative, the encoder uses other signaling techniques.

The encoder repeats the encoding of the motion vector data until there are no more motion vectors for which the motion vector data should be signaled (“No” path from decision step 3525) (steps 3525 and 3530).

The encoder can select from a number of code tables to encode the motion vector block pattern (not shown in FIG. 35A). For example, the encoder selects a code table for an interlaced P-field or P-frame and then uses that table to encode a motion vector block pattern for a macroblock in a picture. As another alternative, the encoder selects the code table more frequently, less frequently, or aperiodically, or the encoder completely skips the code table selection (always using the same code table). Or, the encoder can select a code table from the context information (which makes it unnecessary to signal code table selection). The code table may be the table shown in FIGS. 47J and 47K, other tables and / or additional tables. The encoder signals the selected code table in a bitstream, for example, with an FLC pointing to the selected code table, with a VLC pointing with the selected code table, or with another signaling mechanism.

As another alternative, the encoder performs another technique to encode motion vector data for the macroblock using the motion vector block pattern. For simplicity, FIG. 35A does not represent various ways in which technique 3500 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

C. Decoding Technology

A decoder such as decoder 2100 of FIG. 21 receives and decodes motion vector data for a macroblock of an interlaced P-field or an interlaced P-frame using a motion vector block pattern. For example, the decoder performs the technique 3550 shown in FIG. 35B.

For a given macroblock with multiple motion vectors, the decoder receives and decodes the motion vector block pattern for the macroblock (step 3560). For example, the decoder receives and decodes the four motion vector block pattern, two motion vector block pattern, or other motion vector block pattern described in the previous section. In general, the decoder receives the VLC for the motion vector block pattern and decodes it using a code table such as those shown in FIGS. 47J and 47K. As another alternative, the decoder receives and decodes the motion vector block pattern with respect to other signaling mechanisms.

If there is at least one motion vector for which the motion vector data is signaled (“Yes” path from decision step 3565), the decoder receives and decodes the motion vector data for the motion vector (step 3570). For example, the decoder receives the encoded motion vector data and decodes it as a BLKMVDATA, TOPMVDATA or BOTMVDATA element using the technique described in section IX. As another alternative, the decoder uses other decoding techniques.

The decoder repeats the reception and decoding of the motion vector data until there are no more motion vectors for which the motion vector data is signaled (“No” path from decision step 3565) (steps 3565 and 3570).

The decoder may select from a number of code tables to decode the motion vector block pattern (not shown in FIG. 35B). For example, the table selection and table selection signaling options are very similar to what was previously described for the encoder in this section.

As another alternative, the decoder performs another technique to decode the motion vector data for the macroblock using the motion vector block pattern. For simplicity, FIG. 35B does not represent various ways in which the technique 3550 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

IX . Interlaced  Motion vector in P-field Difference

In some embodiments, the two previously coded / decoded fields are used as reference fields when performing motion-compensated prediction for a single current interlaced P-field (eg, section IV, VI). And VII). The information signaled for the motion vector in the P-field indicates (1) which of the two fields provide a reference to the motion vector, and (2) the motion vector value. The motion vector value is generally signaled as a difference for the motion vector predictor. The selection between two possible reference fields can be signaled with a single additional bit for the motion vector, but the signaling scheme is inefficient in many cases. Normally, these two reference fields are not equally likely for a given motion vector and the choice for the motion vector is not independent of the choice for the other (eg neighbor) motion vectors. Thus, in practice, signaling a reference field selection with a single bit per selection is usually inefficient.

Thus, in some embodiments, the encoder jointly encodes motion vector difference information and reference field selection information. The decoder performs the corresponding decoding of the jointly coded information.

A. Theory and Experimental Results

Two Reference Fields For an interlaced P-field, the two reference fields follow the spatial and temporal relationship below for the P-field. The polarity of the closest reference field in the temporal order is opposite to the polarity of the current P-field. For example, if the current P-field is an even field (consisting of even lines in an interlaced frame), the nearest reference field in the temporal order is the odd field, and the other reference field (the field farther in the temporal order). ) Is an even field.

The encoder and decoder use causal information to predict the reference field selection for the current motion vector. For example, reference field selection information from neighboring previously coded motion vectors is used to predict the reference field used for the current motion vector. The binary value then indicates whether the predicted reference field is used. One value indicates that the actual reference field for the current motion vector is the predicted reference field, and the other value indicates that the actual reference field for the current motion vector is the other reference field. In some implementations, the reference field prediction is represented by the polarity of the previously used reference field and the expected reference field for the current motion vector (eg, as dominant or non-dominant polarity, see section VI). In most scenarios, in this prediction, the probability distribution of the binary value reference field selector is consistent and skewed towards the predicted reference field. In the experiment, the predicted reference field is used for approximately 70% of the motion vector and approximately 30% of the motion vector uses the other reference field.

It is not efficient to send a single bit to signal reference field selection information with such probability distribution. A more efficient way is to code the reference field selection information jointly with the differential motion vector information.

B. Signaling Mechanism  Yes

Various examples of signaling mechanisms for jointly encoding and decoding motion vector difference information and reference field selection information are provided. As another alternative, the encoder and decoder jointly encode and decode the information in relation to other mechanisms.

The pseudo code of FIG. 36 illustrates jointly coding the motion vector difference information and the reference field selection information according to a generalized signaling mechanism. In this pseudo code, the variables DMVX and DMVY are horizontal and vertical differential motion vector components, respectively. Variables AX and AY are absolute values of the difference components, and variables SX and SY are the signs of the difference components. The horizontal motion vector range is from -RX to RX + 1, and the vertical motion vector range is from -RY to RY + 1. RX and RY are powers of two with exponents of MX and MY, respectively. The variables ESCX and ESCY (which are powers of two with exponents of KX and KY, respectively) indicate a threshold and beyond that, escape coding is used. The variable R is a binary value for reference field selection.

When an escape condition is triggered (if AX> ESCX or AY> ESCY), the encoder transmits an escape mode signal and a VLC that jointly represents R. The encoder then transmits DMVX and DMVY as fixed length codes of length MX + 1 and MY + 1, respectively. Thus, two elements in the VLC table are used to signal (1) both DMVX and DMVY are coded using the (MX + MY + 2) bits, and (2) the associated R values. In other words, the two elements are escape codes corresponding to R = 0 and R = 1.

In other cases, the variables NX and NY indicate how many bits are used to signal different values of AX and AY, respectively. AX is in the interval (2 ^NX <= AX <2 ^{NX +1} ), where NX = 0, 1, 2, ..., KX-1 and AX = 0 when NX = -1. AY is in the interval 2 ^NY <= AY <2 ^{NY +1} , where NY = 0, 1, 2, ..., KY-1 and AY = 0 when NY = -1.

The VLC table used to code the size information NX and NY and the field reference information R is a table of (KX + 1) * (KY + 1) * 2 + 1 elements, where each element is a codeword, Code size) pairs. Of the elements in the table, all but two are used to jointly signal the values of NX, NY and R. The other two elements are escape codes.

If signaled with NX and NY, the encoder sends a VLC indicating the combination of NX, NY and R values. The encoder then sends AX as NX bits, sends SX as 1 bit, sends AY as NY bits, and sends SY as 1 bit. If NX is 0 or -1, AX does not need to be transmitted, and so is for NY and AY, since the value of AX or AY can be derived directly from NX or NY in this case.

The case where AX = 0, AY = 0 and R = 0 is signaled by other mechanisms, such as a skip macroblock mechanism or a motion vector block pattern (see section VIII). The [0,0,0] element is neither present in the VLC table for the pseudo code of FIG. 36 nor mentioned in the pseudo code.

The corresponding decoder performs joint decoding much like the encoding shown in FIG. For example, the decoder receives the bits without transmitting the bits, performs variable length decoding instead of variable length encoding, and so on.

The pseudo code of FIG. 50 illustrates the decoding of motion vector difference information and reference field selection information that are jointly coded according to a signaling mechanism in one combined implementation. The pseudo code of FIG. 59 illustrates decoding of motion vector difference information and reference field selection information that are jointly coded according to a signaling mechanism in another combined implementation. The pseudo code of FIGS. 50 and 59 is described in detail in section VII. Specifically, this pseudo code illustrates jointly coding and decoding the prediction selector with vertical difference values or with magnitude of vertical and horizontal difference values.

The corresponding encoder performs joint encoding much like the decoding shown in FIG. 50 or 59. For example, the encoder transmits bits instead of receiving bits, performs variable length encoding instead of variable length decoding, and so on.

C. Encoding Technology

An encoder such as encoder 2000 of FIG. 20 jointly codes reference field prediction selector information and differential motion vector information. For example, the encoder performs the technique 3700 shown in FIG. 37A to jointly encode the information. In general, the encoder performs some form of motion estimation on two reference fields to obtain a motion vector and a reference field. A motion vector is then coded according to the description 3700, at which point one of the two possible reference fields is associated with the motion vector by jointly coding the selector information with, for example, a vertical motion vector difference.

The encoder determines a motion vector predictor for the motion vector (step 3710). For example, the encoder determines the motion vector predictor as described in section XII. As another alternative, the encoder determines the motion vector predictor by another mechanism.

The encoder determines the motion vector difference of the motion vector for the motion vector predictor (step 3720). In general, this difference refers to the component-specific difference between the motion vector and the motion vector predictor.

The encoder also determines reference field prediction selector information (step 3730). For example, the encoder determines the dominant and non-dominant polarity for the motion vector (thus see section VI for the dominant reference field, dominant polarity, etc., for the motion vector predictor), in which case the selector is the dominant polarity. Indicates whether this is used. As another alternative, the encoder uses another technique to determine the reference field prediction selector information. For example, the encoder uses other types of reference field prediction.

The encoder then jointly codes motion vector difference information and reference field prediction selector information for the motion vector (step 3740). For example, the encoder encodes information using one of the mechanisms described in the previous section. As another alternative, the encoder uses a different mechanism.

For simplicity, FIG. 37A does not represent various ways in which the technology 3700 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

D. Decoding Technology

A decoder such as decoder 2100 of FIG. 21 decodes jointly coded reference field prediction selector information and differential motion vector information. For example, the decoder performs the technique 3760 shown in FIG. 37B to decode this jointly coded information.

The decoder decodes the jointly coded motion vector difference information and reference field prediction selector information for the motion vector (step 3760). For example, the decoder decodes the signaled information using one of the mechanisms described in section IX.B. As another alternative, the decoder decodes the signaled information using another mechanism.

The decoder then determines a motion vector predictor for the motion vector (step 3770). For example, the decoder determines the dominant and non-dominant polarity (see section VI) for the motion vector, applies selector information, and determines the motion vector predictor for the selected polarity as described in section VII. As another alternative, the decoder uses another mechanism to determine the motion vector predictor. For example, the decoder uses another type of reference field prediction.

Finally, the decoder reconstructs the motion vector by combining the motion vector difference with the motion vector predictor (3780).

For simplicity, FIG. 37B does not represent various ways in which the technique 3750 may be integrated with decoding and other aspects of decoding. Various combined implementations are described in detail in section XII.

X. Interlaced  In P-field Chroma  Derivation of the Motion Vector

In some embodiments, the encoder and decoder derive the chroma motion vector from the luma motion vector signaled for the macroblock of the interlaced P-field. Chroma motion vectors are not explicitly signaled in the bitstream. Rather, the chroma motion vector is determined from the luma motion vector for the macroblock. Encoders and decoders can use chroma motion vector derivation, which is adapted for sequential P-frames or interlaced P-frames, but this generally provides inadequate performance for interlaced P-fields. Thus, the encoder and decoder use chroma motion vector derivation, which is adapted to the reference field configuration of interlaced P-fields.

Chroma motion vector derivation has two stages: (1) selection, and (2) sub-sampling and chroma rounding. Among these steps, the selection step is particularly suitable for chroma motion vector derivation in an interlaced P-field. The output of the selection phase is an initial chroma motion vector that depends on the number (and possibly polarity) of the luma motion vectors for the macroblock. If no luma motion is used for the macroblock (intra macroblock), no chroma motion vector is derived. If a single luma motion vector is used for the macroblock (1MV macroblock), a single luma motion vector is selected for use in the second and third steps. If four luma motion vectors are used for the macroblock (4MV macroblock), the initial chroma motion vector is selected using logic that prefers the more common polarity of the four luma motion vectors.

A. Chroma  Sub-Sampling and Motion Vector Representation

Chroma motion vector derivation for a macroblock of an interlaced P-field also depends on the type of chroma sub-sampling used for the macroblock and also on the motion vector representation.

Some conventional Clova sub-sampling formats are 4: 2: 0 and 4: 1: 1. 38 shows a sampling grid for a YUV 4: 2: 0 macroblock, whereby chroma samples are sub-sampled for luma samples in a normal 4: 1 pattern. 38 shows the spatial relationship between luma and chroma samples for a 16x16 macroblock with four 8x8 luma blocks, one 8x8 chroma "U" block, and one 8x8 chroma "V" block (as shown in FIG. 22). It is shown. In general, the resolution of the chroma grid is 1/2 of the resolution of the luma grid in both the x and y directions, which is the basis for downsampling in chroma motion vector derivation. In order to scale the motion vector distance for the luma grid to the corresponding distance on the chroma grid, the motion vector value is divided by two. The selection step technique described herein may be applied to YUV 4: 2: 0 macroblocks or to macroblocks having other chroma sub-sampling formats.

The representation of luma and chroma motion vectors for interlaced P-fields depends in part on the precision of the motion vectors and motion compensation. Typical motion vector precision is 1/2 pixel and 1/4 pixel, which handles half pixel and quarter pixel interpolation in motion compensation respectively.

In some embodiments, the motion vector for an interlaced P-field may refer to a reference field of upper or lower, or of the same or opposite polarity. The vertical displacement specified by the motion vector value depends on the polarity of the current P-field and the reference field. The motion vector unit is generally expressed in field picture units. For example, if the vertical component of the motion vector is +6 (in 1/4 pixels), this generally indicates the vertical displacement of the 1½ field picture line (if necessary, different from the current P-field and the reference field). Before adjustment for polarity).

For the combination of various vertical motion vector component values and field polarities, FIG. 39 shows the corresponding spatial positions in the current and reference fields according to the first protocol. Each combination of field polarities has a pair of columns, ie one (left column) is for the pixel for the lines in the current field (lines numbered N = 0, 1, 2, etc.) and the other (right Column) is for the pixel for the lines in the reference field (also, lines numbered N = 0, 1, 2, etc.). Circles represent samples at integer pixel positions and X represents interpolated samples at sub-pixel positions. In this convention, a vertical motion vector component value of zero refers to an integer pixel position in the reference field (ie, a sample on the actual line). If the current field and the reference field have the same polarity, the vertical component value from line N of the current field refers to line N in the reference field at the same actual offset in the frame. If the current field and the reference field have opposite polarities, the vertical component value of zero from line N in the current field still refers to line N in the reference frame, but the referenced position is in the frame due to the interlacing of odd and even lines. It is at a ½-pixel actual offset.

48 shows a corresponding spatial position in the current and reference field according to the second protocol. In this protocol, a vertical motion vector component value of zero refers to a sample at the same actual offset in an interlaced frame. The referenced sample is at an integer pixel position in the same polarity reference field or at a ½-pixel position in the opposite reference field.

As another alternative, the motion vectors for the interlaced P-fields use different representations and / or follow different conventions for handling vertical displacements with respect to polarity.

B. Example Selection Steps

In some embodiments, the selecting step of chroma motion vector derivation is adapted to the reference field pattern used in motion compensation for interlaced P-fields with one or two reference fields. For example, the result of the selection step for the macroblock depends on the number and polarity of the luma motion vectors for the macroblock.

The simplest case is when the entire macroblock is intra coded. In this case, there is no chroma motion vector, and the second and third steps of chroma motion vector derivation are omitted. Chroma blocks of macroblocks are intracoded / decoded and not motion compensated.

The next simplest case is when a macroblock has a single luma motion vector for all four luma blocks. Regardless of whether the current P-field has one reference field or two reference fields, there is no selection operation on its own because a single luma motion vector is simply passed on rounding and sub-sampling.

If the macroblock has up to four luma motion vectors, the selection step is more complicated. In general, the selection step favors dominant polarity among the luma motion vectors of the macroblock. If the P-field has only one reference field, the polarity is the same for all luma motion vectors for the macroblock. However, if the P-field has two reference fields, different luma motion vectors of the macroblock may point to different reference fields. For example, if the polarity of the current P-field is odd, the macroblock refers to two opposite polar luma motion vectors (see even polar reference field) and two equal polar luma motion vectors (odd polar reference field). ) The encoder or decoder determines the dominant polarity for the luma motion vector of the macroblock and determines the initial chroma motion vector from the luma motion vector of the dominant polarity.

In some implementations, a 4MV macroblock has 0-4 motion vectors. The luma block of this 4MV macroblock is intra coded, has an associated equal polar luma motion vector, or has an associated equal polar luma motion vector. In another implementation, a 4MV macroblock always has four luma motion vectors even if none of them are signaled (eg because they have a (0,0) difference). The luma block of this 4MV macroblock has the opposite polarity motion vector or the same polarity motion vector. The selection step logic is slightly different for these other implementations.

1. 0 to 4 Luma  4 with motion vectors MV  Macroblock

The pseudo code of FIG. 40 shows an example of selection step logic applied for a 4MV macroblock with 0-4 luma motion vectors. Among the luma motion vectors, if the number of luma motion vectors referring to the same polarity reference field is greater than the number referring to the opposite polarity reference field, the encoder / decoder is the initial chroma motion vector from the luma motion vector referring to the same polarity reference field. To derive Otherwise, the encoder / decoder derives the initial chroma motion vector from the luma motion vector referring to the opposite polarity reference field.

If the four luma motion vectors have a dominant polarity (eg, all are odd true or all even reference fields), the encoder / decoder calculates the median of the four luma motion vectors. If only three luma motion vectors have a dominant polarity (eg, because one luma block is intra or has a non-dominant polar motion vector), the encoder / decoder calculates the median of the three luma motion vectors. If the two luma motion vectors have a dominant polarity, the encoder / decoder calculates the average of the two luma motion vectors. (If the same polarity count is the same as the opposite polarity count, then the same polarity is preferred (as in the current P-field).) Finally, if there is only one luma motion vector of the dominant polarity (e.g. three luma blocks One luma motion vector is taken as the output of the selection step. If all luma blocks are intra, the macroblock is intra and the pseudo code of FIG. 40 does not apply.

2. 4 Luma  4 with motion vectors MV  Macroblock

55A and 55B show another example of selection step logic applied for a 4MV macroblock that always has four luma motion vectors (eg, because intra coded luma blocks are not allowed). . 55A relates to chroma motion vector derivation for this 4MV macroblock in a 1 reference field interlaced P-field, and FIG. 55B illustrates chroma motion vector derivation for such 4MV macroblock in a 2 reference field interlaced P-field. It is about.

Referring to FIG. 55B, the encoder / decoder determines which polarity of the four luma motion vectors of the 4MV macroblock is dominant (eg, odd or even). If all four luma motion vectors are from the same field (eg all odd or all even), the median of the four luma motion vectors is determined. If three of the four are from the same field, the median of the three luma motion vectors is determined. Finally, if there are two luma motion vectors for each polarity, two luma motion vectors with the same polarity as the current P-field are preferred, and their average is determined. (If a 4MV macroblock always has four luma motion vectors, it is not possible if there is only one luma motion vector and no luma motion vectors with dominant polarity.)

As another alternative, the encoder or decoder uses different selection logic when deriving chroma motion vectors from multiple luma motion vectors of macroblocks of interlaced P-fields. Alternatively, the encoder or decoder may derive chroma motion vectors for different types of macroblocks (e.g., having different numbers of luma motion vectors and / or macroblocks in interlaced P-fields and other types of fixers). Consider the luma motion vector polarity in.

C. Sub-Sampling / Rounding Steps

For the second stage of chroma motion vector derivation, an encoder or decoder is typically used to remove some pixel position from the initial chroma motion vector (e.g., such that the chroma motion vector after downsampling does not indicate a quarter-pixel displacement). Apply rounding logic (to round pixel positions). The use of rounding can be adjusted to compromise the complexity of prediction quality versus interpolation. For example, for more aggressive rounding, the encoder or decoder removes all ¼-pixel chroma displacements in the resulting chroma motion vector, so only integer-pixel and ½-pixel displacements are allowed, which is motion for chroma blocks. Simplify interpolation in compensation.

In the second step, the encoder and decoder also downsample the initial chroma motion vector to obtain the chroma motion vector at an appropriate scale for chroma resolution. For example, if the chroma resolution is 1/2 of the luma resolution in both horizontal and vertical, the horizontal and vertical motion vector components are downsampled by two.

As another alternative, the encoder or decoder applies other and / or additional mechanisms for rounding, sub-sampling, pullback, or other adjustment of the chroma motion vector.

D. Derivation Technology

An encoder such as encoder 2000 of FIG. 20 derives a chroma motion vector for the macroblock of the interlaced P-field. Alternatively, a decoder such as decoder 2100 of FIG. 21 derives a chroma motion vector for the macroblock of the interlaced P-field. For example, the encoder / decoder performs the technique 4100 shown in FIG. 41 to derive the chroma motion vector.

The encoder / decoder determines whether the current macroblock is an intra macroblock (step 4110). In the case of an intra macroblock, the encoder / decoder skips chroma motion vector derivation, and instead of motion compensation, intra coding / decoding is used for the macroblock.

If the macroblock is not an intra macroblock, the encoder / decoder determines whether the macroblock is a 1MV macroblock (step 4120). In the case of a 1MV macroblock, the encoder / decoder uses a single luma motion vector for the macroblock as the initial chroma motion vector passed to later adjustment step (s) 4150 of the technique 4100.

If the macroblock is not a 1MV macroblock, the encoder / decoder determines the dominant polarity among the luma motion vectors of the macroblock (step 4130). For example, the encoder / decoder determines the dominant polarity among one or more luma motion vectors of the macroblock as described in FIG. 40 or 55A and 55B. As another alternative, the encoder / decoder applies other and / or additional decision logic to determine the dominant polarity. If a P-field containing a macroblock has only one reference field, the dominant polarity in the luma motion vector is simply the polarity of that one reference field.

The encoder / decoder then determines an initial chroma motion vector from the luma motion vectors of the macroblock with the dominant polarity (step 4140). For example, the encoder / decoder determines the initial chroma motion vector as shown in FIG. 40 or 55A and 55B. As another alternative, the encoder / decoder uses other and / or additional logic to determine the initial chroma motion vector as a median, mean or other combination of dominant polar motion vectors.

Finally, the encoder / decoder adjusts the initial chroma motion vector generated by one of the previous steps (step 4150). For example, the encoder / decoder performs rounding and sub-sampling as described above. As another alternative, the encoder / decoder performs other and / or additional adjustments.

As another alternative, the encoder / decoder checks the various macroblock types and polarity conditions in different orders. Or, the encoder / decoder derives chroma motion vectors for other and / or additional types of macroblocks in interlaced P-fields or other types of pictures.

For simplicity, FIG. 41 does not represent various ways in which the technique 4100 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

XI . Interlaced  Century compensation for P-fields

Fading, morphing, and blending are widely used in the creation and editing of video content. These techniques smooth the visual progress of video across content transitions. In addition, some video sequences involve fading naturally due to changes in illumination. For predicted pictures affected by fading, morphing, blending, etc., the overall change in luminance compared to the reference picture reduces the effectiveness of conventional motion estimation and compensation. As a result, the motion compensated prediction is worse, and the predicted picture requires more bits to represent it. This problem is more complicated for interlaced P-fields with one reference field or multiple reference fields.

In some embodiments, the encoder and decoder perform fading compensation (also called intensity compensation) on the reference field for the interlaced P-field. The encoder performs a corresponding fading estimate. Fading estimation and compensation, and signaling mechanisms for fading compensation parameters are adapted to the reference field configuration of interlaced P-fields. For example, in the case of an interlaced P-field with one reference field or two reference fields, the determination of whether to perform fading compensation is made separately for each reference field. Each reference field that uses fading compensation may have its own fading compensation parameter. The fading compensation decision and the signaling mechanism for the parameter efficiently represent this information. As a result, the quality of interlaced video is improved and / or the bit rate is reduced.

A. for reference fields fading  Estimation and Reward

Fading compensation includes modifying one or more reference fields to compensate for fading, blending, morphing, and the like. In general, fading compensation includes fading (i.e. fading to or fading from black), blending, morphing, or other natural or composite lighting that affects pixel value intensity. It includes any reward for the effect. For example, the overall luminance change can be expressed as a change in brightness and / or contrast of the scene. In general, the change is linear, but it can also be defined as including any smooth, non-linear mapping within the same framework. The current P-field is then predicted by motion estimation and / or compensation from the adjusted one or more reference fields.

For the reference field in the YUV color space, the adjustment is made by adjusting the samples in the luminance and chrominance channels. This adjustment may include scaling and shifting of the luminance value and scaling and shifting of the chrominance value. As another alternative, color spaces are different (eg YIQ or RGB) and / or compensation uses different adjustment techniques.

The encoder / decoder performs fading estimation / compensation for each field. As another alternative, the encoder / decoder performs fading estimation / compensation in some other way. Thus, fading compensation adjustments affect defined areas that may be fields or portions of a field (eg, individual blocks or macroblocks, or a group of macroblocks), and the fading compensation parameters are for the defined areas. . Alternatively, the fading compensation parameter is for the entire field, but optionally and also applies to areas within the field as needed.

B. Interlaced  Configure Reference Fields for P-Fields

In some embodiments, the interlaced P-field has one or two reference fields for motion compensation (see, eg, section IV). 24A-24F illustrate the positions of reference fields that can be used for motion compensated prediction for interlaced P-fields. The encoder and decoder may use the reference field at other and / or additional position or timing for motion compensated prediction for the P-field. For example, a reference field in the same frame as the current P-field is allowed. Alternatively, the upper field or lower field of the frame may be coded / decoded first.

For an interlaced P-field with one or two reference fields for motion compensation, the P-field has only one reference field. Alternatively, the P-field may have two reference fields and switch between the two reference fields for different motion vectors or in some other way.

As another alternative, the P-field has more reference fields and / or has reference fields at different locations.

C. Encoder and Decoder

42 illustrates an example encoder framework 4200 for performing intensity estimation and compensation on interlaced P-fields with one or two reference fields. In this framework 4200, the encoder conditionally remaps the reference field using the parameters obtained by fading estimation. If the encoder detects fading with a good degree of confidence and consistency across the field, the encoder performs remapping or fading compensation. Otherwise, the fading compensation is an identity operation (ie output = input).

Referring to FIG. 42, the encoder uses the fading detection module 4230 to compare the current P-field 4210 with the first reference field 4220 to determine if fading occurs between the fields 4220 and 4210. . The encoder separately compares the current P-field 4210 and the second reference field 4225 using the fading detection module 4230 to determine if fading occurs between the fields 4225 and 4210. The encoder generates a “fading on” or “fading off” signal or signals 4240 based on the result of the fading detection. The signal (s) indicate whether fading compensation is to be used, and if used, for the first of both reference fields 4220, 4225 only, for the second field or both.

If fading compensation is on for first reference field 4220, fading estimation module 4250 estimates fading parameter 4260 for first reference field 4220. (Fade estimation details are described below.) Similarly, if fading compensation is on for the second reference field 4225, the fading estimation module 4250 determines the fading parameter for the second reference field. Estimate 4260 separately.

Fading compensation module 4270, 4275 remaps one or both of reference field 4220 using fading parameter 4260. Although FIG. 42 illustrates two fading compensation modules 4270 and 4275 (one per reference field), as an alternative, the encoder framework 4200 may operate on a single reference field 4220 and 4225. A fading compensation module.

Another encoder module 4280 (eg, motion estimation and compensation, frequency converter, and quantization module) compresses the current P-field 4210. The encoder outputs motion vectors, residuals, and other information 4290 that defines the encoded P-field 4210. Apart from motion estimation / compensation with translational motion vectors, the framework 4200 is applicable across a wide variety of motion compensation-based video codecs.

43 illustrates an example decoder framework 4300 for performing intensity compensation. The decoder generates a decoded P-field 4310. To decode the encoded fading-compensated P-field, the decoder uses fading compensation module 4370 and 4375 to perform fading compensation on one or two previously decoded reference fields 4320 and 4325. . As another alternative, the decoder framework 4300 includes a single fading compensation module that operates on either reference field 4320, 4325.

If the fading on / off signal (s) 4340 indicates that fading compensation has been used for the first reference field 4320 and the P-field 4310, the decoder may perform fading compensation for the first reference field 4320. To perform. Similarly, if the fading on / off signal (s) 4340 indicates that fading compensation has been used for the second reference field 4325 and the P-field 4310, then the decoder may apply to the second reference field 4325. Perform fading compensation for The decoder performs fading compensation using a separate set of fading parameters 4360 obtained during fading estimation for the first and second reference fields 4320 and 4325 (as in the encoder). If fading compensation is off, fading compensation is an equality operation (ie output = input).

Other decoder modules 4360 (eg, motion compensation, frequency inverse transformer, and inverse quantization module) may use the motion vector, residual, and other information 4390 provided by the encoder to encode the P-field 4310. Decompress

D. Parameterization  And reward

The parameter indicates fading, blending, morphing or other change between the P-field and the first reference field and / or between the P-field and the second reference field. The parameter is then applied in fading compensation.

In video editing, synthetic fading is sometimes realized by applying simple pixel-by-pixel linear transformations to luminance and chrominance channels. Similarly, cross-fading is sometimes realized as a linear summation of two video sequences, and this composition changes over time. Thus, in some embodiments, fading or other intensity compensation adjustment is parameterized as a pixel-by-pixel linear transformation and cross-fading is parameterized as a linear summation.

Assume that I (n) is a P-field n and I (n-1) is one reference field. If there is little motion, simple fading is modeled by the linear relationship in the equation below. The relationship in this equation is approximation due to the possible movement in the video sequence.

Here, fading parameters B1 and C1 correspond to brightness and contrast changes with respect to the reference field, respectively. (Parameters B2 and C2 respectively correspond to brightness and contrast changes for the remaining reference fields.) When nonlinear fading occurs, the primary component generally accounts for most of the change.

Cross-fading from image sequence U (n) to image sequence V (n) can be modeled with the relationship in the following equation. In other words, the relationship in the equation is approximation due to the possible movement in the sequence.

는 교차-페이딩의 시작을 나타내고,

는 교차-페이딩의 끝을 나타낸다. 몇개의 필드에 걸친 교차-페이딩의 경우,

가 작다. 교차-페이딩의 시작 시에, n번째 필드는 n-1번째 필드의 감쇄된(콘트라스트 < 1) 버전에 가깝다. 끝 무렵에, n번째 필드가 n-1번째 필드의 증폭된(콘트라스트 > 1) 버전이다.here,

Indicates the start of cross-fading,

Indicates the end of cross-fading. For cross-fading over several fields,

Is small. At the beginning of cross-fading, the nth field is close to the attenuated (contrast <1) version of the n-1th field. At the end, the nth field is an amplified (contrast> 1) version of the n-1th field.

The encoder performs strength compensation by remapping the reference field. The encoder remaps the reference field pixel by pixel or in other ways. The original unmapped reference field is discarded by default (though in some implementations, the non-remapped reference field can still be used for motion compensation).

로 재매핑한다.The following linear rule sets the luminance value of the reference field R to a reference field remapped with respect to two parameters B1 and C1.

Remap to.

The luminance value of the reference field is scaled (or "weighted") by the contrast value and shifted by the brightness value (ie, adds an offset). In the case of chrominance, remapping follows the following rules.

는 크로미넌스 값의 평균이다. 일 실시예에서, 128이 크로미넌스 값의 부호를 갖지 않는(unsigned) 8비트 표현에 대한 평균인 것으로 가정한다. 크로미넌스 재매핑에 대한 이 규칙은 밝기 성분을 사용하지 않는다. 어떤 실시예에 서, 2-파라미터 선형 재매핑은 고차항으로 확장된다. 예를 들어,

의 루미넌스 값을

로 재매핑하는 2차식은 다음과 같다.here,

Is the average of the chrominance values. In one embodiment, assume 128 is the average for an unsigned 8-bit representation of chrominance values. This rule for chrominance remapping does not use the brightness component. In some embodiments, two-parameter linear remapping extends to higher order terms. E.g,

The luminance value of

The quadratic equation remapping to

Other embodiments use different remapping rules. In one category of this remapping rule, for nonlinear fading, linear mapping is replaced with nonlinear mapping.

Fading compensation may be applied to the reference field before motion compensation. Alternatively, this may be applied to the reference field as needed during motion compensation, eg only to the region of the reference field actually referenced by the motion vector.

E. Estimation of Parameters

Estimation is the process of calculating the compensation parameters during the encoding process. An encoder such as that shown in framework 4200 of FIG. 42 calculates the brightness B1, B2 and contrast C1, C2 parameters during the encoding process. As another alternative, such an encoder calculates other compensation parameters.

To speed up the estimation, the encoder independently considers and estimates the parameters for each reference field. In addition, the encoder only analyzes the luminance channel. As another alternative, the encoder includes chrominance in the analysis when more computational resources are available. For example, the encoder solves C1 (or C2) in the luminance and chrominance remapping equations for the first reference field, not just the luminance remapping equation, to make C1 (or C2) more robust. .

Movement in the scene is ignored during the fading estimation process. This is based on the following observations: (a) fading and cross-fading generally occur in still or low motion scenes, and (b) the effectiveness of intensity compensation in high motion scenes is very low. As another alternative, the encoder jointly finds solutions to fading compensation parameters and motion information. The motion information is then used to improve the accuracy of the fading compensation parameter at later stages of this technique or at some other time. One way to use motion information is to omit that portion of the reference field from which motion was detected from the fading estimate calculation.

또는

의 절대 오차 합산은 페이딩의 존재 및 파라미터를 결정하기 위한 메트릭으로서 역할한다. 다른 대안으로서, 인코더는 동일한 오차 항에 걸친 제곱 오차 또는 평균 제곱 오차의 합 등의 다른 또는 부가의 메트릭을 사용하거나 인코더는 다른 오차 항을 사용한다.

or

The absolute error summation of 역할 serves as a metric for determining the presence and parameter of fading. As another alternative, the encoder uses other or additional metrics, such as the sum of squared errors or the mean squared error over the same error term, or the encoder uses different error terms.

The encoder can end the estimation upon satisfying the termination condition such as described below. For other termination conditions, the encoder checks whether the contrast parameter C1 (or C2) is close to 1.0 (in one implementation, .99 <C <1.02) at the start of the estimation or at an intermediate stage thereof, and if Terminate the skill.

The encoder begins to estimate by downsampling the current field and the selected reference field (first or second). In one implementation, the encoder downsamples four times horizontally and vertically. As another alternative, the encoder does not downsample or downsample at all by another multiple.

및

에 걸쳐 절대 오차 합산

을 계산한다. 절대 오차 합산은 다운샘플링된 현재 필드 및 다운샘플링된 참조 필드 간의 값의 차이를 측정한다. 절대 오차 합산이 어떤 임계값(예를 들어, 미리 정해진 차이 척도)보다 작은 경우, 인코더는 페이딩이 일어나지 않았거나 페이딩 보상이 사용되지 않은 것으로 결론짓는다.The encoder then performs a low resolution version of the current field and the reference field.

And

Sum the absolute error over

. Absolute error summation measures the difference in values between the downsampled current field and the downsampled reference field. If the absolute error sum is less than a certain threshold (eg, a predetermined difference measure), the encoder concludes that no fading has occurred or no fading compensation is used.

의 항으로

을 모델링함으로써 획득된다. 예를 들어, 밝기 및 콘트라스트 파라미터는 다운샘플링된 필드 전체에 걸친 선형 회귀(linear regression)를 통해 획득된다. 또는, 인코더는 보다 강건한 분석을 위해 총 최소 제곱(total least square), 최소 제곱 메디안(least median of square), 기타 등등의 다른 형태의 통계적 분석을 사용한다. 예를 들어, 인코더는 오차항

의 MSE 또는 SSE를 최소화한다. 어떤 환경에서, MSE 또는 SSE는 강건하지 않으며, 따라서 인코더는 또한 오차항에 대한 절대 오차 합산을 테스트한다. 인코더는 특정의 점에 대한 높은 오차값(이는 페이딩 보다는 움직임으로 인할 것일 수 있음)을 버린다.Otherwise, the encoder estimates the brightness B1 (or B2) and contrast C1 (or C2) parameters. The first cut estimate is for different parameter values

In terms of

Is obtained by modeling. For example, brightness and contrast parameters are obtained through linear regression across the downsampled field. Or, the encoder uses other forms of statistical analysis, such as total least square, least median of square, and the like, for more robust analysis. For example, the encoder

Minimize MSE or SSE. In some circumstances, MSE or SSE is not robust, so the encoder also tests the absolute error summation for the error term. The encoder discards high error values for certain points (which may be due to movement rather than fading).

The first cut parameters are quantized and dequantized to ensure that these parameters are within acceptable ranges and for conformance testing. In some embodiments, for a typical 8-bit depth image, the parameters are quantized to 6 bits each. B1 (or B2) takes an integer value from -32 to 31 represented as a signed 6-bit integer. C1 (or C2) varies from 0.5 to 1.484375 in a uniform step of 0.015625 (1/64), corresponding to the quantized values 0 through 63 for C1 (or C2). Quantization is performed by rounding B1 (or B2) and C1 (or C2) to the nearest valid dequantized value and selecting the appropriate bin index.

및 재매핑된 유한 절대 오차 합산

을 계산한다. 어떤 실시예에서, 인코더는 적합성(goodness-of-fit) 분석을 사용하여 이 합산을 계산한다. 원래의 해상도에서의 랜덤한 또는 의사 랜덤한 일련의 픽셀에 대해, 인코더는 재매핑된 유한 절대 오차 합산

을 계산하며, 여기서 인코딩되고 있는 필드의 양자화 파라미터의 배수 등의 어떤 한계(bound) M에 대해

이다. 한계 M은 양자화 파라미터가 거칠 때 더 높고, 양자화 파라미터가 미세할 때 더 낮다. 인코더는 또한 원래의 유한 절대 오차 합산

을 누적한다. 계산 자원이 이용가능한 경우, 인코더는 필드 전체에 걸쳐 유한 오차 합산을 계산할 수 있다.The encoder sums the original bounded absolute error

And remapped finite absolute error summation

. In some embodiments, the encoder calculates this summation using a goodness-of-fit analysis. For a random or pseudo-random series of pixels at the original resolution, the encoder sums the remapped finite absolute error

For any bound M, such as a multiple of the quantization parameter of the field being encoded.

to be. The limit M is higher when the quantization parameter is coarse and lower when the quantization parameter is fine. The encoder also adds the original finite absolute error

Accumulate. If computational resources are available, the encoder can calculate finite error summation throughout the field.

보다 작거나 같지 않는 한 페이딩 보상을 수행하지 않는다. 한 구현에서,

= .95이다.Based on the relative values of the original and remapped finite absolute error summation, the encoder determines whether to use fading compensation. For example, in some embodiments, the encoder is such that the remapped finite absolute error sum is some threshold percentage of the original finite absolute error sum.

Do not perform fading compensation unless it is less than or equal to. In one implementation,

= .95.

If fading compensation is used, the encoder then recalculates the fading parameters at full resolution based on a linear regression between I (n) and R. To save computation time, the encoder can perform repeated linear regression over random or pseudo-random sampling of the field. In other words, the encoder may use other forms of statistical analysis (eg, total least squares, least square median, etc.) for more robust analysis as another alternative.

In some implementations, the encoder allows a special case where the reconstructed value of C1 (or C2) is -1. This special case is signaled by the syntax element for C1 (or C2) being zero. In this "invert" mode, the reference field is inverted before transitioning by B1 (or B2), and the range of B1 (or B2) is 193 to 319 in two uniform steps. As another alternative, some or all of the fading compensation parameters may use different representations or other and / or additional parameters.

F. Signaling

At a higher level, the signaled fading compensation information includes (1) compensation on / off information and (2) compensation parameters. The on / off information is in turn determined by (a) whether fading compensation is allowed globally (e.g. for the entire sequence) or not (b) if fading compensation is allowed, the fading compensation is assigned to a specific P-field. Whether or not the reference field should be adjusted by fading compensation if fading compensation is used for a particular P-field. When fading compensation is used for the reference field, the fading compensation parameter applied is as follows.

1. Full on / off off Signaling

At the sequence level, one bit indicates whether fading compensation is enabled for that sequence. If fading compensation is allowed, later factors indicate when and how it is performed. As another alternative, fading compensation is enabled / disabled at some other syntax level. Or, fading compensation is always allowed, and overall on / off signaling is omitted.

2. P-field on / off off Signaling

If fading compensation is allowed, one or more additional signals indicate when to use fading compensation. Among the fields in a typical interlaced video sequence, intensity compensation is rarely performed. It is possible to signal the use of fading compensation for the P-field by adding 1 bit per field (eg 1 bit signaled at the field level). However, it is more economical to jointly signal the use of fading compensation with other information.

One option is to jointly signal the use of fading compensation for the P-field with the motion vector mode (eg, number and configuration of motion vectors, sub-pixel interpolation schemes, etc.). For example, VLC jointly informs the minimum frequency motion vector mode and the activity of fading compensation for the P-field. For further details, see US Patent Application Publication No. 2003-0206593-A1 entitled "Fading Estimation / Compensation". Or, use / disuse of fading compensation for the P-field is signaled along with the motion vector mode information, as described in some combination implementations below. See the MVMODE and MVMODE2 elements in section XII. As another alternative, another mechanism for signaling P-field fading compensation on / off information is used.

3. Reference Fields On / Off off Signaling

If fading compensation is used for the P-field, there may be several options for which reference field receives fading compensation. If the P-field uses fading compensation and has two reference fields, there are three cases. Fading compensation includes (1) for both reference fields, (2) only for a first reference field (e.g., the second most recent reference field in time), and (3) a second reference field (e.g., Only for the most recent reference field in time). Fading compensation reference field pattern information may be signaled as FLC or VLC per P-field. The table of FIG. 44 shows one VLC set for pattern information of an element INTCOMPFIELD signaled with a P-field header. As another alternative, the table or other table shown in FIG. 47G is used at the field level or other syntax level.

In some implementations, the reference field pattern for fading compensation is signaled for all P-fields. As another alternative, for one reference field P-field using fading compensation, signaling of the reference field pattern is omitted because the fading compensation is automatically applied to a single reference field.

4. fading  Compensation parameters Signaling

If fading compensation is used for the reference field, the fading compensation parameter for the reference field is signaled. For example, the first fading compensation parameter set is in the header for the P-field. If fading compensation is used for only one reference field, the first parameter set is for that one reference field. If fading compensation is used for two reference fields of the P-field, the first parameter set is for one reference field and the second fading compensation parameter set is in the header for fading compensation of the remaining reference fields.

Each fading compensation parameter set includes, for example, a contrast parameter and a brightness parameter. In one combined implementation, the first parameter set includes LUMSCALE1 and LUMSHIFT1 elements, which are present in the P-field header when strength compensation is signaled for the P-field. If INTCOMPFIELD indicates that both reference fields or only the second most recent reference field uses fading compensation, LUMSCALE1 and LUMSHIFT1 are applied to the second most recent reference field. Otherwise (when INTCOMPFIELD indicates that only the most recent reference field uses fading compensation), LUMSCALE1 and LUMSHIFT1 apply to the most recent reference field. When the intensity compensation is signaled for the P-field and INTCOMPFIELD indicates that both reference fields use fading compensation, a second parameter set containing the LUMSCALE2 and LUMSHIFT2 elements is present in the P-field header. LUMSCALE2 and LUMSHIFT2 apply to more recent reference fields.

LUMSHIFT1, LUMSCALE1, LUMSHIFT2, and LUMSCALE2 correspond to parameters B1, Cl, B2, and C2. LUMSCALE1, LUMSCALE2, LUMSHIFT1, and LUMSHIFT2 are each signaled using 6-bit FLC. As another alternative, the parameter is signaled using VLC. FIG. 56 illustrates pseudo code for performing fading compensation on a first reference field based on LUMSHIFT1 and LUMSCALE1. A similar process is performed for fading compensation on the second reference field based on LUMSHIFT2 and LUMSCALE2.

As another alternative, the fading compensation parameter has a different representation and / or is signaled with a different signaling mechanism.

G. Estimation and Signaling  Technology

An encoder such as encoder 2000 of FIG. 20 or an encoder in framework 4200 of FIG. 42 performs fading estimation and corresponding signaling for an interlaced P-field having two reference fields. For example, the encoder performs the technique 4500 shown in FIG. 45A.

The encoder performs fading detection 4510 on the first of two reference fields for the P-field. If fading is detected (“Yes” path from decision step 4512), the encoder performs fading estimation 4524 of the P-field for the first reference field, which is a fading compensation parameter for the first reference field. Create The encoder also performs fading detection 4520 for the second of the two reference fields for the P-field. If fading is detected (“Yes” path from decision step 4522), the encoder performs fading estimation 4524 of the P-field for the second reference field, which is a fading compensation parameter for the second reference field. Create For example, the encoder performs fading detection and estimation as described in the section entitled "Estimate of Fading Parameters." As another alternative, the encoder uses other techniques to detect fading and / or obtain fading compensation parameters. If the current P-field has only one reference field, the operation for the second reference field can be omitted.

The encoder signals whether fading compensation is on or off for the P-field (step 4530). For example, the encoder jointly codes this information with motion vector mode information for the P-field. As another alternative, the encoder uses other and / or additional signals to indicate whether fading compensation is on or off for the P-field. If fading compensation is not on for the current P-field (“No” path from decision step 4532), the description 4500 ends.

If not (“Yes” path from decision step 4532), the encoder signals a reference field pattern for fading compensation (step 4540). For example, the encoder signals a VLC indicating whether fading compensation is used for both reference fields, only for the first reference field or only for the second reference field. As another alternative, the encoder uses another signaling mechanism (eg, FLC) to inform the reference field pattern. In this path, the encoder also signals the first and / or second fading compensation parameter set calculated in the fading estimate (step 4542). For example, the encoder uses signaling as described in section XI.F. As another alternative, the encoder uses other signaling.

Although the encoder generally performs fading compensation, motion estimation and motion compensation as well, for simplicity, FIG. 45A does not show these operations. In addition, fading estimation may be performed prior to or concurrent with motion estimation. 45A does not represent the various ways in which this technique 4500 may be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

H. Decoding and Compensation Technology

A decoder such as decoder 2100 of FIG. 21 or a decoder in framework 4300 of FIG. 43 performs decoding and fading compensation on an interlaced P-field having two reference fields. For example, the decoder performs the technique 4550 shown in FIG. 45B.

The decoder receives and decodes one or more signals that indicate whether fading compensation is on or off for the P-field (step 4560). For example, this information is coded jointly with motion vector mode information for the P-field. As another alternative, the decoder receives and decodes other and / or additional signals to indicate whether fading compensation is on or off for the P-field. If fading compensation is not on for the P-field (the "no" path from decision step 4652), this technique 4550 ends.

If not (“Yes” path from decision step 4462), the decoder receives and decodes the reference field pattern for fading compensation (step 4570). For example, the decoder receives and decodes VLC indicating whether fading compensation is used for both reference fields, only for the first reference field, or only for the second reference field. As another alternative, the decoder operates in conjunction with another signaling mechanism (eg, FLC) to determine the reference field pattern.

In this path, the decoder also receives and decodes the first set of fading compensation parameters (step 4572). For example, the decoder operates with signaling as described in section XI.F. As another alternative, the decoder operates with other signaling.

If fading compensation is performed on only one of the two reference fields (the "no" path from decision step 4475), the first parameter set is applied to the first or second reference field as indicated by the reference field pattern. will be. The decoder performs fading compensation on the indicated reference field with the first set of fading compensation parameters (step 4592) and the description 4500 ends.

Otherwise, fading compensation is performed on both reference fields (“YES” path from decision step 4475), and the decoder receives and decodes the first set of fading compensation parameters (step 4580). For example, the decoder operates with signaling as described in section XI.F. As another alternative, the decoder operates with other signaling. In this case, the first parameter set is for one of two reference fields and the second parameter set is for the remaining reference fields. The decoder performs fading compensation on one reference field with the first parameter set (step 4592) and fading compensation on the other reference field with the second parameter set (step 4582).

For simplicity, FIG. 45B does not represent various ways in which this technique 4550 can be integrated with other aspects of encoding and decoding. Various combined implementations are described in detail in section XII.

XII . Combined implementation

We now describe a detailed combined implementation for bitstream syntax, semantics, and decoders, centered on interlaced P-fields. The following description includes a first combined implementation and an alternative second combined implementation. In addition, US patent application Ser. No. 10 / 857,473, filed May 27, 2004, discloses aspects of a third combined implementation.

While it is centered on interlaced P-fields, in many places in this section, it is possible to use different picture types (for example, interlaced P-frames and B-frames, interlaced I, BI, PI, and B-fields). Reference is also made to the applicability of syntax elements, semantics and decoding.

A. In the first combined implementation sequence  And Semantics

In a first combined implementation, the compressed video sequence consists of data structured in a hierarchical structure, that is, a picture layer, a macroblock layer and a block layer. The sequence layer may precede the sequence and the entry point layer may be interspersed in the sequence. 46A to 46E illustrate bitstream elements constituting various layers.

One. sequence  hierarchy Syntax  And Semantics

The sequence-level header includes a sequence-level parameter used to decode the compressed picture sequence. In some profiles, sequence-related metadata is delivered to the decoder by the transport layer or other means. However, for profiles (advanced profiles) with interlaced P-fields, this header syntax is part of the video data bitstream.

46A illustrates syntax elements that constitute a sequence header for an advanced profile. The PROFILE 4601 and LEVEL 4602 elements define the profile used to encode the sequence and encoding level into the profile, respectively. Of particular interest in the case of interlaced P-fields is that the INTERLACE 4603 element is a 1-bit syntax element that signals whether the source content is sequential (INTERLACE = 0) or interlaced (INTERLACE = 1). When INTERLACE = 1, individual frames can still be coded using sequential or interlaced syntax.

2. Entry point hierarchy Syntax  And Semantics

Entry point headers exist in the advanced profile. The entry point has two purposes. First, the entry point is used to signal a random access point in the bitstream. Second, the entry point is used to signal a change in the coding control parameter.

46B illustrates syntax elements constituting an entry point hierarchy. Of particular interest for interlaced P-fields is that the reference frame distance flag REFDISTFLAG 4611 element is a 1-bit syntax element. REFDISTFLAG = 1 indicates that a REFDIST 4624 element is present in the I / I, I / P, P / I or P / P field picture header. REFDISTFLAG = 0 indicates that a REFDIST 4624 element is not present in the I / I, I / P, P / I or P / P field picture header.

The extended motion vector flag EXTENDED_MV 4612 element is a 1-bit element indicating whether the extended motion vector function is on (EXTENDED_MV = 1) or off (EXTENDED_MV = 0). The extended differential motion vector range flag EXTENDED_DMV 4613 element is a 1-bit syntax element that exists when EXTENDED_MV = 1. If EXTENDED_DMV = l, motion vector differences in the extended differential motion vector range are signaled in the picture layer within the entry point segment. If EXTENDED_DMV = 0, motion vector differences in the extended differential motion vector range are not signaled. Extended differential motion vector range is an option for interlaced P-pictures and B-pictures, including interlaced P-fields and P-frames and interlaced B-fields and B-frames.

3. Picture  hierarchy Syntax  And Semantics

Data for a picture consists of data for the picture header and the following macroblock layer. 46C shows a bitstream element constituting a frame header for an interlaced field picture. In the following description, the emphasis is on elements used with interlaced P-fields, but the header shown in FIG. 46C is applicable to various combinations of interlaced I-, P-, B- and BI-fields. .

The frame coding mode FCM 4651 element is present only in the advanced profile and only when the sequence layer INTERLACE 4603 has a value of 1. FCM 4651 indicates whether the picture is a sequential, interlaced-field or interlaced frame. The table of FIG. 47A includes the VLC used to indicate the picture coding type to the FCM.

The field picture type FPTYPE 4462 element is a 3-bit syntax element present in the picture header for an interlaced field picture. The FPTYPE is decoded according to the table of Fig. 47B. As the table shows, an interlaced frame includes two interlaced I-fields, one interlaced I-field and one interlaced P-field, two interlaced P-fields, two interlaced B-fields, It may include one interlaced B-field and one interlaced BI-field or two interlaced BI-fields.

The upper field first TFF 4603 element is a 1-bit element present in the advanced profile picture header when the sequence header element PULLDOWN = 1 and the sequence header element INTERLACE = 1. TFF = 1 implies that the upper field is the first decoded field. If TFF = 0, the lower field is the first decoded field.

The P reference distance REFDIST 4624 element is a variable present in the interlaced field picture header when the entry level flag REFDIST_FLAG = 1 and when the picture type is not B / B, B / BI, BI / B, BI / BI. Size syntax element. If REFDIST_FLAG = 0, REFDIST 4624 is set to a default value of zero. REFDIST 4624 indicates the number of frames between the current frame and the reference frame. The table of FIG. 47C includes the VLC used for the REFDIST 4624 value. The last row in the table represents a codeword used to indicate a reference frame distance greater than two. They are coded as 1 (binary) 11 and N-3 (where N is the reference frame distance). The last bit in the codeword is zero. The value of REFDIST 4624 is less than or equal to 16. E.g,

N = 3, VLC codeword = 110, VLC size = 3,

N = 4, VLC codeword = 1110, VLC size = 4, and

N = 5, VLC codeword = 11110, VLC size = 5.

The field picture layer FIELDPICLAYER 4625 element is data for one of the individual interlaced fields of the interlaced frame. If the interlaced frame is a P / P frame (FPTYPE = 011), the bitstream includes two FIELDPICLAYER 4625 elements for two interlaced P-fields. 46D illustrates bitstream elements constituting a field picture header for an interlaced P-field picture.

The number of reference picture NUMREF (4631) elements is a 1-bit syntax element present in the interlaced P-field header. This indicates whether the interlaced P-field has one (NUMREF = 0) or two (NUMREF = 1) reference fields. Reference field picture indicator REFFIELD 4452 is a 1-bit syntax element present in an interlaced P-field when NUMREF = 0. This indicates which of the two possible reference pictures the interlaced P-field uses.

Extended MV range flag MVRANGE 4463 is a variable-size syntax element that generally indicates an extended range for the motion vector (ie, a longer possible horizontal and / or vertical displacement for the motion vector). Extended differential MV range flag DMVRANGE 4464 is a variable-size syntax element that exists when EXTENDED_DMV = 1. The table of FIG. 47D is used for the DMVRANGE 4464 element. Both MVRANGE 4463 and DMVRANGE 4464 are used to decode motion vector differences and extended differential motion vector ranges are applied to interlaced P-fields, interlaced P-frames, interlaced B-fields, and interlaced B-frames. Is an option.

The motion vector mode MVMODE 4635 element is a variable-size syntax element that signals one or one intensity compensation mode of four motion vector coding modes. The motion vector coding mode includes three "1MV" modes with different sub-pixel interpolation rules for motion compensation. 1 MV means that each macroblock in a picture has at most one motion vector. In the "mixed-MV" mode, each macroblock in a picture may have one or four motion vectors or may be skipped. Depending on the value of PQUANT (quantization coefficient for the picture), any of the tables shown in FIG. 47E is used for the MVMODE 4635 element.

The motion vector mode 2 MVMODE2 4636 element is a variable-size syntax element present in the interlaced P-field header when the MVMODE 4635 signals intensity compensation. Depending on the value of PQUANT, any one of the tables shown in FIG. 47F is used for the MVMODE 4635 element.

The intensity compensation field INTCOMPFIELD 4637 is a variable-size syntax element present in the interlaced P-field picture header. As shown in the table of FIG. 47G, INTCOMPFIELD 4637 is used to indicate which reference field (s) are intensity compensated. INTCOMPFIELD 4637 is present even when NUMREF = 0.

The field picture luma scale 1 LUMSCALE1 (4638), field picture luma transition 1 LUMSHIFT1 (4639), field picture luma scale 2 LUMSCALE2 (4640), and field picture luma transition 2 LUMSHIFT2 (4641) elements are each used for intensity compensation. Bit value. The LUMSCALE1 4638 and LUMSHIFT1 4739 elements are present when the MVMODE 4635 signals intensity compensation. If the INTCOMPFIELD 4637 element is '1' or '00', LUMSCALE1 4638 and LUMSHIFT1 4639 apply to the upper field. Otherwise, LUMSCALE1 4638 and LUMSHIFT1 4639 apply to the lower field. The LUMSCALE2 4640 and LUMSHIFT2 4641 elements are present when the MVMODE 4635 signals intensity compensation and the INTCOMPFIELD 4637 element is '1'. LUMSCALE2 4640 and LUMSHIFT2 4641 apply to the lower field.

The macroblock mode table MBMODETAB 4464 element is a fixed length field with a 3-bit value for the interlaced P-field header. MBMODETAB 4464 indicates which of the eight code tables (tables 0 through 7 specified as 3-bit values) is used to encode / decode the macroblock mode MBMODE 4471 syntax element in the macroblock layer. There are two sets of eight code tables, and the set used depends on whether 4MV macroblocks are possible in the picture as indicated by MVMODE 4465. 47H shows eight tables available for MBMODE 461 in an interlaced P-field in mixed-MV mode. 47I shows eight tables available for MBMODE 461 in an interlaced P-field in 1MV mode.

The motion vector table MVTAB 4641 element is a fixed length field. For an interlaced P-field with NUMREF = 0, MVTAB 4641 has a 2-point indicating which of the four code tables (tables 0 to 3, specified as 2-bit values) are used to decode the motion vector data. Bit syntax element. For interlaced P-fields with NUMREF = 1, MVTAB 4641 indicates that which of the eight code tables (tables 0 through 7 specified as 3-bit values) is used to encode / decode the motion vector data. -Bit syntax element.

In the interlaced P-field header, the 4MV block pattern table 4MVBPTAB 4644 element is MVMODE 4635 (or MVMODE2 4636, if MVMODE 4635 is set to strength compensation), indicating that the picture is of mixed-MV type. It is a 2-bit value that exists if indicated. The 4MVBPTAB 4644 syntax element signals which of the four tables (tables 0 to 3 specified as 2-bit values) is used for the 4MV block pattern 4MVBP 4466 syntax element in the 4MV macroblock. 47J shows four tables available for 4MVBP 4466.

The interlaced P-frame header (not shown) has many of the same elements as the field-coded interlaced frame header shown in FIG. 46C and the interlaced P-field shown in FIG. 46D. These include FCM 4651, MVRANGE 4463, DMVRANGE 4464, MBMODETAB 4462, and MVTAB 4641, but the exact syntax and semantics for interlaced P-frames may differ from interlaced P-fields. have. The interlaced P-frame header also includes other elements for picture type and intensity compensation signaling to switch between 1MV and 4MV modes.

Since interlaced P-frames may include field-coded macroblocks with two motion vectors per macroblock, the interlaced P-frame header includes two motion vector block pattern table 2MVBPTAB elements. 2MVBPTAB is a 2-bit value present in an interlaced P-frame. This syntax element signals which of the four tables (tables 0 to 3 specified as 2-bit values) is used to decode the 2MV block pattern (2MVBP) in the 2MV field-coded macroblock. 47K shows four tables available for 2MVBP.

Interlaced B-fields and interlaced B-frames have many of the same elements of interlaced P-fields and interlaced P-frames. In particular, the interlaced B-field may include a 4MVBPTAB 4644 syntax element. Interlaced B-frames include both 2MVBPTAB and 4MVBPTAB 4644 syntax elements, but the semantics of the elements may be different.

4. Macroblock Layer Syntax  And Semantics

The data for a macroblock consists of a macroblock header and the following block hierarchy. 46e shows a macroblock hierarchy for an interlaced P-field.

The macroblock mode MBMODE 4661 element is a variable-size element. This indicates the number of motion vectors for the macroblock (1MV, 4MV or intra), whether there is a block pattern CBPCY 4466 element coded for the macroblock, and (in some cases) the motion vector difference data for the macroblock. Point out information such as whether it exists or not. 47H and 47I show tables available for MBMODE 461 for interlaced P-fields.

The motion vector data MVDATA 4703 element is a variable-size element that encodes motion vector information (eg, horizontal and vertical differentials) for the motion vector. For interlaced P-fields with two reference fields, MVDATA 4703 also encodes information for selecting between a number of possible motion vector predictors for the motion vector.

The four motion vector block pattern 4MVBP 4466 element is a variable-size syntax element that can exist in macroblocks for interlaced P-fields, B-fields, P-frames, and B-frames. In the macroblocks for interlaced P-fields, B-fields, and P-frames, the 4MVBP 4466 element is present when MBMODE 4661 indicates that the macroblock has four motion vectors. In this case, 4MVBP 4466 indicates which of the four luma blocks contains non-zero motion vector differences.

In the macroblock for an interlaced B-frame, 4MVBP 4466 is present when MBMODE 461 indicates that the macroblock contains two field motion vectors and when the macroblock is an interpolated macroblock. In this case, 4MVBP 4466 indicates which of the four motion vectors (upper and lower field forward motion vectors, and upper and lower field backward motion vectors) are present.

The two motion vector block pattern 2MVBP element (not shown) is a variable-size syntax element present in macroblocks in interlaced P-frames and B-frames. In an interlaced P-frame macroblock, 2MVBP is present when MBMODE 461 indicates that the macroblock has two field motion vectors. In this case, 2MVBP indicates which of the two fields (top and bottom) contains a non-zero motion vector difference. In an interlaced B-frame macroblock, 2MVBP is present when MBMODE 461 indicates that the macroblock contains one motion vector and that the macroblock is an interpolated macroblock. In this case, 2MVBP indicates which of the two motion vectors (front and back motion vectors) is present.

The block-level motion vector data BLKMVDATA 4665 element is a variable-size element that exists in some situations. This includes motion information for blocks of macroblocks.

Hybrid Motion Vector Prediction HYBRIDPRED 4666 is a 1-bit syntax element per motion vector that may exist in a macroblock in an interlaced P-field. If hybrid motion vector prediction is used, HYBRIDPRED 4666 indicates which of the two motion vector predictors should be used.

5. Block Layer Syntax  And Semantics

The block layer for interlaced pictures follows the syntax and semantics of the block layer for sequential pictures. In general, information about the DC and AC coefficients of blocks and sub-blocks is signaled at the block layer.

B. Decoding in the First Combined Implementation

If the video sequence consists of interlaced video frames or includes a mixture of interlaced frames and sequential frames, the FCM 4612 element indicates whether a given picture is coded as a sequential frame, interlaced field, or interlaced frame. For a frame coded as an interlaced field, FPTYPE 4462 indicates that the frame has two interlaced I-fields, one interlaced I-field and one interlaced P-field, two interlaced P-fields, and two. Whether it includes two interlaced B-fields, one in-B-field and one interlaced BI-field or two interlaced BI-fields. The decoding of the interlaced field is as follows. The following section focuses on the decoding process for interlaced P-fields.

One. Interlaced  Reference to P-field Decoding

An interlaced P-field may refer to one or two previously decoded fields in motion compensation. The NUMREF 4463 element indicates whether the current P-field can reference one or two previous reference fields. If NUMREF = 0, the current P-field may refer to only one field. In this case, the REFFIELD (4632) element follows into the bitstream. REFFIELD 4452 indicates which previously decoded field is used as a reference. If REFFIELD = 0, the nearest I-field or P-field in time (in display order) is used as a reference. If REFFIELD = 1, the second most recent I-field or P-field in time is used as a reference. If NUMREF = 1, the current P-field uses the two closest I-fields or P-fields in time (in display order) as references. Examples of reference field pictures for NUMREF = 0 and NUMREF = 1 shown in FIGS. 24A-F apply to the first combined implementation as described above.

2. Picture  type

Interlaced P-fields can be of two types: 1MV or mixed-MV. In a 1MV P-field, each macroblock is a 1MV macroblock. In a mixed-MV P-field, each macroblock can be encoded as a 1MV or 4MV macroblock. The 1 MV or mixed-MV mode is signaled for the interlaced P-field by the MVMODE 4635 and MVMODE 2 4636 elements.

3. Macro Block mode

Macroblocks in interlaced P-fields can be one of three possible types: 1MV, 4MV and intra. The MBMODE 4471 element also indicates the macroblock type (1MV, 4MV, intra) and the presence of CBP and MV data. Depending on whether the MVMODE 4635 / MVMODE2 4636 syntax element is interlaced P-field is mixed-MV or both 1MV, MBMODE 461 signals information as follows.

The table in FIG. 26 shows how MBMODE 461 all signals information about macroblocks in the 1MV P-field. As shown in FIG. 47I, one of the eight tables is used to encode / decode MBMODE 461 for the 1MV P-field. The table in FIG. 27 shows how MBMODE 461 signals information about macroblocks in the mixed-MV P-field. As shown in FIG. 47H, one of eight tables is used to encode / decode MBMODE 4471 for the Mixed-MV P-field.

Thus, a 1MV macroblock may occur in 1-MV and mixed-MV interlaced P-fields. A 1MV macroblock is a macroblock when a single motion vector represents a displacement between the current picture and the reference picture for all six blocks in the macroblock. For a 1MV macroblock, the MBMODE (4661) element has three elements: (1) the macroblock type is 1MV, (2) whether a CBPCY (4662) element exists for the macroblock, and (3) the macroblock. Indicates whether an MVDATA (4663) element exists for.

If the MBMODE 4661 element indicates that a CBPCY 4702 element exists, then the CBPCY 4466 element is present in the macroblock hierarchy at the corresponding location. CBPCY 4462 indicates which of the six blocks is coded in the block layer. If the MBMODE 461 element indicates that CBPCY 4462 does not exist, then CBPCY 4462 is assumed to be zero and there is no block data for any of the six blocks in the macroblock.

If the MBMODE 4661 element indicates that an MVDATA 4663 element is present, then the MVDATA 4466 element is present in the macroblock hierarchy at the corresponding location. The MVDATA 4703 element encodes a motion vector difference synthesized with a motion vector predictor to reconstruct the motion vector. When the MBMODE 4661 element indicates that the MVDATA 4703 element does not exist, the motion vector difference is assumed to be zero, so the motion vector is equal to the motion vector predictor.

The 4MV macroblock is in the mixed-MV P-field. A 4MV macroblock is a macroblock when each of the four luma blocks in the macroblock can have an associated motion vector indicating the displacement between the current picture and the reference picture for that block. The displacement for the chroma block is derived from four luma motion vectors. The difference between the current block and the reference block is encoded in the block layer. For a 4MV macroblock, the MBMODE 4466 element indicates two things: (1) the macroblock type is 4MV, and (2) the CBPCY 4662 element is present.

Intra macroblocks can occur in 1MV or mixed-MV P-fields. An intra macroblock is a macroblock when all six blocks are coded without reference to any previous picture data. For intra macroblocks, the MBMODE 4661 element indicates two things: (1) the macroblock type is intra, and (2) the CBPCY 4662 element is present. In the case of an intra macroblock, if a CBPCY 4702 element is present, this element indicates which of the six blocks has AC coefficient data coded in the block layer. The DC coefficient is still present for each block in all cases.

4. Movement Vector Block Pattern

The 4MVBP 4466 element indicates which of the four luma blocks contains non-zero motion vector differences. 4MVBP 4466 is decoded to a value of 0 to 15 representing a bit syntax element indicating whether a motion vector for the corresponding luma block is present when represented as a binary value. The table in FIG. 34 shows the association for 4MVBP 4466 of the luma block. As shown in FIG. 47J, one of the four tables is used to encode / decode 4MVBP 4466.

For each of the four bit positions in 4MVBP 4466, a value of zero indicates that there is no motion vector difference (in BLKMVDATA) for the block at the corresponding position, and the motion vector difference is assumed to be zero. A value of 1 indicates that there is a motion vector difference (in BLKMVDATA) for the block at the corresponding position. For example, if 4MVBP 4466 is decoded to a binary value of 1100, the bitstream includes BLKMVDATA 4665 for blocks 0 and 1, and there is no BLKMVDATA 4665 for blocks 2 and 3. 4MVBP 4466 is similarly used to indicate the presence or absence of motion vector difference information for 4MV macroblocks in interlaced B-fields and interlaced P-frames.

A field-coded macroblock in an interlaced P-frame or an interlaced B-frame may comprise two motion vectors. For a two field MV macroblock, the 2MVBP element indicates which of the two fields has a non-zero differential motion vector. As shown in Fig. 47K, one of the four tables is used to encode / decode 2MVBP.

5. Field Picture  Coordinate system

In the following sections, the motion vector unit is expressed in field picture units. For example, if the vertical component of the motion vector indicates that the displacement is +6 (in quarter-pixel units), then this indicates the displacement of the 1½ field picture line.

FIG. 48 illustrates the relationship between the vertical component of the motion vector and the spatial position for both the combination (opposite and equal) of the current and reference field polarities. 48 shows one vertical pixel column in the current field and the reference field. Circles represent integer pixel positions and x represents quarter-pixel positions. A value of zero indicates no vertical displacement between the current field position and the reference field position. If the current field and the reference field are of opposite polarity, the 0 vertical vector indicates the position (½-pixel transition) in the middle of the field lines in the reference field. If the current field and the reference field are of the same polarity, the zero vertical vector points to the corresponding field line in the reference field.

6. Decoding Motion Vector Difference

The MVDATA 4703 and BLKMVDATA 4665 elements encode motion information for the macroblock or blocks within the macroblock. The 1MV macroblock may have a single MVDATA 4466 element and the 4MV macroblock may have 0-4 BLKMVDATA 4665. The process of calculating the motion vector difference from MVDATA 4703 or BLKMVDATA 4665 is different for the one-reference (NUMREF = 0) case and the two-reference (NUMREF = 1) case.

In a field picture with only one reference field, each MVDATA 4703 or BLKMVDATA 4665 syntax element jointly encodes two, (1) horizontal motion vector difference component and (2) vertical motion vector difference component. do. The MVDATA 4703 or BLKMVDATA 4665 element is a VLC followed by a FLC. The value of VLC determines the size of the FLC. The MVTAB 4641 syntax element specifies the table used to decode the VLC.

49A illustrates pseudo code describing motion vector differential decoding for a block or macroblock in a field picture having one reference field. In this pseudo code, the values dmv_x and dmv_y are calculated, where dmv_x is the differential horizontal motion vector component and dmv_y is the differential vertical motion vector component. The variables k_x and k_y are fixed length values that depend on the motion vector range defined by MVRANGE 4463 according to the table shown in FIG. 49B.

The variable extend_x is for extended range horizontal motion vector difference and the variable extend_y is for extended range vertical motion vector difference. The variables extend_x and extend_y are derived from the DMVRANGE 4646 syntax element. If DMVRANGE 4464 indicates that an extended range for the horizontal component is used, extend_x = 1. Otherwise, extend_x = 0. Similarly, if DMVRANGE 4464 indicates that an extended range for the vertical component is used, extend_y = 1. Otherwise, extend_y = 0. offset_table is an array defined as follows.

offset_tablel [9] = {0, 1, 2, 4, 8, 16, 32, 64, 128,} and

offset_table2 [9] = {0, 1, 3, 7, 15, 31, 63, 127, 255},

Here, offset_table2 [] is used for horizontal or vertical components when the difference range for that component is extended. While FIGS. 49A and 49B show extended differential motion vector decoding for interlaced P-fields, extended differential motion vector decoding in a first combined implementation also provides for interlaced B-fields, interlaced P-frames, and It is also used for interlaced B-frames.

In a field picture with two reference fields, each MVDATA 4663 or BLKMVDATA 4665 syntax element has three elements: (1) horizontal motion vector difference component, (2) vertical motion vector difference component, and (3) Jointly encode whether a dominant or non-dominant predictor is used, ie which of the two fields is referenced by the motion vector. As in the case of the 1 reference field, the MVDATA (4663) or BLKMVDATA (4665) element is the VLC and the following FLC, the value of the VLC determines the size of the FLC, and the MVTAB 4464 syntax element is used to decode the VLC. Specifies the table to be used.

FIG. 50 illustrates pseudo code describing motion vector difference and dominant / non-dominant predictor decoding for a motion vector of a block or macroblock in a field picture having two reference fields. In this pseudo code, the value predictor_flag is a binary flag indicating whether a dominant or non-dominant motion vector predictor is used. If predictor_flag = 0, the dominant predictor is used. If predictor_flag = 1, the non-dominant predictor is used. Various other variables (including dmv_x, dmv_y, k_x, k_y, extend_x, extend_y, offset_tablel [], and offset_table2 []) are as described for the one reference field case. The table size_table is an array defined as follows.

size_table [l6] = {0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7}.

7. Motion vector Predictor

The motion vector is calculated by adding the motion vector difference calculated in the previous section to the motion vector predictor. The predictor is calculated from up to three neighboring motion vectors. Even if the motion vector mode is 1 / 2-pixel, the calculation for the motion vector predictor is done in quarter pixels.

In a 1MV interlaced P-field, up to three neighboring motion vectors are used to calculate the predictor for the current macroblock. The location of the neighboring macroblock with the motion vector under consideration is as shown in FIGS. 5A and 5B and described for the 1MV sequential P-frame.

In a mixed-MV interlaced P-field, up to three neighboring motion vectors are used to calculate the predictor for the current block or macroblock. The location of the neighboring block and / or macroblock with the motion vector under consideration is as shown in FIGS. 6A-10 and described for the mixed-MV sequential P-frame.

If the NUMREF 4471 syntax element in the picture header is zero, the current interlaced P-field may only refer to one previously coded field. If NUMREF = 1, the current interlaced P-field may refer to the two most recent reference field pictures. In the former case, a single predictor is calculated for each motion vector. In the latter case, two motion vector predictors are calculated. 51A and 51B show how the motion vector predictor is calculated for the 1 reference field case. The variables fieldpred_x and fieldpred_y in the pseudo code represent the horizontal and vertical components of the motion vector predictor.

2 Reference Fields In an interlaced P-field (NUMREF = 1), the current field may refer to the two most recent reference fields. In this case, two motion vector predictors are calculated for each inter-coded macroblock. One predictor is from a reference field of the same polarity and the other is from a reference field with the opposite polarity. Of the same and opposite polarity fields, one is the dominant field and the other is the non-dominant field. The dominant field is a field containing most of the motion vector predictor candidates. In case of an even number, the motion vector derived from the opposite field is considered to be the dominant predictor. Intra-coded macroblocks are not considered in the calculation of dominant / non-dominant predictors. If all candidate predictor macroblocks are intra-coded, the dominant and non-dominant motion vector predictors are set to zero and the dominant predictor is considered to come from the opposite field.

The pseudo codes of FIGS. 52A-52F explain how the motion vector predictor for the 2 reference field case is calculated when three motion vector predictor candidates are given. The variables samefieldpred_x and samefieldpred_y represent the horizontal and vertical components of the motion vector predictor from the same field, and the variables oppositefieldpred_x and oppositefieldpred_y represent the horizontal and vertical components of the motion vector predictor from the opposite field. The variables samecount and oppositecount are initialized to zero. The variable dominantpredictor indicates which field contains the dominant predictor. The value predictor_flag (decoded from the motion vector difference) indicates whether a dominant or non-dominant predictor is used.

The pseudo code of FIGS. 52G and 52H illustrates the scaling operation referred to in the pseudo code of FIGS. 52A-52F, which scaling operation is used to derive the predictor of one field from the predictor of another field. The values of SCALEOPP, SCALESAME1, SCALESAME2, SCALEZONE1_X, SCALEZONE1_Y, ZONElOFFSET_X and ZONElOFFSET_Y are shown in the table of FIG. 52I for the case where the current field is the first field, and shown in the table of FIG. 52J for the case where the current field is the second field. have. The reference frame distance is encoded in the REFDIST 4624 field in the picture header. The reference frame distance is REFDIST + 1.

52K-52N are pseudo codes and tables for scaling operations alternative to those shown in FIGS. 52H-52J. Instead of the scaling codes and tables of FIGS. 52H-52J (but still use the pseudo codes of FIGS. 52A-52G), the scaling pseudo codes and tables of FIGS. 52K-52N are used. The reference frame distance is obtained from the element of the field layer header. The value of N depends on the motion vector range, as shown in the table of FIG. 52N.

8. hybrid  Motion vector prediction

Computed in the previous section, the motion predictor is tested against the A (top) and C (left) predictors to determine if the predictor is explicitly coded in the bitstream. If so, there is a bit indicating whether predictor A or predictor C should be used as the motion vector predictor. The pseudo code of FIG. 53 illustrates hybrid motion vector prediction decoding. In this pseudo code, the variables predictor_pre_x and predictor_pre_y are the horizontal and vertical motion vector predictors respectively calculated in the previous section. The variables predictor_post_x and predictor_post_y are the horizontal and vertical motion vector predictors after the test for hybrid motion vector prediction, respectively. The variables predictor_pre, predictor_post, predictorA, predictorB, and predictorC all represent polar fields that the value of predictor_flag points to. For example, if predictor_flag indicates that the opposite field predictor is used,

predictor_pre_x = oppositefieldpred_x

predictor_pre_x = oppositefieldpred_y

predictorA_x = oppositefieldpredA_x

predictorA_y = oppositefieldpredA_y

predictorB_x = oppositefieldpredB_x

predictorB_y = oppositefieldpredB_y

predictorC_x = oppositefieldpredC_x

predictorC_y = oppositefieldpredC_y

Similarly, if predictor_flag indicates that the same field predictor is used,

predictor_pre_x = samefieldpred_x

predictor_pre_x = samefieldpred_y

predictorA_x = samefieldpredA_x

predictorA_y = samefieldpredA_y

predictorB_x = samefieldpredB_x

predictorB_y = samefieldpredB_y

predictorC_x = samefieldpredC_x

predictorC_y = samefieldpredC_y

Where the values of oppositefieldpred and samefieldpred are calculated as described in the previous section.

9. Luma  Reconstruction of motion vector

For both 1MV and 4MV macroblocks, the luma motion vector is reconstructed by adding the difference to the predictor as follows, where the variables range_x and range_y depend on MVRANGE 473 and are specified in the table shown in FIG. 49B. For NUMREF = 0 (1 reference field interlaced P-field),

mv_x = (dmv_x + predictor_x) smod range_x

mv_y = (dmv_y + predictor_y) smod (range_y)

to be.

If NUMREF = 1 (2 reference field interlaced P-field),

mv_x = (dmv_x + predictor_x) smod range_x,

mv_y = (dmv_y + predictor_y) smod (range_y / 2)

to be.

If an interlaced P-field uses two reference pictures (NUMREF = 1), predictor_flag (derived when decoding the motion vector difference) is used to determine which field is used as a reference, as shown in FIG. Is combined with the value of the dominantpredictor (derived from the motion vector prediction).

In a 1MV macroblock, there is a single motion vector for the four blocks that make up the luma component of the macroblock. If the MBMODE (4661) syntax element indicates that there is no MV data in the macroblock layer, then dmvx = 0 and dmvy = 0 (mvx = predictor-x and mv_y = predictor_y).

In a 4MV macroblock, each inter-coded luma block in the macroblock has its own motion vector. Thus, there are four luma motion vectors in each 4MV macroblock. 4MVBP (4664) If the syntax element indicates that there is no motion vector information for a block, then dmv_x = 0 and dmv_y for that block (mv_x = predictor_x and mv_y = predictor_y).

10. Chroma  Derivation of the Motion Vector

The chroma motion vector is derived from the luma motion vector. The chroma motion vector is reconstructed in two steps. As a first step, a nominal chroma motion vector is obtained by properly synthesizing and scaling the luma motion vector. Scaling is performed such that a 1 / 2-pixel offset is preferred over a 1 / 4-pixel offset. In a second step, a 1-bit FASTUVMC syntax element is used to determine if additional rounding of chroma motion vectors is needed. If FASTUVMC = 0, no rounding is performed in the second step. When FASTUVMC = 1, the chroma motion vector at the quarter-pixel offset is rounded to the nearest half and full-pel position. Only bilinear filtering is used for all chroma interpolation. The variables cmv_x and cmv_y represent chroma motion vector components, respectively, and lmv_x and lmv_y represent luma motion vector components, respectively.

In a 1MV macroblock, the chroma motion vector is derived from the luma motion vector as follows.

cmv_x = (lmv_x + round [lmv_x & 3]) '' 1 and

cmv_y = (lmv_y + round [lmv_y & 3]) '' 1,

Where round [0] = 0, round [1] = 0, round [2] = 0, round [3] = 1

55A and 55B show the first step of how the chroma motion vector is derived from the motion information in four luma blocks in the 4MV macroblock. In this pseudo code, ix and iy are temporary variables. 55A is a pseudo code for chroma motion vector derivation for a 1 reference field interlaced P-field, and FIG. 55B is a pseudo code for chroma motion vector derivation for a 2 reference field interlaced P-field.

11. Century Reward

If MVMODE 4635 indicates that intensity compensation was used for an interlaced P-field, pixels in one or both of the reference fields are remapped before using them as predictors for the current P-field. If intensity compensation is used, the LUMSCALE1 4638 and LUMSHIFT1 4637 syntax elements are present in the bitstream for the first reference field, and the LUMSCALE2 4640 and LUMSHIFT2 4464 elements may also exist for the second reference field. have. The pseudo code of FIG. 56 shows how the LUMSCALE1 4638 and LUMSHIFT1 4639 values are used to create a lookup table used to remap reference field pixels for the first reference field. (This pseudo code is likewise applicable for LUMSCALE2 4640 and LUMSHIFT2 4641 for the second reference field.)

As follows, the Y component of the reference field is remapped using the LUTY [] table, and the C _b / C _r component is remapped using the LUTUV [] table.

및

And

는 참조 필드에서의 원래의 루마 픽셀 값이고,

는 참조 필드에서의 재매핑된 루마 픽셀 값이며,

는 참조 필드에서의 원래의 C_b 또는 C_r 픽셀값이고,

는 참조 필드에서의 재매핑된 C_b 또는 C_r 픽셀값이다.here,

Is the original luma pixel value in the reference field,

Is the remapped luma pixel value in the reference field,

Is the original C _b or C _r pixel value in the reference field,

Is the remapped C _b or C _r pixel value in the reference field.

12. Rest decoding

The decoder decodes this element if there is a CBPCY 4466 element for the macroblock, where the CBPCY 4466 element indicates the presence of coefficient data. In the block layer, the decoder decodes the coefficient data for the inter-coded block and the intra-coded block (except for the 4MV macroblock). To reconstruct an inter-coded block, the decoder selects (1) the transform type (8x8, 8x4, 4x8, or 4x4), (2) decodes the sub-block pattern (s), and (3) Decode, (4) perform inverse transform, (5) perform inverse quantization, (6) obtain predictions for blocks, and (7) add predictions and error blocks.

C. In the Second Combined Implementation sequence  And Semantics

In a second combined implementation, the compressed video sequence consists of data structured in hierarchical layers. From top to bottom, these layers are the picture layer, macroblock layer and block layer. The sequence layer precedes the sequence. 57A to 57C illustrate bitstream elements constituting various layers.

One. sequence  hierarchy Syntax  And Semantics

The sequence-level header includes a sequence-level parameter used to decode the compressed picture sequence. This header is made available to the decoder as decoder configuration information to be delivered externally or as part of a video data stream. 57A is a syntax diagram for a sequence layer bitstream showing elements constituting the sequence layer. Clip Profile The PROFILE 5701 element defines the encoding profile used to create the clip. If PROFILE is an "advanced" profile, the clip level LEVEL 5702 element specifies the encoding level for the clip. As another alternative (eg for other profiles), the clip level is delivered to the decoder by external means.

The INTERLACE 5703 element is a 1-bit field that exists when PROFILE is an advanced profile. INTERLACE 5703 specifies whether the video is coded in sequential or interlaced mode. If INTERLACE = 0, the video frame is coded in sequential mode. If INTERLACE = 1, the video frame is coded in interlace mode. If PROFILE 5701 is not an advanced profile, the video is coded in sequential mode.

The extended motion vector EXTENDED_MV 5704 element is a 1-bit field indicating whether the extended motion vector function is on or off. If EXTENDED_MV = l, the motion vector has an extended range. If EXTENDED_MV = 0, the motion vector does not have an extended range.

2. Picture  hierarchy Syntax  And Semantics

Data for a picture consists of data for the picture header and the macroblock layer following it. 57B is a syntax diagram for a picture layer bitstream showing elements constituting a picture layer for an interlaced P-field.

The picture type PTYPE 5722 element is a 1-bit field or variable-size field. In the absence of a B-picture, only I-pictures and P-pictures are present in the sequence, and the PTYPE is encoded with a single bit. If PTYPE = 0, the picture type is I. If PTYPE = 1, the picture type is P. If the number of B-pictures is greater than zero, PTYPE 5722 is a variable size field indicating the picture type of the frame. If PTYPE = 1, the picture type is P. If PTYPE = 01 (binary value), the picture type is I. If PTYPE = 00 (binary value), the picture type is B.

The number of reference pictures NUMREF 5731 is a 1-bit syntax element present in the interlaced P-field header. This element indicates whether the interlaced P-field has one (NUMREF = 0) or two (NUMREF = 1) reference pictures. Reference field picture indicator REFFIELD 5732 is a 1-bit syntax element present in an interlaced P-field when NUMREF = 0. This indicator indicates which of the two possible reference pictures the interlaced P-field uses.

Extended MV range flag MVRANGE 5735 is a variable-size syntax element present in a P-picture of a sequence coded using a particular profile (“main” profile), with the BROADCAST element set to one. In general, MVRANGE 5739 points to an extended range for the motion vector (ie, the longer possible horizontal and / or vertical displacement for the motion vector). MVRANGE 5733 is used to decode the motion vector difference.

The motion vector mode MVMODE 5735 element is a variable-size syntax element that signals one of one of the four motion vector coding modes or one intensity compensation mode. The motion vector coding mode includes three "1MV" modes with different sub-pixel interpolation rules for motion compensation. 1 MV means that each macroblock in a picture has at most one motion vector. In the "mixed-MV" mode, each macroblock in a picture may have one or four motion vectors or may be skipped. Depending on the value of PQUANT (quantization coefficient for the picture), any one of the tables shown in FIG. 47E is used for the MVMODE 5735 element.

The motion vector mode 2 MVMODE2 5736 element is a variable-length syntax element present in the interlaced P-field header when the MVMODE 5735 signals intensity compensation. The previous table (no code for intensity compensation) can be used for MVMODE2 5736.

The luminance scale LUMSCALE 5738 and luminance transition LUMSHIFT 5739 elements are each 6-bit values used in intensity compensation. LUMSCALE 5738 and LUMSHIFT 5739 are present in the interlaced P-field header when MVMODE 5735 signals intensity compensation.

The macroblock mode table MBMODETAB 5574 element is a 2-bit field for the interlaced P-field header. MBMODETAB 5574 indicates which of the four code tables (tables 0 to 3 specified as 2-bit values) is used to encode / decode the macroblock mode MBMODE 5576 syntax element in the macroblock layer.

The motion vector table MVTAB 5743 element is a 2-bit field for an interlaced P-field. MVTAB 5743 indicates which of the four code tables (tables 0 to 3 specified as 2-bit values) are used to encode / decode the motion vector data.

4MV Block Pattern Table The 4MVBPTAB (5744) element contains an interlaced P-field if MVMODE (5735)] or MVMODE (5735) is set to strength compensation, and MVMODE2 (5736)] indicates that the picture is a mixed-MV type. It is a 2-bit value that exists. 4MVBPTAB 5744 signals which of the four code tables (tables 0 to 3 specified as 2-bit values) are used to encode / decode the 4MV block pattern 4MVBP 5764 field in the 4MV macroblock.

The interlaced P-frame header (not shown) has many of the same elements as the interlaced P-field header shown in FIG. 57B. These include PTYPE 5722, MBMODETAB 5574, MVTAB 5843, and 4MVBPTAB 5744, but the exact syntax and semantics for interlaced P-frames may differ from interlaced P-fields. For example, 4MVBPTAB, again, is a 2-bit indicating which of the four code tables (tables 0 to 3 specified as 2-bit values) are used to encode / decode 4MV block pattern 4MVBP elements in 4MV macroblocks. Field. The interlaced P-frame header also includes other elements for switching between 1MV and 4MV modes and for intensity compensation signaling.

Since interlaced P-frames may include field-coded macroblocks with two motion vectors per macroblock, the interlaced P-frame header includes two motion vector block pattern table 2MVBPTAB elements. 2MVBPTAB is a 2-bit field present in an interlaced P-frame. This syntax element signals which of the four tables (tables 0 to 3 specified as 2-bit values) are used to encode / decode a 2MV Block Pattern (2MVBP) element in a 2MV field-coded macroblock. . 47K shows four tables available for 2MVBP.

Interlaced B-fields and interlaced B-frames have many of the same elements of interlaced P-fields and interlaced P-frames. Specifically, interlaced B-frames include both 2MVBPTAB and 4MVBPTAB 5721 syntax elements, but the semantics of these elements may differ from interlaced P-fields and P-frames.

3. Macroblock Layer Syntax  And Semantics

The data for a macroblock consists of a macroblock header followed by a block hierarchy. FIG. 57C is a syntax diagram for a macroblock layer bitstream showing elements constituting a macroblock layer for a macroblock of an interlaced P-field.

The macroblock mode MBMODE 5576 element is a variable-size element. This element is the number of motion vectors for the macroblock (1MV, 4MV or intra), whether there is a block pattern CBPCY 5576 element coded for the macroblock, and (in some cases) the motion vector for the macroblock. Information such as whether there is difference data is indicated jointly.

The motion vector data MVDATA 5763 element is a variable-length element that encodes motion vector information (eg, horizontal and vertical differences) for the motion vector of the macroblock. For an interlaced P-field with two reference fields, the MVDATA 5763 also encodes information for selecting between the dominant motion vector predictor and the non-dominant motion vector predictor for the motion vector.

The four motion vector block pattern 4MVBP 5764 element is present when MBMODE 5861 indicates that the macroblock has four motion vectors. The 4MVBP 5764 element indicates which of the four luminance blocks contains a non-zero motion vector difference. A code table is used to decode the 4MVBP 5764 element to a value between 0 and 14. This decoded value represents a bit field indicating whether a motion vector for the corresponding luminance block exists as shown in FIG. 34 when represented as a binary value.

The two motion vector block pattern 2MVBP element (not shown) is a variable-size syntax element present in a macroblock in an interlaced P-frame. In an interlaced P-frame macroblock, 2MVBP is present when MBMODE 5861 indicates that the macroblock has two field motion vectors. In this case, 2MVBP indicates which of the two fields (top and bottom) contains a non-zero motion vector difference.

The block-level motion vector data BLKMVDATA 5765 element is a variable-size element that exists in some situations. This element contains motion information for a block of macroblocks.

Hybrid Motion Vector Prediction HYPRIDPRED (5766) element is a 1-bit syntax element per motion vector that may exist in a macroblock in an interlaced P-field. If hybrid motion vector prediction is used, HYBRIDPRED 5766 indicates which of the two motion vector predictors to use.

4. Block Layer Syntax  And Semantics

The block layer for interlaced pictures follows the syntax and semantics of the block layer for sequential pictures. In general, information about the DC and AC coefficients of blocks and sub-blocks is signaled at the block layer.

D. Decoding in the Second Combined Implementation

The following sections focus on the decoding process for interlaced P-fields.

One. Interlaced  Reference to P-field Decoding

An interlaced P-field may refer to one or two previously decoded fields in motion compensation. The NUMREF 5731 field in the picture layer indicates whether the current field can reference one or two previous reference field pictures. If NUMREF = 0, the current interlaced P-field may refer to only one field. In this case, a REFFIELD 5732 element is followed in the picture layer bitstream, indicating which field is used as a reference. If REFFIELD = 0, the nearest I or P-field in time (in display order) is used as reference. If REFFIELD = 1, the second most recent I or P-field picture in time is used as a reference. If NUMREF = 1, the current interlaced P-field picture uses the two temporally closest I or P field pictures (in display order) as reference. Examples of reference fields for NUMREF = 0 and NUMREF = 1 shown in FIGS. 24A-F apply to the second combined implementation as described above.

2. Picture  Types and Picture  Hierarchical Table Selection

Interlaced P-fields can be of two types: 1MV or mixed-MV. In the 1MV P-field, for a 1MV macroblock, a single motion vector is used to indicate the predicted block displacement for all six blocks in the macroblock. In a mixed-MV P-field, macroblocks may be encoded as 1MV or 4MV macroblocks. In the case of a 4MV macroblock, each of the four luminance blocks may have an associated motion vector. The 1MV mode or mixed-MV mode is signaled by the MVMODE 5735 and MVMODE2 5736 picture layer fields.

In the case of an interlaced P-field, the picture layer includes a syntax element that controls the motion compensation mode and intensity compensation for the field. MVMODE 5735 signals either 1) one of four motion vector modes for the field, or 2) whether intensity compensation is used in the field. If intensity compensation is signaled, the MVMODE2 5736, LUMSCALE 5738, and LUMSHIFT 5739 fields are followed in the picture layer. One of the two tables in FIG. 47E is used to decode the MVMODE 5735 and MVMODE2 5736 fields depending on whether PQUANT is greater than 12. FIG.

If the motion vector mode is mixed-MV mode, MBMODETAB 5574 signals which of the four mixed-MV MBMODE tables is used to signal the mode for each macroblock in the field. If the motion vector mode is not mixed-MV (in which case all inter-coded macroblocks use one motion vector), MBMODETAB 5574 can display four 1MVs to signal the mode of each macroblock in the field. Signals which of the MBMODE tables is used.

MVTAB 5743 points to a code table used to decode motion vector differences for macroblocks in interlaced P-fields. 4MVBPTAB 5744 points to a code table used to decode 4MVBP 5764 for 4MV macroblocks in an interlaced P-field.

3. Macro Block mode  And movement vector block pattern

Macroblocks in interlaced P-fields can be one of three possible types, 1MV, 4MV and intra. The macroblock type is signaled by MBMODE 5576 in the macroblock layer.

The 1MV macroblock may be in the 1MV and Mixed-MV P-field. A 1MV macroblock is a macroblock when a single motion vector represents a displacement between the current picture and the reference picture for all six blocks in the macroblock. The difference between the current block and the reference block is encoded in the block layer. For 1MV macroblocks, MBMODE 5761 has three types: (1) macroblock type is 1MV, (2) whether CBPCY (5762) exists, and (3) whether MVDATA (5763) exists. Point to.

If MBMODE 5601 indicates that CBPCY 5702 is present, CBPCY 5702 is present in the macroblock layer, which indicates which of the six blocks is coded in the block layer. If MBMODE 5601 indicates that CBPCY 5702 does not exist, CBPCY 5702 is assumed to be zero, and no block data exists for any of the six blocks in the macroblock.

If MBMODE 5711 indicates that MVDATA 5763 is present, MVDATA 5763 encodes a motion vector difference present in the macroblock layer and synthesized with the motion vector predictor to reconstruct the motion vector. If MBMODE 5711 indicates that MVDATA 5763 does not exist, then the motion vector difference is assumed to be zero and thus the motion vector is equal to the motion vector predictor.

4MV macroblocks are only in the mixed-MV P-field. A 4MV macroblock is a macroblock when each of the four luminance blocks in the macroblock can have an associated motion vector that indicates the displacement between the current picture and the reference picture for that block. The displacement for the chroma block is derived from four luminance motion vectors. The difference between the current block and the reference block is encoded in the block layer. For 4MV macroblocks, MBMODE 5761 has three types: (1) macroblock type is 4MV, (2) whether CBPCY (5762) exists, and (3) whether 4MVBP (5764) exists. Point to.

If MBMODE 5701 indicates that 4MVBP 5764 is present, 4MVBP 5764 is present in the macroblock layer and indicates which of the four luminance blocks contains non-zero motion vector differences. 4MVBP 5764 is decoded to a value of 1 to 14, and as shown in FIG. 27, this value represents a bit field indicating whether motion vector data for the corresponding luminance block, when represented as a binary value, exists. For each of the four bit positions in 4MVBP 5764, a value of zero indicates that there is no motion vector difference (BLKMVDATA 5765) for that block, and the motion vector difference is assumed to be zero. A value of 1 indicates that there is a motion vector difference (BLKMVDATA 5765) for that block. When MBMODE 5711 indicates that 4MVBP 5764 does not exist, it is assumed that motion vector difference data BLKMVDATA 5655 exists for all four luminance blocks.

A field-coded macroblock in an interlaced P-frame may include two motion vectors. For a two field MV macroblock, the 2MVBP element indicates which of the two fields has a non-zero differential motion vector.

Intra macroblocks may occur in the 1 MV or mixed-MV field. An intra macroblock is a macroblock when all six blocks are coded without reference to any previous picture data. The difference between the current block pixel and a constant value of 128 is encoded in the block layer. For intra macroblocks, MBMODE 5701 indicates two things: (1) that the macroblock type is intra, and (2) whether CBPCY 5576 exists. For intra macroblocks, when CBPCY 5702 is present it indicates which of the six blocks has AC coefficient data coded in the block layer.

4. Decoding Motion Vector Difference

MVDATA (5763) and BLKMVDATA (5765) fields encode the motion information for the macroblock or blocks within the macroblock. A 1MV macroblock may have a single MVDATA 5763 field, and a 4MV macroblock may have 0 to 4 BLKMVDATA 5765. Computing motion vector differences is performed differently for the 1-reference (NUMREF = 0) and the 2-reference (NUMREF = 1) cases.

In a field picture with only one reference field, each MVDATA 5763 or BLKMVDATA 5765 field in the macroblock layer is divided into two, (1) horizontal motion vector difference component, and (2) vertical motion vector difference. Encode the components jointly. The MVDATA (5763) or BLKMVDATA (5765) field is a Huffman VLC followed by the FLC. The value of VLC determines the size of the FLC. The MVTAB 5743 field in the picture layer specifies the table used to decode the VLC.

58A is a pseudo code illustrating motion vector differential decoding of a motion vector of a block or a macroblock in a field picture having one reference field. In this pseudo code, the values dmv_x and dmv_y are calculated. The value dmv_x is a differential horizontal motion vector component and the value dmv_y is a differential vertical motion vector component. The variables k_x and k_y are fixed length values for the long motion vector and depend on the motion vector range defined by MVRANGE 5735 as shown in the table of FIG. 58B. The value halfpel_flag is a binary value indicating whether ½-pixel or ¼-pixel precision is used for motion compensation for the picture. The value of halfpel_flag is determined by the motion vector mode. If the mode is 1 MV or mixed-MV, halfpel_flag = 0 and ¼-pixel precision is used for motion compensation. If the mode is 1MV ½-pixel or 1MV ½-pixel bilinear, halfpel_flag = 1 and ½-pixel precision is used. offset_table is an array defined as follows.

offset_table [9] = {0, 1, 2, 4, 8, 16, 32, 64, 128}.

In a field picture having two reference fields, each MVDATA 5763 or BLKMVDATA 5765 field in the macroblock layer has three fields: (1) horizontal motion vector difference component, (2) vertical motion vector difference component, And (3) jointly encode whether a dominant or non-dominant motion vector predictor is used. The MVDATA (5763) or BLKMVDATA (5765) field is Huffman VLC followed by FLC, and the value of VLC determines the size of the FLC. The MVTAB 5743 field specifies the table used to decode the VLC.

59 shows pseudo codes describing motion vector difference and dominant / non-dominant predictor decoding for a motion vector of a block or macroblock in a field picture having two reference fields. In this pseudo code, the value predictor_flag is a binary flag indicating whether the dominant or non-dominant motion vector predictor is used (0 = dominant predictor used, 1 = non dominant predictor used). Various other variables (including dmv_x, dmv_y, k_x, k_y, halfpel_flag, and offset_table []) are as described for the one reference field case. The table size_table is an array defined as follows.

size_table [14] = {0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6}.

5. Motion vector Predictor

The motion vector is calculated by adding the motion vector difference calculated in the previous section to the motion vector predictor. The predictor is calculated from up to three neighboring motion vectors.

In a 1MV interlaced P-field, up to three motion vectors are used to calculate the predictors for the current macroblock. The locations of neighbor predictors A, B, and C are shown in FIGS. 5A and 5B. As described for the sequential P-frames, the neighbor predictors are taken from the left, top and right-top macroblocks except when the current macroblock is the last macroblock in the row. In this case, predictor B is taken from the left-top macroblock instead of the top right. In the special case where a frame is one macroblock wide, the predictor is always predictor A (upper predictor).

In a mixed-MV interlaced P-field, up to three motion vectors are used to calculate the predictor for the current block or macroblock. 6A-10 illustrate three candidate motion vectors for 1MV and 4MV macroblocks in a mixed-MV P-field, as described for sequential P-frames. In the special case where a frame is one macroblock wide, the predictor is always predictor A (upper predictor).

If the NUMREF 5731 field in the picture header is zero, the current interlaced P-field may only refer to one previously coded picture. If NUMREF = 1, the current interlaced P-field may refer to the two most recent reference field pictures. In the former case, a single predictor is calculated for each motion vector. In the latter case, two motion vector predictors are calculated. 60A and 60B show how the motion vector predictor is calculated for the 1 reference field case. The variables fieldpred_x and fieldpred_y represent the horizontal and vertical components of the motion vector predictor.

2 Reference Fields In an interlaced P-field (NUMREF = 1), the current field may refer to the two most recent reference fields. In this case, two motion vector predictors are calculated for each inter-coded macroblock. One predictor is from a reference field of the same polarity and the other is from a reference field of the opposite polarity.

61A to 61F describe how the motion vector predictor is calculated for the 2 reference field case when three motion vector predictor candidates are given. The variables samefieldpred_x and samefieldpred_y represent the horizontal and vertical components of the motion vector predictor from the same field, and the variables oppositefieldpred_x and oppositefieldpred_y represent the horizontal and vertical components of the motion vector predictor from the opposite field. The variable dominantpredictor indicates which field contains the dominant predictor. The value predictor_flag (decoded from the motion vector difference) indicates whether a dominant or non-dominant predictor is used.

6. hybrid  Motion vector prediction

If the interlaced P-field is 1MV or mixed-MV, the motion vector predictor computed in the previous section is passed to the A (top) and C (left) predictors to determine if the predictor is explicitly coded in the bitstream. Is tested against. If so, there is a bit indicating whether to use predictor A or predictor C as a motion vector predictor. The pseudo codes of FIGS. 14A and 14B illustrate hybrid motion vector prediction decoding using a variable as follows. The variables predictor_pre_x and predictor_pre_y and the candidate predictors A, B, and C are as calculated in the previous section (ie they are opposite field predictors or the same field predictors, as indicated by the predictor flags). The variables predictor_post_x and predictor_post_y are the horizontal and vertical motion vector predictors after checking for hybrid motion vector prediction, respectively.

7. Reconstruction of Motion Vectors

For both 1MV and 4MV macroblocks, the luminance motion vector is reconstructed by adding the difference to the predictor as follows.

mv_x = (dmv_x + predictor_x) smod range_x, and

mv_y = (dmv_y + predictor_y) smod range_y,

Here, the variables range_x and range_y depend on MVRANGE 5731 and are specified in the table shown in FIG. 58B, and the operation "smod" is a signed modulus with a sign defined as follows.

A smod b = ((A + b)% 2 b)-b,

This ensures that the reconstructed vector is valid. (A smod b) is within -b to b-1.

In a 1MV macroblock, it is a single motion vector for the four blocks that make up the luminance component of the macroblock. If dmv_x indicates that the macroblock is intra-coded, no motion vector is associated with the macroblock. If the macroblock is skipped, then dmv_x = 0 and dmv_y = 0, so mv_x = predictor_x and mv_y = predictor_y.

In a 4MV macroblock, each intra-coded luminance block in the macroblock has its own motion vector. Thus, there are zero to four luminance motion vectors for each 4MV macroblock. Uncoded blocks within a 4MV macroblock are in two ways: (1) the macroblock is skipped and the macroblock is 4MV (all blocks within the macroblock are skipped in this case), and (2) for the macroblock. One of the cases where CBPCY 5702 indicates that a block is uncoded. If the block is not coded, then dmv_x = 0 and dmv_y = 0, so mv_x = predictor_x and mv_y = predictor_y.

8. Chroma  Derivation of the Motion Vector

The chroma motion vector is derived from the luminance motion vector. In addition, in the case of the 4MV macroblock, the determination of whether to code the chroma block as inter or intra is made based on the state of the luminance block. The chroma motion vector is reconstructed in two steps. In a first step, a nominal chroma motion vector is obtained by properly synthesizing and scaling the luminance motion vector. Scaling is performed so that the ½-pixel offset is preferred over the ¼-pixel offset. In the second step, a sequence level 1-bit field FASTUVMC field is used to determine if further rounding of the chroma motion vector is needed. If FASTUVMC = 0, rounding is not performed in the second step. If FASTUVMC = 1, the chroma motion vector at the quarter-pixel offset is rounded up to the nearest full-pixel position. In addition, when FASTUVMC = 1, only bilinear filtering is used for all chroma interpolation.

In a 1MV macroblock, the chroma motion vector is derived from the luminance motion vector as follows.

// s_RndTbl [0] = 0, s_RndTbl [l] = 0, s_RndTbl [2] = 0, s_RndTbl [3] = 1

cmv_x = (lmv_x + s_RndTbl [lmv_x & 3]) >> 1

cmv_y = (lmv_y + s_RndTbl [lmv_y & 3]) >> 1

The pseudo code of FIG. 16B shows the first step of how the chroma motion vector is derived from the motion information for the four luminance blocks in the 4MV macroblock using variables as follows. The dominant polarity among the four luminance motion vectors for the 4MV macroblock is determined, and the chroma motion vector is determined from the luminance motion vector having the dominant polarity (but not determined from the luminance motion vector of the other polarity).

9. Century Reward

If intensity compensation is used for the reference field, the pixels in the reference field are remapped before using them as predictors. When intensity compensation is used, LUMSCALE 5738 and LUMSHIFT 5739 are in the picture bitstream. 18 or 56 shows how LUMSCALE 5738 and LUMSHIFT 5739 are used to remap reference field pixels. As follows, the Y component of the reference is remapped using the LUTY [] table, and the U and V components are remapped using the LUTUV [] table.

및

And

는 참조 필드에서의 원래의 루미넌스 픽셀 값이고,

는 참조 필드에서의 재매핑된 루미넌스 픽셀값이며,

는 참조 필드에서의 원래의 U 또는 V 픽셀값이고,

는 참조 필드에서의 재매핑된 U 또는 V 픽셀값이다.here,

Is the original luminance pixel value in the reference field,

Is the remapped luminance pixel value in the reference field,

Is the original U or V pixel value in the reference field,

Is the remapped U or V pixel value in the reference field.

10. Decode rest

The decoder decodes the CBPCY 5762 element, if present, for the macroblock, where the CBPCY 5762 element indicates the presence of coefficient data. At the block layer, the decoder decodes coefficient data for inter-coded blocks and intra-coded blocks. To reconstruct an inter-coded block, the decoder selects (1) the transform type (8x8, 8x4, 4x8 or 4x4), (2) decodes the sub-block pattern, (3) decodes the coefficients, ( 4) perform inverse transform, (5) perform inverse quantization, (6) obtain predictions for blocks, and (7) add predictions and error blocks.

While the principles of the invention have been described and illustrated with reference to various embodiments, it will be appreciated that these various embodiments may be modified in construction and detail without departing from these principles. Unless stated otherwise, it is to be understood that the programs, processes, or methods described herein are not related to or limited to any particular type of computing environment. Various types of general purpose or specialty computing environments may be used with or otherwise perform operations with the disclosures disclosed herein. The components of the embodiments shown in software may be implemented in hardware and vice versa.

In view of the many possible embodiments to which the principles of the invention may be applied, all such embodiments which fall within the scope and spirit of the appended claims and their equivalents are claimed as the invention.

1 is a diagram illustrating motion estimation in a video encoder according to the prior art.

2 is a block-based compression of 8x8 block of prediction residual in a video encoder according to the prior art.

3 illustrates block-based decompression for 8x8 blocks of prediction residuals in a video decoder according to the prior art.

4 illustrates an interlaced frame according to the prior art.

5A and 5B are diagrams illustrating positions of macroblocks for candidate motion vector predictors for 1MV macroblocks in sequential P-frames according to the prior art.

6A and 6B are diagrams showing positions of blocks for candidate motion vector predictors for 1MV macroblocks in a mixed 1MV / 4MV sequential P-frame according to the prior art.

7A, 7B, 8A, 8B, 9 and 10 show candidate motion vector predictors for blocks at various positions in a 4MV macroblock in a mixed 1MV / 4MV sequential P-frame according to the prior art. It is a figure which shows the position of the block with respect to.

11 illustrates a candidate motion vector predictor for the current frame-coded macroblock in an interlaced P-frame according to the prior art.

12A and 12B are diagrams illustrating candidate motion vector predictors for a current field-coded macroblock in an interlaced P-frame according to the prior art.

13A to 13C are pseudo codes for calculating a motion vector predictor according to the prior art.

14A and 14B are pseudo codes illustrating hybrid motion vector prediction for sequential P-frames according to the prior art.

15A to 15C are pseudo codes and tables illustrating decoding of motion vector difference information according to the prior art.

16A-16C and 13C are pseudo codes illustrating derivation of chroma motion vectors for sequential P-frames according to the prior art.

17 is a pseudo code illustrating derivation of a chroma motion vector for an interlaced P-frame according to the prior art.

18 is a pseudo code illustrating intensity compression for sequential P-frames according to the prior art.

19 is a block diagram of a suitable computing environment in which several described embodiments may be implemented.

20 is a block diagram of a generalized video encoder system in which several described embodiments may be implemented.

21 is a block diagram of a generalized video decoder system in which several described embodiments may be implemented.

FIG. 22 is a diagram illustrating a macroblock format used in some described embodiments.

FIG. 23A is a diagram illustrating a portion of an interlaced video frame showing alternating lines of the upper field and the lower field. FIG.

FIG. 23B illustrates an interlaced video frame configured for encoding / decoding as a frame. FIG.

23C illustrates an interlaced video frame configured for encoding / decoding as a field.

24A to 24F are charts showing examples of reference fields for interlaced P-fields.

25A and 25B are flowcharts illustrating a technique for encoding and decoding reference fields and selection information, respectively.

26 and 27 are tables showing MBMODE values.

28A and 28B are flowcharts illustrating a technique for encoding and decoding macroblock mode information for macroblocks of interlaced P-fields, respectively.

29 is pseudo code for determining a dominant and non-dominant reference field.

30 is a pseudo code signaling whether a dominant or non-dominant reference field is used for a motion vector.

31A and 31B are flowcharts illustrating a technique for determining dominant and non-dominant polarity for motion vector prediction in encoding and decoding motion vectors for two reference field interlaced P-fields, respectively.

32 is a pseudo code for hybrid motion vector prediction during decoding.

33A and 33B are flowcharts illustrating a technique for hybrid motion vector prediction during encoding and decoding, respectively.

34 shows an association between luma blocks and 4MVBP elements.

35A and 35B are flowcharts illustrating a technique for encoding and decoding using a motion vector block pattern, respectively.

36 is a pseudo code encoding motion vector difference information and dominant / non-dominant predictor selection for a two-reference field interlaced P-field.

37A and 37B are flowcharts illustrating a technique for encoding and decoding motion vector difference information and dominant / non-dominant predictor selection for two reference field interlaced P-fields, respectively.

FIG. 38 illustrates chroma sub-sampling patterns for 4: 2: 0 macroblocks.

39 shows a relationship between a current field and a reference field for the vertical motion vector component.

40 is a pseudo code for selecting a luminance motion vector that contributes to a chroma motion vector for a motion-compensated macroblock of an interlaced P-field.

FIG. 41 is a flowchart illustrating a technique for deriving a chroma motion vector from a luma motion vector for a macroblock of an interlaced P-field.

42 and 43 are diagrams illustrating an encoder framework and a decoder framework in which intensity compensation is performed on an interlaced P-field, respectively.

FIG. 44 is a table illustrating syntax elements signaling a strength compensation reference field pattern for an interlaced P-field.

45A and 45B are flowcharts illustrating techniques for performing fading estimation in encoding and decoding in encoding for interlaced P-fields, respectively.

46A-46E are syntax diagrams for the layers of the bitstream according to the first combined implementation.

47A-47K are tables for code in the first combined implementation.

48 illustrates a relationship between a current field and a reference field for the vertical motion vector component in the first combined implementation.

49A and 49B are pseudo codes and tables for motion vector differential decoding for one reference field interlaced P-field, respectively, in a first combined implementation.

50 is pseudo code for decoding motion vector difference information and dominant / non-dominant predictor selection for a two-reference field interlaced P-field in a first combined implementation.

51A and 51B are pseudo codes for motion vector prediction for a 1 reference field interlaced P-field in the first combined implementation.

52A-52J are pseudo codes and tables for motion vector prediction for two reference field interlaced P-fields in a first combined implementation.

52K-52N are pseudo codes and tables for scaling operations alternative to those shown in FIGS. 52H-52J.

53 is a pseudo code for hybrid motion vector prediction for an interlaced P-field in the first combined implementation.

54 is a pseudo code for motion vector reconstruction for two reference field interlaced P-field in the first combined implementation.

55A and 55B are pseudo codes for chroma motion vector derivation for an interlaced P-field in a first combined implementation.

56 is pseudo code for intensity compensation for interlaced P-fields in a first combined implementation.

57A-57C are syntax diagrams for the layers of the bitstream according to the second combined implementation.

58A and 58B are pseudo codes and tables for motion vector differential decoding for one reference field interlaced P-field, respectively, in a second combined implementation.

59 is pseudo code for decoding motion vector difference information and dominant / non-dominant predictor selection for a two-reference field interlaced P-field in a second combined implementation.

60A and 60B are pseudo codes for motion vector prediction for a 1 reference field interlaced P-field in a second combined implementation.

61A-61F are pseudo codes for motion vector prediction for two reference field interlaced P-fields in a second combined implementation.

<Description of the symbols for the main parts of the drawings>

1900: computing environment

1910 processing unit

1920: memory

1940: store

1950: input device (s)

1960: output device (s)

1970: communication connection (s)

Claims (15)

A method in a computing device implementing a video decoder, Using the computing device implementing the video decoder, an interlaced forward-predicted field has one reference field or two possible reference fields for motion compensation. processing a first field level signal indicating whether there is a reference field; If the first field level signal indicates that the interlaced forward-prediction field has one reference field, using the computing device implementing the video decoder, the one of the two possible reference fields Processing a second field level signal that identifies an identifier, wherein the identified reference field is used for all the motion compensation for the interlaced forward-prediction field; And Performing the motion compensation on the interlaced forward-prediction field using the computing device implementing the video decoder. How to include. The method of claim 1, And the first field level signal is a single bit. The method of claim 1, And the second field level signal is a single bit. The method of claim 1, If the first field level signal indicates that the interlaced forward-prediction field has two possible reference fields, using the computing device implementing the video decoder, blocks of the interlaced forward-prediction field and For each of a plurality of motion vectors for at least one of the macroblocks, processing a third signal to select from the two possible reference fields, wherein the selected reference field is configured to use the motion vector. Used for motion compensation. The method of claim 4, wherein The third signal is at a macroblock level. The method of claim 1, The two possible reference fields are (1) the oldest interlaced intra or forward-prediction field that is most recent in time, and (2) the second most recent previous interlaced intra or forward-prediction in time. Method constrained by field. The method of claim 1, If the first signal indicates that the interlaced forward-prediction field has one reference field, the video decoder uses field-level reference field selection, and the first signal is interlaced. If the type forward-prediction field indicates that there are two possible reference fields, the video decoder uses per motion vector reference field selection. A method in a computing device implementing a video decoder, Using the computing device implementing the video decoder, processing a first signal indicating whether the interlaced forward-prediction field has one reference field or two possible reference fields for motion compensation; Using the computing device implementing the video decoder, switching between per field reference field selection and reference field selection by motion vector based at least in part on the first signal. Step-the conversion, Using the per-field reference field selection when the first signal indicates that the interlaced forward-prediction field has one reference field; And Using a reference field selection by motion vector when the first signal indicates that the interlaced forward-prediction field has two possible reference fields; Performing motion compensation on the interlaced forward-prediction field using the computing device implementing the video decoder; And Using the computing device implementing the video decoder, updating the reference field buffer for subsequent motion compensation without processing additional signals to manage the reference field buffer How to include. The method of claim 8, When the first signal indicates that the interlaced forward-prediction field has one reference field, using the computing device implementing the video decoder, identifying the one reference field from the two possible reference fields Processing the second signal to perform, Wherein the first and second signals are each at a picture level for the interlaced forward-prediction field. The method of claim 8, The two possible reference fields are (1) the oldest interlaced intra or forward-prediction field that is most recent in time, and (2) the second most recent previous interlaced intra or forward-prediction field in time. Constrained. The method of claim 8, Wherein the one reference field is constrained to (1) the previous interlaced intra or forward-prediction field that is most recent in time, or (2) the second most recent previous interlaced intra or forward-prediction field in time. . A processor; Memory; At least one of an input device, an output device, and a communication connection; And A decoder comprising a repository, The reservoir is Whether the interlaced forward-prediction field uses (1) field level reference field selection for motion compensation with one reference field, or (2) reference field per motion vector for the motion compensation with two possible reference fields. Means for processing a first signal indicating whether to use selection; When the first signal indicates that the interlaced forward-prediction field has one reference field and uses field level reference field selection, a second signal identifying the one reference field among the two possible reference fields Means for processing; Means for processing a third signal for each of a plurality of motion vectors when the first signal indicates that the interlaced forward-prediction field has two possible reference fields and uses reference field selection by motion vector-the third Each signal is for selection between the two possible reference fields; And Means for performing the motion compensation on the interlaced forward-prediction field Store computer executable instructions that cause the decoder to decode the video, Decoder. The method of claim 12, Wherein the first signal and the second signal are each at a picture level for the interlaced forward-prediction field and the third signal is at a macroblock level. The method of claim 12, The two possible reference fields are constrained to be (1) the oldest interlaced intra or forward-prediction field that is most recent in time, and (2) the oldest interlaced intra or forward-prediction field that is second most recent in time. That is, a decoder. The method of claim 12, The storage further comprises computer executable instructions for causing the decoder to decode the video using means for updating the reference field buffer for subsequent motion compensation without processing additional signals to manage the reference field buffer. To store, decoder.

Images