Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/513—Processing of motion vectors
  • H04N19/517—Processing of motion vectors by encoding
  • H04N19/52—Processing of motion vectors by encoding by predictive encoding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals

Definitions

• Embodiments of the present invention relate to the field of video coding for compression.
• embodiments of the invention are related to the prediction and differential coding of motion vectors.
• Block-based motion compensation is an integral operation in a variety of video codecs that exploits temporal correlation to achieve compression of video data.
• the efficiency of compression achieved by motion compensation is dependent on the efficiency with which motion vectors are signaled.
• a predictor is derived for each motion vector from a causal neighborhood and only the difference is coded as part of the bitstream.
• Existing techniques do not exploit all the redundancy in deriving the predictor and hence there is scope for improvement.
• a method and apparatus for motion vector prediction and coding.
• the method comprises: deriving N motion vector predictors for a first block that has N motion vectors corresponding to N lists of reference frames and a current frame, including constructing one of the N motion vector predictors when a second block that neighbors the first block and is used for prediction has at least one invalid motion vector, where N is an integer greater than 1 ; generating N differential motion vectors based on the N motion vectors and N motion vector predictors; and encoding the N differential motion vectors.
• Figure 1 illustrates causal neighbor blocks used to construct predictors for the motion vectors of another block.
• Figure 2 is a flow diagram of one embodiment of a process for constructing a predictor set.
• Figure 3 illustrates cases for which a predictor is constructed from a neighbor block.
• Figure 4 is a flow diagram of one embodiment of a process for refining a candidate predictor.
• Figure 5 illustrates an example of generating a MVP.
• Figure 6 is a flow diagram of one embodiment of a process for deriving motion vector predictors.
• Figure 7 is a flow diagram of one embodiment of a process for computing differential motion vectors at an encoder.
• Figure 8 is a flow diagram of one embodiment of a process for
• Figure 9 illustrates an example of scaling one MVD to predict a second
• Figures 10A and B are flow diagrams of embodiments of processes for coding motion vectors at an encoder and a decoder, respectively.
• Figure 11 is a block diagram for a motion vector encoder.
• Figure 12 is a block diagram for a motion vector decoder.
• Figure 13 is a block diagram of a computer system. DETAILED DESCRIPTION OF THE PRESENT INVENTION
• Embodiments of the current invention include methods to enhance the predictors thereby reducing the number of bits spent to signal the motion vectors.
• embodiments of the current invention include techniques to enhance the prediction of motion vectors for blocks coded in bi-predictive or multi- predictive modes by exploiting the correlation across two or more lists of reference frames.
• the present invention also relates to apparatus for performing the operations herein.
• This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer.
• a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.
• a machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer).
• a machine-readable medium includes read only memory ("ROM");
• RAM random access memory
• magnetic disk storage media magnetic disk storage media
• optical storage media flash memory devices
• a motion vector ( MV ) is signaled by coding a differential motion vector ( MVD ) where the differential motion vector is the difference between the motion vector and a motion vector predictor i.e.
• a bi-predictive block has two motion vectors, MV 0 and MV l , corresponding to two lists of reference frames, list 0 and list 1, respectively, and they are both signaled using differential motion vectors.
• MVD 0 and MVD l represent the differential motion vectors
• M represent the motion vector predictors for list 0 and list 1 respectively.
• multi-predictive block is a generalization of the concept of a bi-predictive block.
• a multi-predictive block has N motion vectors corresponding to N lists of reference frames where N is an integer greater than 1.
• N is an integer greater than 1.
• the present invention is explained in the context of a bi-predictive block for the sake of clarity. However, it is to be noted that the present invention is also applicable to the general case of a multi-predictive block.
• the motion vector predictors are formed using a set of candidate predictors called the predictor set.
• Each candidate predictor represents the motion vector(s) of a causal neighbor block (a block in the neighborhood whose motion vector(s) is/are already signaled) of the target block (block whose motion vector(s) is/are being signaled).
• Figure 1 illustrates causal neighbor blocks L and T that are used to construct predictors for the motion vector block C.
• the construction of the predictor set can be done in such a way that the candidate predictor added corresponding to a causal neighbor block need not exactly represent the motion vector(s) of the neighbor block.
• the candidate predictor's motion vector for a list is set equal to neighbor block's motion vector for the same list if the target block and neighbor block refer to same reference frames for that list; else, the candidate predictor's motion vector for that list is set to invalid status.
• Figure 2 is a flow diagram of one embodiment of a process for constructing a predictor set. The process is performed by processing logic that may comprise hardware, software, or a combination of both.
• processing logic begins by processing logic initializing a predictor set to an empty set (processing block 201). Next, processing logic fetches reference frame indices for list 0 and list 1 motion vectors of a target block (processing block 202). Thereafter, processing logic determines whether there are more causal neighbor blocks (processing block 203). If not, the process ends. If there are more causal neighbor blocks, the process transitions to processing block 204 where the processing logic creates a candidate predictor with list 0 and list 1 motion vectors set as invalid.
• Processing logic determines whether the current neighbor block has a list 0 motion vector (processing block 205). If not, the process transitions to processing block 209. If it does, the process transitions to processing block 206 where processing logic fetches the reference frame index for the list 0 motion vector of the current neighbor block and then determines whether the list 0 reference frame index is the same for the target block and the neighbor block (processing block 207). If so, processing logic sets the list 0 motion vector of the candidate predictor equal to that of the current neighbor block (processing block 208) and the process transitions to processing block 209. If the list 0 reference frame index is not the same for the target block and the neighbor block, the process transitions directly to processing block 209.
• processing logic determines whether the current neighbor block has a list 1 motion vector. If not, the process transitions to processing block 213. If the current neighbor block does have a list 1 motion vector, processing logic fetches the reference frame index for the list 1 motion vector of the current neighbor block (processing block 210). Next, processing logic tests whether the list 1 reference frame index is the same for the target and the neighbor
• processing logic sets the list 1 motion vector of the candidate predictor equal to that of the current neighbor block (processing block 212) and the process transitions to processing block 213. If not, the process transitions directly to processing block 213. [0021] At processing block 213, processing logic adds the candidate predictor into the predictor set and then the process transitions to processing block 203.
• the predictor set is refined by constructing a predictor from a neighbor block when the reference frame index matches for one of the two lists of reference frames. This occurs when a reference index matches only one list or when the neighbor block uses only one list.
• Figure 3 illustrates cases for which a predictor is constructed from a neighbor block.
• the predictor set is refined by refining one (or more) candidate predictors in the predictor set as described below: if the candidate predictor's motion vector corresponding to one list (list a ), MVP a , is valid and the candidate predictor's motion vector corresponding to the other list (list b ), MVP b , is invalid, a valid value for MVP b is calculated using MVP a .
• MVP b f pred (MVP a ,T a ,T b )
• T a and T b represent the signed temporal distances from the current frame to the reference frames referred by MV a and MV b respectively.
• Figure 4 is the flow diagram of one embodiment of a process for refining a candidate predictor based on the status of the candidate predictor refinement flag. The process is performed by a processing logic that may comprise hardware, software, or a combination of both.
• the process begins by processing logic determining whether the candidate predictor refinement flag is on (processing block 401). If not, the process ends. If it is, the process transitions to processing block 402 where processing logic determines whether both MVPo and MVPi are valid. If they are, the process ends. If not, the process transitions to processing block 403, where processing logic tests whether either MVPo or MVPi is valid. If not, the process transitions to processing block 404 where the processing logic drops the candidate predictor from the predictor set and then the process ends. If either MVPo or MVP i is valid, the process transitions to processing block 405 where processing logic determines whether MVPo is valid.
• processing logic sets a equal to 0 and b equal to 1 (processing block 406) and sets MVPb equal to f pre d (MVP a , T a , Tb) (processing block 407) and then the process ends. If processing logic determines that MVPo is not valid, processing logic sets a equal to 1 and b equal to 0
• processing block 408 sets MVP b equal to f pred (MVP a , T a , T b ) at processing block 407. Thereafter, the process ends.
• the candidate predictor refinement flag is On' for all candidate predictors.
• the candidate predictor refinement flag is On' only when one (or more) constraints are satisfied.
• the constraints enforced can be one (or more) of the following: where is a predetermined parameter.
• embodiment is equal to zero.
• the function f d (MVP a ,T a ,T b ) can be one of the following:
• f T (T a ) is a lookup table whose entries are filled such that
• Figure 5 illustrates an example of generating a MVP for two cases where the MVP is only available for one list.
• the function used is option '2' above.
• the motion vector predictors are formed using the candidate predictors in the
• Figure 6 is a flow diagram of one embodiment for the process for deriving motion vector predictors. The process may be performed by processing logic that may comprise hardware, software, or a combination of both.
• processing logic begins by processing logic determining whether the predictor set is empty (processing block 601). If it is, processing logic sets the motion vector predictors to a default value (processing block 602) and the process ends. If not, processing logic forms motion vector predictors using the predictor set (processing block 603) and the process ends.
• the default value is the zero vector
• motion vector predictors are formed using the predictor set in one of the following methods.
• the notation ⁇ is used for the predictor set, where CP i
• N CPS represents the total number of candidate predictors in the predictor set.
• the index 'i' of the candidate predictor is included as part of the video bitstream when N cps > 1 .
• predictors in the predictor set. is set equal to the median of list 1
• the differential motion vector for one list can also be used in computing the differential motion vector of the other list (list d ).
• FIG. 7 is a flow diagram of one embodiment of a process for computing the differential motion vectors at the encoder. The process is performed by processing logic that may comprise hardware, software, or a combination of both.
• processing logic determines whether the status of differential motion vector prediction is on (processing block 702). If it is, processing logic decides the mapping of list 0 and list 1 to list c and list d (processing block 703) and sets the motion vector differential for c and d according to the following formulas:
• the processing logic sets the differential motion vector for list 0 and list 1 according to the following formulas:
• Figure 8 is a flow diagram of one embodiment of a process for reconstructing the motion vectors at the decoder. The process is performed by processing logic that may comprise hardware, software, or a combination of both.
• processing logic begins by processing logic deciding the on/off status for differential motion vector prediction (processing block 801). Next, processing logic determines whether the status of differential motion vector prediction is on (processing block 802). If it is, processing logic decides the mapping of list 0 and list 1 to list c and list d (processing block 803) and sets the motion vector for c and d according to the following formulas:
• processing logic sets the motion vector for list 0 and list 1 according to the following formulas:
• motion vector prediction is set to on) is only used when one (or more) constraints are satisfied; otherwise, the standard approach is used (in
• constraints enforced can be one (or) more of the following:
• ⁇ is equal to zero.
• c and d are determined in one of the following ways: 1. If
• the function f diff ⁇ MVP D , MVD c ,T c ,T d can be one of the following:
• Figure 9 illustrates an example of scaling MVD of one list to predict the MVD of the other list using option '2' above.
• Figures 10A and 10B are flow diagrams describing the process of encoding motion vectors at the encoder and the process of decoding motion vectors at the decoder, respectively.
• the processes are performed by processing logic that may comprise hardware, software, or a combination of both.
• the process of coding motion vectors at the encoder begins by processing logic constructing a predictor set (processing block 1001). Next, processing logic refines the predictor set (processing block 1002) and derives motion vector predictors (processing block 1003). After deriving motion vector predictors, processing logic computes differential motion vectors (processing block 1004) and encodes the differential motion vectors (processing block 1005). Thereafter the process ends.
• the process for decoding motion vectors at the decoder begins by processing logic constructing a predictor set (processing block 1011), refining the predictor set (processing block 1012), and deriving motion vector predictors (processing block 1013). Processing logic decodes differential motion vectors (processing block 1014) and reconstructs the motion vectors using the derived motion vector predictors (processing block 1015). Thereafter, the process ends.
• Figure 11 is a block diagram for an exemplary motion vector encoder which would be part of a video encoder.
• the attributes used for the target block in Figures 11 (and 12) are described as follows:
• Type This indicates the lists used by the target block.
• Ref_info This represents the reference frame information for each list used.
• MV_info This represents the motion vectors for each list used.
• a motion information memory 1101 stores the attributes of previously encoded blocks.
• predictor set constructor 1102 constructs a predictor set. In one embodiment, this is constructed as described above in Figure 2.
• predictor set refiner 1103 refines the predictor set. In one embodiment, refining of the predictor set includes the refinement of one (or) more candidate predictors. In one
• the refinement of a candidate predictor is performed as set forth in
• predictor generator 1104 derives the motion vector predictors. In one embodiment, predictor generator 1104 derives the motion vector predictors as set forth in Figure 6 above.
• Predictor generator 1104 sends the derived motion vector predictors to differential motion vector generator 1105 which, in response to the motion vector predictors and the motion vectors of the target block, computes the differential motion vectors.
• Differential motion vector generator 1105 sends the computed differential motion vectors to entropy encoder 1106 which encodes the differential motion vectors and includes them as part of the video bitstream. Thereafter, the attributes of the target block are sent to the motion information memory 1101 for storage (The Z -1 in the block diagram indicates that this does not happen until the motion vectors of target block are encoded).
• Figure 12 is a block diagram for an exemplary motion vector decoder which would be part of a video decoder.
• a motion information memory 1201 stores the attributes of previously decoded blocks.
• a predictor set constructor 1202 receives motion information from motion information memory 1201 and target block information. In response to these inputs, predictor set constructor 1202 constructs a predictor set. In one embodiment, the construction of the predictor set is performed as set forth in Figure 2 above.
• Predictor set constructor 1202 sends the predictor set to predictor set refiner 1203 which refines the predictor set. In one embodiment, refining of the predictor set includes the refinement of one (or) more candidate predictors.
• the refinement of a candidate predictor is performed as set forth in Figure 4 above.
• Predictor set refiner 1203 sends the refined predictor set to predictor generator 1204.
• predictor generator 1204 derives motion vector predictors.
• predictor generator 1204 derives the motion vector predictors as set forth in Figure 6 above.
• Predictor generator 1204 sends the derived motion vector predictors to motion vector generator 1206.
• Entropy decoder 1205 receives the video bit stream and performs entropy decoding on the video bitstream. This generates a decoded bitstream that includes decoded differential motion vectors. Entropy decoder 1205 sends the decoded differential motion vectors to motion vector generator 1206. In response to the derived motion vector predictors and the decoded differential motion vectors, motion vector generator 1206 reconstructs the motion vectors for the target block. Thereafter, motion vector generator 1206 sends the reconstructed motion vectors to be included in the attributes of the target block. Finally, the attributes of the target block are sent to the motion information memory 1201 for storage (The Z -1 in the block diagram indicates that this does not happen until the motion vectors of target block are reconstructed). [0052] Note that the motion vector encoders and decoders described herein can be part of any block-based hybrid video encoders/decoders that are well known in the art.
• Figure 13 is a block diagram of an exemplary computer system that may perform one or more of the operations described herein.
• computer system 1300 may comprise an exemplary client or server computer system.
• Computer system 1300 comprises a communication mechanism or bus 1311 for communicating information, and a processor 1312 coupled with bus 1311 for processing information.
• Processor 1312 includes a microprocessor, but is not limited to a microprocessor, such as, for example, PentiumTM, PowerPCTM, AlphaTM, etc.
• System 1300 further comprises a random access memory (RAM), or other dynamic storage device 1304 (referred to as main memory) coupled to bus 1311 for storing information and instructions to be executed by processor 1312.
• main memory dynamic storage device 1304
• Main memory 1304 also may be used for storing temporary variables or other intermediate information during execution of instructions by processor 1312.
• Computer system 1300 also comprises a read only memory (ROM) and/or other static storage device 1306 coupled to bus 1311 for storing static information and instructions for processor 1312, and a data storage device 1307, such as a magnetic disk or optical disk and its corresponding disk drive.
• ROM read only memory
• Data storage device 1307 is coupled to bus 1311 for storing information and instructions.
• Computer system 1300 may further be coupled to a display device
• FIG. 1321 such as a cathode ray tube (CRT) or liquid crystal display (LCD), coupled to bus 1311 for displaying information to a computer user.
• An alphanumeric input device 1322 may also be coupled to bus 1311 for communicating information and command selections to processor 1312.
• cursor control 1323 such as a mouse, trackball, trackpad, stylus, or cursor direction keys, coupled to bus 1311 for communicating direction information and command selections to processor 1312, and for controlling cursor movement on display 1321.
• cursor control 1323 such as a mouse, trackball, trackpad, stylus, or cursor direction keys
• bus 1324 which may be used for marking information on a medium such as paper, film, or similar types of media.
• Another device that may be coupled to bus 1311 is a wired/wireless communication capability 1325 to communication to a phone or handheld palm device.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Priority Applications (1)

Applications Claiming Priority (4)

Publications (1)

Family

ID=43823153

Family Applications (1)

Country Status (3)

Cited By (11)

Families Citing this family (53)

Citations (2)

Family Cites Families (24)

• 2010
  • 2010-09-24 US US12/890,400 patent/US9060176B2/en active Active
  • 2010-09-28 WO PCT/US2010/050572 patent/WO2011041321A1/en not_active Ceased
  • 2010-09-28 JP JP2012532241A patent/JP2013507055A/ja active Pending
• 2015
  • 2015-05-01 JP JP2015094146A patent/JP6039740B2/ja active Active

Patent Citations (2)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events