WO2011070730A1

WO2011070730A1 - Video coding device and video decoding device

Info

Publication number: WO2011070730A1
Application number: PCT/JP2010/006732
Authority: WO
Inventors: 蝶野慶一; 仙田裕三; 田治米純二; 青木啓史; 先崎健太
Original assignee: 日本電気株式会社
Priority date: 2009-12-07
Filing date: 2010-11-17
Publication date: 2011-06-16

Abstract

Provided is a video coding device and a video decoding device which are capable of efficiently compressing a video without reducing accuracy of a motion vector prediction even in a scene in which a motion vector changes between pictures due to the existence of acceleration or the like. The video coding device is provided with a prediction motion vector calculation means for calculating a prediction motion vector (temporal axis prediction motion vector or spatial axis prediction motion vector) of a picture to be coded, on the basis of the motion vector of two reference pictures before and after the picture to be coded. The video decoding device is provided with a prediction motion vector calculation means for calculating a prediction motion vector of a picture to be decoded, on the basis of the motion vector of two reference pictures before and after the picture to be decoded.

Description

Video encoding device and video decoding device

The present invention relates to a video encoding device and a video decoding device to which a video encoding technique for predicting a motion vector is applied.

Generally, a video encoding device digitizes a moving image signal input from the outside, and then performs encoding processing in accordance with a predetermined video encoding method to generate encoded data, that is, a bit stream.

There is ISO / IEC 14496-10 Advanced Video Coding (AVC) described in Non-Patent Document 1 as a predetermined video encoding method. A Joint Model method is known as a reference model for an AVC encoder (hereinafter referred to as a general video encoding device).

Referring to FIG. 11, the configuration and operation of a general video encoding apparatus that outputs each bit stream of a digitized video and outputs a bit stream will be described. One frame is composed of one frame picture of progressive scanning or two field pictures of interlace scanning. Hereinafter, for simplification of description, it is assumed that one frame is composed of one frame picture of progressive scanning.

As shown in FIG. 11, a general video encoding apparatus includes an MB buffer 101, a frequency conversion unit 102, a quantization unit 103, an entropy encoding unit 104, an inverse quantization unit 105, an inverse frequency conversion unit 106, a picture buffer. 107, a block distortion removal filter unit 108, a decoded picture buffer 109, an intra prediction unit 110, an inter-frame prediction unit 111, a motion vector prediction unit 112, an encoding control unit 113, and a switch 100.

A general video encoding apparatus divides each frame into 16 × 16 pixel size blocks called MB (Macro Block), and encodes each MB in order from the upper left of the frame. In AVC described in Non-Patent Document 1, the MB is further divided into blocks of 4 × 4 pixel size, and each 4 × 4 block is encoded.

FIG. 12 is an explanatory diagram showing an example of block division when the spatial resolution of the frame is QCIF (Quarter Common Intermediate Format). Hereinafter, for simplification of description, the operation of each device will be described by focusing on only the luminance pixel value.

The MB buffer 101 stores the pixel value of the encoding target MB of the input image frame. Hereinafter, the encoding target MB is simply referred to as an input MB.

The prediction signal supplied from the intra prediction unit 110 or the inter-frame prediction unit 111 via the switch 100 is subtracted from the input MB supplied from the MB buffer 101. Hereinafter, the input MB from which the prediction signal is reduced is referred to as a prediction error image block.

The intra prediction unit 110 generates an intra prediction signal using a reconstructed picture image stored in the picture buffer 107 having the same display time as the current picture as a reference image.

As for intra prediction, as described in 8.3.1 to 8.3.3 of Non-Patent Document 1, there are three types of block prediction modes, Intra_4 × 4, Intra_8 × 8, and Intra_16 × 16.

Referring to FIGS. 13A and 13C, it can be seen that Intra_4 × 4 and Intra_8 × 8 are intra predictions of 4 × 4 block size and 8 × 8 block size, respectively. However, a circle (◯) in the figure indicates a reference pixel for intra prediction, that is, a reconstructed image stored in the picture buffer 107.

In Intra_4 × 4 intra prediction, the surrounding pixels of the reconstructed image are used as reference pixels as they are, and reference signals are padded (extrapolated) in nine types of directions shown in FIG. In Intra_8 × 8 intra prediction, the peripheral pixels of the reconstructed picture are smoothed by the low-pass filters (1/2, 1/4, 1/2) described immediately below the right arrow in FIG. The predicted pixel is formed by extrapolating the reference pixel in nine types of directions shown in FIG.

Referring to FIG. 14 (a), it can be seen that Intra — 16 × 16 is intra prediction of 16 × 16 block size. Similarly to the example shown in FIG. 13, in FIG. 14, circles (◯) in the drawing indicate reference pixels used for intra prediction, that is, reconstructed picture images stored in the picture buffer 107. In Intra_16 × 16 intra prediction, surrounding pixels of the reconstructed image are used as reference pixels as they are, and prediction signals are formed by extrapolating reference pixels in the four types of directions shown in FIG.

Hereinafter, an MB encoded using an intra prediction signal is referred to as an intra MB. The block size of intra prediction is called intra prediction mode. The extrapolation direction is referred to as an intra prediction direction.

The inter-frame prediction unit 111 generates an inter-frame prediction signal using, as a reference image, an image of a reference picture stored in the decoded picture buffer 109 that has a display time different from that of the current picture. Hereinafter, an MB encoded using an inter-frame prediction signal is referred to as an inter MB. As the inter MB block size, 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4 can be selected.

FIG. 15 is an explanatory diagram illustrating an example of inter-frame prediction using a 16 × 16 block size as an example. The motion vector MV = (mv _x , mv _y ) shown in FIG. 15 is one of prediction parameters for inter-frame prediction that represents the amount of translation of the inter-frame prediction block (inter-frame prediction signal) of the reference picture with respect to the encoding target block. One. In AVC, in order to identify the reference picture used for the inter-frame prediction of the encoding target block in addition to the inter-frame prediction direction indicating the direction of the reference picture of the inter-frame prediction signal with respect to the encoding target picture of the encoding target block. The reference picture index is also a prediction parameter for inter-frame prediction. This is because in AVC, a plurality of reference pictures stored in the decoded picture buffer 109 can be used for inter-frame prediction.

Note that more detailed explanation of inter-frame prediction is described in 8.4 Interprediction process of Non-Patent Document 1.

Hereinafter, an MB encoded using an inter-frame prediction signal is referred to as an inter MB. The block size for inter-frame prediction is called inter prediction mode. The direction of inter-frame prediction is referred to as inter prediction direction.

Note that a picture encoded only with an intra MB is called an I picture. A picture coded including not only an intra MB but also an inter MB is called a P picture. A picture that is encoded including inter MBs that use not only one reference picture but also two reference pictures at the same time for inter-frame prediction is called a B picture. Further, in the B picture, the reference picture direction of the inter-frame prediction signal with respect to the encoding target picture of the encoding target block is the forward prediction with respect to the past inter-frame prediction, and the inter-frame prediction signal with respect to the encoding target picture of the encoding target block. Prediction of the future reference frame is referred to as backward prediction, and interframe prediction including the past and future is referred to as bidirectional prediction.

As shown in FIG. 16, the motion vector predicting unit 112 assumes that an encoding target block is E, an adjacent block on the left is A, an upper adjacent block is B, and an upper right adjacent block is C. The block predicted motion vector PMV = (pmv _x , pmv _y ) is obtained based on the median value of the respective motion vectors of the blocks A, B, and C. Hereinafter, for ease of explanation, a predicted motion vector obtained based on a block adjacent to the spatial axis is referred to as a spatial axis predicted motion vector SPMV = (spmv _x , spmv _y ). Each component of SPMV can be expressed formally as follows.

_{_{_{spmv x = Median (mv x _A}}} , mv x _B, mv x _C) ··· (1)
spmv _y = Median (mv _{y —} A, mv _{y —} B, mv _{y —} C) (2)

However, Median (X, Y, Z) are X, Y, a function that returns the median value of _{_{Z, mv x _A, mv x}} _B, mv x _C Block A, B, C motion vectors of the respective horizontal component, mv _{_{_{y _A, mv y _B, mv}}} y _C block a, B, a motion vector of the C respectively of the vertical component.

The encoding control unit 113 compares the intra prediction signal and the inter-frame prediction signal with the input MB stored in the MB buffer 101, and selects a prediction signal that reduces the energy of the prediction error image block.

Further, the encoding control unit 113 controls the switch 100 so that an image is predicted by the selected prediction signal, and supplies information related to the selected prediction signal to the entropy encoding unit 104. The information related to the selected prediction signal is the intra prediction mode and the intra prediction direction when the intra prediction signal is selected, and when the inter prediction signal is selected, the inter prediction mode, the inter prediction direction, And a differential motion vector (of the motion vector and the spatial axis predicted motion vector).

Subsequently, the encoding control unit 113 selects a base block size of integer DCT suitable for frequency conversion of the prediction error image block based on the input MB or the prediction error image block. As base block size options, there are 16 × 16, 8 × 8, and 4 × 4 block sizes. As the pixel value of the input MB or prediction error image block becomes flat, a larger base block size is selected. Information on the base size of the selected integer DCT is supplied to the frequency transform unit 102 and the entropy encoding unit 104.

Hereinafter, information related to the selected prediction signal and information related to the base size of the selected integer DCT will be referred to as auxiliary information.

In addition, the encoding control unit 113 monitors the bit number of the bit stream output from the entropy encoding unit 104 in order to encode the picture with the target bit number, and the bit number of the output bit stream is the target bit number. If the number of bits is less than the target number of bits, a quantization parameter that reduces the quantization step size is supplied. Supply. As such, the output bitstream is encoded to approach the target number of bits.

The frequency conversion unit 102 converts the prediction error image block by frequency conversion from the spatial domain to the frequency domain with the selected base size of the integer DCT. The prediction error converted to the frequency domain is called a conversion coefficient. For the frequency transformation, orthogonal transformation such as DCT (Discrete Cosine Transform) or Hadamard transformation can be used. The integer DCT means frequency conversion based on a base obtained by approximating a DCT base with an integer value in a general video encoding apparatus.

The quantization unit 103 quantizes the transform coefficient with a quantization step size corresponding to the quantization parameter supplied from the encoding control unit 113. Note that the quantization index of the quantized transform coefficient is also called a level. The entropy coding unit 104 entropy codes the auxiliary information and the quantization index, and outputs the bit string, that is, the bit stream.

The inverse quantization unit 105 and the inverse transform unit 106 inversely quantize the quantization index supplied from the quantization unit 103 for subsequent encoding, and perform inverse frequency transform to return to the original space region. Hereinafter, the prediction error image block returned to the original space area is referred to as a reconstructed prediction error image block.

The picture buffer 107 stores a reconstructed image block obtained by adding a prediction signal to a reconstructed prediction error image block until all MBs included in the current picture are encoded. Hereinafter, a picture constituted by the reconstructed image block is referred to as a reconstructed picture. The picture buffer 107 also stores auxiliary information related to the reconstructed image block.

The block distortion removal filter unit 108 removes block distortion of the reconstructed picture stored in the picture buffer 107.

The decoded picture buffer 109 stores the reconstructed picture from which the block distortion supplied from the block distortion removal filter 108 is removed as a reference picture. The decoded picture buffer 109 also stores auxiliary information related to the reconstructed image block from which block distortion has been removed.

The video encoding device shown in FIG. 11 generates a bit stream by the processing as described above.

International Publication No. 2007/074543

In a general video coding technique, time-axis motion vector prediction assuming that the motion vectors between pictures are consistent is used in the coding of temporal direct mode (Temporal DIRECT mode) of a B picture.

In the temporal axis motion vector prediction, as shown in FIG. 17, the motion vector MV _{base of} a block of one future reference picture (Collocated block) in the display order adjacent to the encoding target block (Current block) in the temporal axis. Is used.

The motion vector MV _col of the collocated block is obtained by calculating the time difference tb between the coding target picture (pic (t)) of the coding target block and the reference picture (pic (t + 1)) of the collocated block and the reference picture (pic (t + 1) of the collocated block. ) And the reference picture (pic (t−1)) referenced by the motion vector of the collocated block is interpolated on the basis of the time difference td, and the following formula (3) and formula (4) The backward predicted motion vectors TPMV _L0 and TPMV _L1 are calculated (see FIG. 18).

Non-Patent Document 1 proposes Symmetric Mode that applies temporal-axis motion vector prediction to inter-frame prediction for backward prediction and generates a motion vector for forward prediction from the motion vector for backward prediction. In Non-Patent Document 2 and Non-Patent Document 3, when the motion vector of the Collocated block is not available (that is, when the Collocated block is in the intra MB mode), the temporal axis motion vector prediction is prohibited, and instead, the spatial axis motion vector Technologies that use predictions have been proposed. In Patent Document 1, when the motion vector of the Collocated block cannot be used (that is, in the intra MB mode), the motion vector of another reference picture adjacent in the time direction is extrapolated to obtain the time-axis predicted motion vector. Techniques for calculating have been proposed.

However, the techniques described in non-patent literature and patent literature assume that the motion vectors between pictures are consistent. Therefore, when there is a change in the motion vector between pictures, appropriate time-axis motion vector prediction cannot be performed. As a result, for example, when there is a change in the motion vector between pictures due to the presence of acceleration, the techniques described in non-patent literature and patent literature reduce the accuracy of motion vector prediction and efficiently There is a problem that cannot be compressed.

Therefore, the present invention provides a video encoding apparatus and video that can efficiently compress video without reducing the accuracy of motion vector prediction even in a scene in which there is a change in the motion vector between pictures due to the presence of acceleration or the like. An object is to provide a decoding device.

The video encoding apparatus according to the present invention includes a prediction motion vector calculation unit that calculates a prediction motion vector of an encoding target image based on motion vectors of two reference pictures before and after the encoding target image. To do.

The video decoding apparatus according to the present invention is characterized in that it includes a predicted motion vector calculation means for calculating a predicted motion vector of a decoding target image based on motion vectors of two reference pictures before and after the decoding target image.

A video encoding method according to the present invention is a video encoding method executed by a video encoding device, and predictive motion of an encoding target image based on motion vectors of two reference pictures before and after the encoding target image. It is characterized by calculating a vector.

A video decoding method according to the present invention is a video decoding method executed by a video decoding device, and calculates a predicted motion vector of a decoding target image based on motion vectors of two reference pictures before and after the decoding target image. It is characterized by.

The video encoding program according to the present invention is characterized by causing a computer to calculate a predicted motion vector of an encoding target image based on motion vectors of two reference pictures before and after the encoding target image.

The video decoding program according to the present invention is characterized by causing a computer to calculate a predicted motion vector of a decoding target image based on motion vectors of two reference pictures before and after the decoding target image.

According to the present invention, even in a scene where there is a change in the motion vector between pictures due to the presence of acceleration or the like, the video can be efficiently compressed without reducing the accuracy of motion vector prediction.

It is a block diagram of the video coding apparatus of 1st Embodiment. It is explanatory drawing which shows the read-in operation | movement of two reference pictures before and behind the encoding object image by a motion vector estimation part. It is an explanatory diagram showing the motion vector MV _base based on the weighted sum of the motion vectors before and after two reference pictures of the encoding target image. Motion vector MV _base time based on the axis of the encoding target picture prediction motion is an explanatory diagram showing a vector TPMV _L0 and TPMV _L1. It is a block diagram of the video decoding apparatus of 2nd Embodiment. It is a block diagram which shows the structural example of the information processing system which can implement | achieve the function of the video coding apparatus and video decoding apparatus by this invention. It is a block diagram which shows the main structures of the video coding apparatus by this invention. It is a block diagram which shows the main structures of the video decoding apparatus by this invention. It is a flowchart which shows the process of the video coding apparatus by this invention. It is a flowchart which shows the process of the video decoding apparatus by this invention. It is a block diagram which shows the structure of a general video coding apparatus. It is explanatory drawing which shows the example of a block division. It is explanatory drawing for demonstrating intra prediction of Intra_4x4 and Intra_8x8. It is explanatory drawing for demonstrating Intra_16x16 intra prediction. It is explanatory drawing which shows the example of the inter-frame prediction which made the block size of 16x16 an example. It is explanatory drawing which shows the image block adjacent to a process target image block within a screen. It is explanatory drawing which shows the motion vector MV _base in a prior art. It is explanatory drawing which shows the time-axis prediction motion vector TPMV _L0 and TPMV _L1 in a prior art.

Embodiment 1. FIG.
The video encoding apparatus according to the present embodiment, as a feature of the motion vectors of the two reference pictures before and after the encoding target image, is based on the norm of the motion vectors of the two reference pictures before and after the encoding target image. There is provided means for detecting a change in a motion vector, and further generating a predicted motion vector by compensating for the change in the motion vector based on a weighted sum of motion vectors of two reference pictures before and after.

FIG. 1 is a block diagram showing a video encoding apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the video encoding apparatus according to the present embodiment includes an MB buffer 101, a frequency conversion unit 102, a quantization unit 103, an entropy encoding unit 104, an inverse quantization unit 105, an inverse frequency conversion unit 106, a picture. A buffer 107, a block distortion removal filter unit 108, a decoded picture buffer 109, an intra prediction unit 110, an inter-frame prediction unit 111, a motion vector prediction unit 112, an encoding control unit 113, and a switch 100 are provided.

Compared with the general video encoding apparatus shown in FIG. 11, the video encoding apparatus according to the present embodiment has a motion vector predicting unit 112 that performs not only a reconstructed picture stored in the picture buffer 107 but also a decoded picture. A feature is that the auxiliary information of the reconstructed picture stored in the buffer 109 can also be accessed. The motion vector prediction unit 112 detects a motion vector change between pictures based on the norms of the motion vectors of the two reference pictures before and after the encoding target image, and further weights the motion vectors of the two reference pictures before and after. Compensate for motion vector changes based on the sum. That is, the motion vector prediction unit 112 corresponds to a unit that generates a predicted motion vector. Therefore, in the following description, the operation of the motion vector predicting unit 112, which is a feature of the video encoding device of the present embodiment, will be described in detail.

The MB buffer 101 stores the pixel value of the encoding target MB of the input picture.

The prediction signal supplied from the intra prediction unit 110 or the inter-frame prediction unit 111 via the switch 100 is subtracted from the input MB supplied from the MB buffer 101.

The inter-frame prediction unit 111 generates an inter-frame prediction signal using, as a reference image, an image of a reference picture stored in the decoded picture buffer 109 that has a display time different from that of the current picture.

For the P picture, the motion vector prediction unit 112 generates a spatial axis prediction motion vector SPMV = (spmv _x , spmv _y ) as in a general video encoding technique, and sends the prediction motion vector to the encoding control unit 113. Supply as.

The motion vector prediction unit 112 operates differently from a general video encoding technique for B pictures. Details of the operation of the motion vector prediction unit 112 of the present embodiment for B pictures will be described.

First, the motion vector prediction unit 112 receives blocks (Collocated block L0 and Collocated block L1) of two reference pictures (pic (t-1) and pic (t + 1)) before and after the encoding target picture from the decoded picture buffer 206. Each motion vector (MV _baseL0 and MV _baseL1 ) is read from (see FIG. 2).

Subsequently, the motion vector prediction unit 112 calculates the norm DNMV_norm of the difference between the normalized motion vectors NMV _baseL0 and NMV _baseL1 normalized by the distance between reference pictures (td _L0 and td _L1 ) of the motion vectors of the two reference pictures before and after. Is compared with a predetermined threshold th_norm. Normalized motion vectors NMV _baseL0 and NMV _baseL1 and norm DNMV_norm are expressed by the following equations (5), (6), and (7).

NMV _baseL0 = MV _baseL0 / td _L0 (5)
NMV _baseL1 = MV _baseL1 / td _L1 (6)
_{_{DNMV_norm = ∥NMV baseL0 -NMV baseL1 ∥ ···}} (7)

However, ∥X∥ is a function for calculating the L1 norm of the vector X. In place of the L1 norm, the L2 norm or the maximum absolute value difference of each component may be used.

When the norm DNMV_norm is larger than the predetermined threshold th_norm, the motion vector prediction unit 112 does not have an acceleration of a motion vector between the reference picture pic (t−1) and the reference picture pic (t + 1) (that is, the reference picture pic (T-1) and the reference picture pic (t + 1) are determined to have a motion vector change due to another factor), and only the MV _baseL1 is used, and the following equations (8), (9) , (10), TPMV _L0 and TPMV _L1 are generated by supplying predicted motion vectors (that is, time-axis predicted motion vectors similar to the time-axis predicted motion vectors based on a general video encoding technique) and supplied to the encoding control unit 113. .

MV _base = MV _baseL1 (8)
TPMV _L0 = tb * MV _base / td _L1 (9)
TPMV _L1 = (tb−td _L1 ) * MV _base / td _L1 (10)

However, TPMV _L0 is a predicted motion vector for the forward prediction motion vector, and TPMV _L1 is a predicted motion vector for the backward prediction motion vector.

When the norm is equal to or smaller than the predetermined threshold th_norm, the motion vector prediction unit 112 determines that there is an acceleration of the motion vector between the reference picture pic (t−1) and the reference picture pic (t + 1), and MV _baseL0 and MV _Using both of _baseL1 , the following equations (11), (12), and (13) are used to predict motion vectors (time-axis prediction according to the present invention, which is different from motion vector prediction motion vectors based on general video coding techniques). Motion vectors) TPMV _L0 and TPMV _L1 are generated and supplied to the encoding control unit 113.

MV _base = td _L1 * (MV _baseL0 / td _L0 + MV _baseL1 / td _L1 ) / 2 (11)
TPMV _L0 = tb * MV _base / td _L1 (12)
TPMV _L1 = (tb−td _L1 ) * MV _base / td _L1 (13)

However, the motion vector in Expression (11) is a motion vector based on the weighted sum of motion vectors of two reference pictures before and after the encoding target image (see FIG. 3). Therefore, the obtained prediction motion vector is a time-axis prediction motion vector based on the weighted sum of the motion vectors of the two reference pictures before and after the encoding target image (see FIG. 4).

When the MV _baseL0 and the MV _baseL1 are motion vectors having the same value, the temporal axis predicted motion vector in the present embodiment is the same as the temporal axis predicted motion vector by a general video coding technique. However, when there is a difference due to acceleration between MV _baseL0 and MV _baseL1 , as is clear from equation (11) and FIG. 4, the temporal axis predicted motion vector in this embodiment has a change in motion vector due to acceleration. It becomes a compensated prediction motion vector. Therefore, even in a scene in which there is a change in the motion vector between pictures due to the presence of acceleration or the like, the accuracy of motion vector prediction by the motion vector prediction unit 112 does not decrease, so that the video can be efficiently compressed.

This completes the description of the operation details of the motion vector prediction unit 112 of the present embodiment in the B picture.

The encoding control unit 113 compares the intra prediction signal and the inter-frame prediction signal with the input MB of the MB buffer 101, respectively, and selects a prediction signal that reduces the energy of the prediction error image block.

In addition, the encoding control unit 113 monitors the bit number of the bit stream output from the entropy encoding unit 104 in order to encode the picture with the target bit number, and the bit number of the output bit stream is the target bit number. If the number of bits is less than the target number of bits, a quantization parameter that reduces the quantization step size is supplied. Supply. As a result, the output bit stream is encoded so as to approach the target number of bits.

The frequency conversion unit 102 converts the prediction error image block by frequency conversion from the spatial domain to the frequency domain with the selected base size of the integer DCT. The prediction error converted to the frequency domain is called a conversion coefficient. For frequency conversion, orthogonal transform such as DCT or Hadamard transform can be used. The integer DCT means frequency conversion based on a base obtained by approximating a DCT base with an integer value in a general video encoding apparatus.

The quantization unit 103 quantizes the transform coefficient with a quantization step size corresponding to the quantization parameter supplied from the encoding control unit 113. Note that the quantization index of the quantized transform coefficient is also called a level.

The entropy coding unit 104 entropy codes the auxiliary information and the quantization index, and outputs the bit string, that is, the bit stream.

Based on the above-described operation, the video encoding device of this embodiment generates a bit stream.

Embodiment 2. FIG.
The video decoding apparatus according to the present embodiment detects a change in motion vector between pictures based on the norm as the feature of the motion vector of two reference pictures before and after the decoding target image, and further, Means for generating a motion vector predictor by compensating for a change in motion vector based on a weighted sum of motion vectors.

FIG. 5 is a block diagram showing a video decoding apparatus according to the second embodiment of the present invention. As shown in FIG. 5, the video decoding apparatus according to the present embodiment includes an entropy decoding unit 201, an inverse quantization unit 202, an inverse frequency conversion unit 203, a picture buffer 204, a block distortion removal filter unit 205, a decoded picture buffer 206, an intra A prediction unit 207, an inter-frame prediction unit 208, a motion vector prediction unit 209, a decoding control unit 210, and a switch 200 are provided.

The entropy decoding unit 201 performs entropy decoding on the bitstream to obtain information related to the prediction signal of the decoding target MB, the base size of the integer DCT, and the quantization index.

The decoding control unit 210 prepares to supply an intra prediction signal or an inter-frame prediction signal as a prediction signal.

When the decoding control unit 210 prepares to supply an inter-frame prediction signal, the motion vector prediction unit 209 calculates a prediction motion vector and sends it to the inter-frame prediction unit 208 in the same manner as the motion vector prediction unit 112 in the first embodiment. Supply.

The intra prediction unit 207 performs entropy decoding supplied to the intra MB from the reconstructed image stored in the picture buffer 204 having the same display time as the currently decoded frame, via the decoding control unit 210. An intra prediction signal is generated based on the intra prediction mode and the intra prediction direction.

The inter-frame prediction unit 208 performs entropy-decoded inter-picture supplied via the decoding control unit 210 from a reference image stored in the decoded picture buffer 206 that has a display time different from that of the currently decoded frame. An inter-frame prediction signal is generated based on the prediction mode, the inter prediction direction, the difference motion vector, and the prediction motion vector supplied from the motion vector prediction unit 209.

The decoding control unit 210 controls the switch 200 based on information (intra MB or inter MB) related to the entropy-decoded prediction signal, and predicts an image by the intra prediction signal or the inter-frame prediction signal.

The inverse quantization unit 202 and the inverse transform unit 203 inversely quantize the quantization index supplied from the entropy decoding unit 201, further perform inverse frequency transform, and return the original spatial region (reconstructed prediction error image block).

The picture buffer 204 stores a reconstructed image block obtained by adding a prediction signal supplied via the switch 200 to the reconstructed prediction error image block until all MBs included in the current frame are encoded.

The block distortion removal filter unit 205 removes block distortion of the reconstructed image picture stored in the picture buffer 204.

The decoded picture buffer 206 stores the reconstructed image picture from which block distortion has been removed by the block distortion removal filter unit 205 as a reference picture. The reference image picture is output as an expanded frame at an appropriate display timing.

Based on the above-described operation, the video decoding apparatus according to the present embodiment decompresses the bit stream.

Embodiment 3. FIG.

In the first and second embodiments described above, the motion vectors of the two reference pictures before and after the encoding target picture or the decoding target picture are characterized by the motion vector norm between the pictures based on the norm of the motion vectors of the two reference pictures. A video encoding device including a motion vector prediction unit that detects a change in a motion vector and further generates a predicted motion vector by compensating for the change in the motion vector based on a weighted sum of motion vectors of two reference pictures before and after A video decoding device is shown.

As another embodiment, an embodiment in which a predicted motion vector based on a motion vector of an image adjacent in the screen is generated is also conceivable.

That is, a change in a motion vector between pictures is detected based on the norm of motion vectors of two reference pictures before and after the encoding target image or the decoding target image, and further, based on the motion vectors of adjacent images in the screen. An embodiment using a motion vector prediction unit (motion vector prediction unit 112 and motion vector prediction unit 209) that generates a predicted motion vector by compensating for a change in the motion vector is also conceivable.

The operation of the motion vector prediction unit in the B picture in such an embodiment will be described below.

First, the motion vector prediction unit performs blocks (Collocated block L0 and Collocated block L1) of two reference pictures (pic (t−1) and pic (t + 1)) stored in the decoded picture buffer 109 or the decoded picture buffer 206. ) _Read each motion vector (MV _baseL0 and MV _baseL1 ).

Subsequently, the motion vector prediction unit determines whether or not the reference picture indicated by the motion vector MV _baseL1 of the Collocated block L1 is a reference picture to which the Collocated block L0 belongs. When the reference picture _pointed to by the motion vector MV _baseL1 of the Collocated block L1 is not the reference picture to which the Collocated block L0 belongs, a large change in the motion vector between the reference picture pic (t−1) and the reference picture pic (t + 1) Based on the motion vectors of the image blocks A, B, and C (see FIG. 16) adjacent in the screen, the predicted motion vector (spatial axis predicted motion vector) ) Generate SPMV = (spmv _x , spmv _y ).

_{_{_{spmv x = Median (mv x _A}}} , mv x _B, mv x _C) ··· (14)
spmv _y = Median (mv _{y —} A, mv _{y —} B, mv _{y —} C) (15)

When it is determined that there is a large change in the motion vector between the reference picture pic (t−1) and the reference picture pic (t + 1), the spatial axis predicted motion vector should be used rather than the temporal axis predicted motion vector. Thus, the accuracy of motion vector prediction is improved. Specifically, the accuracy of motion vector prediction is improved by generating a predicted motion vector by compensating for a large change in the motion vector between reference pictures based on the motion vector of an image adjacent in the screen.

When the reference picture _pointed to by the motion vector MV _baseL1 of the collocated block L1 is a reference picture to which the collocated block L0 belongs, the motion vector prediction unit further determines the inter-reference picture distances (td) of the motion vectors of the two preceding and following reference pictures. The norm DNMV_norm of the difference _{between the} normalized motion vectors NMV _baseL0 and NMV _baseL1 normalized by _L0 and td _L1 ) is compared with a predetermined threshold th_norm. The normalized motion vectors NMV _baseL0 and NMV _baseL1 and the norm DNMV_norm are expressed by the following formulas (16), (17), and (18).

NMV _baseL0 = MV _baseL0 / td _L0 (16)
NMV _baseL1 = MV _baseL1 / td _L1 (17)
DNMV_norm = || NMV _baseL0 -NMV _baseL1 || (18)

When the norm DNMV_norm is larger than the predetermined threshold th_norm, the motion vector prediction unit does not have an acceleration of a motion vector between the reference picture pic (t−1) and the reference picture pic (t + 1) (that is, the reference picture pic ( t-1) and the reference picture pic (t + 1) are determined to have a motion vector change due to another factor), and only the MV _baseL1 is used, and the following equations (19), (20), In (21), prediction motion vectors (that is, time-axis prediction motion vectors by a general video encoding technique and time-axis prediction motion vectors similar to the technique) TPMV _L0 and TPMV _L1 are generated to generate the encoding control unit 113 or decoding It supplies to the control part 210.

MV _base = MV _baseL1 (19)
TPMV _L0 = tb * MV _base / td _L1 (20)
TPMV _L1 = (tb−td _L1 ) * MV _base / td _L1 (21)

When the norm is equal to or smaller than a predetermined threshold, it is determined that there is an acceleration of a motion vector between the reference picture pic (t−1) and the reference picture pic (t + 1), and both MV _baseL0 and MV _baseL1 are used, Predicted motion vectors (time-axis predicted motion vectors according to the present invention, which are different from the time-axis predicted motion vectors according to a general video coding technique) in the following equations (22), (23), and (24) TPMV _L0 and TPMV _L1 Is supplied to the encoding control unit 113 or the decoding control unit 210.

MV _base = td _L1 * (MV _baseL0 / td _L0 + MV _baseL1 / td _L1 ) / 2 (22)
TPMV _L0 = tb * MV _base / td _L1 (23)
TPMV _L1 = (tb−td _L1 ) * MV _base / td _L1 (24)

However, the motion vector of Expression (22) is a motion vector based on the weighted sum of motion vectors of two reference pictures before and after (see FIG. 3). Therefore, the obtained prediction motion vector is a time-axis prediction motion vector based on the weighted sum of the motion vectors of two reference pictures before and after (see FIG. 4).

Embodiment 4 FIG.
A motion vector prediction device that detects a change in motion vector between pictures using an inner product or outer product instead of a norm as a feature of motion vectors of two reference pictures before and after an encoding target image or a decoding target image ( An embodiment using the motion vector prediction unit 112 and the motion vector prediction unit 209) is also conceivable.

By using the inner product, for example, the angle θ formed by the motion vectors of the two front and rear reference pictures can be obtained. When the angle θ is larger than the predetermined threshold th_theta, there is no motion vector acceleration between the reference picture pic (t−1) and the reference picture pic (t + 1) (that is, the reference picture pic (t−1) and the reference It can be determined that there is a change in motion vector due to another factor between the picture pic (t + 1)).

By using the outer product, for example, it can be determined whether or not the motion vectors of two reference pictures before and after are parallel. If the motion vectors of the two front and rear reference pictures are parallel (that is, the outer product is zero) and each has the same direction, the reference picture pic (t−1) and the reference picture pic (t + 1) are between It can be determined that the acceleration of the motion vector exists.

Also, an embodiment using a motion vector prediction device that detects a change in motion vector between pictures by using a combination of norm and inner product is also conceivable.

Specifically, when the norm of the motion vectors of the two reference pictures before and after is non-zero and the angle θ formed by each motion vector is smaller than a predetermined value th_theta, the reference picture pic (t−1) and the reference picture It can be determined that there is an acceleration of the motion vector between pic (t + 1).

Also, an embodiment using a motion vector prediction device that detects a change in motion vector between pictures by using a combination of norm and outer product is also conceivable.

Specifically, when the norm of the motion vectors of the two preceding and following reference pictures is non-zero and the outer product of the respective motion vectors is zero, the reference picture pic (t−1) and the reference picture pic (t + 1) It can be determined that motion vector acceleration exists between them.

Furthermore, an embodiment using a motion vector prediction device that detects a change in motion vector between pictures by using a combination of a norm, an inner product, and an outer product is also conceivable.

Specifically, when the norm of the motion vectors of the two preceding and following reference pictures is non-zero, the inner product is a value larger than 0, and the outer product is zero, the reference picture pic (t−1) and the reference picture pic (t + 1) It can be determined that there is an acceleration of the motion vector.

Note that the concept of the present embodiment may be applied to the first embodiment and the second embodiment, or may be applied to the third embodiment.

In each of the embodiments described above, the video encoding device and the video decoding device detect and compensate for a change in the motion vector between pictures based on the motion vectors of two reference pictures before and after the encoding target image or the decoding target image. Means. Specifically, the video encoding device and the video decoding device according to an embodiment can calculate a motion vector between pictures based on one or more of an inner product, an outer product, and a norm of motion vectors of two reference pictures. Means for detecting a change, and means for generating a motion vector predictor that compensates for a change in a motion vector based on a weighted sum of motion vectors of two reference pictures before and after. Further, the video encoding device and video decoding device according to another embodiment can change a motion vector between pictures based on any one or more of an inner product, an outer product, and a norm of motion vectors of two reference pictures. Means for detecting, and means for generating a predicted motion vector for compensating for a change in a motion vector based on a motion vector of an image adjacent in the screen.

Note that the present invention and the invention described in Patent Document 1 are similar in that the motion vectors of the two reference pictures before and after are used, but in the present invention, the motion vectors of the two reference pictures before and after, Alternatively, the difference is that a predicted motion vector in which a change in motion vector is compensated is calculated based on a motion vector of an adjacent block in a picture. In the present invention, a prediction motion vector in which a change in motion vector due to acceleration is compensated can be generated by this difference.

In the present invention, the inner product, outer product, and norm of the motion vectors of the two reference pictures before and after are not simple criteria such as the presence or absence of motion vectors of reference pictures adjacent in the time direction (whether or not the intra MB mode is used). The point of detecting a change of a motion vector between pictures using any one or more of the above criteria also indicates an inventive step.

Further, although each of the above embodiments can be configured by hardware, it can also be realized by a computer program.

The information processing system shown in FIG. 6 includes a processor 1001, a program memory 1002, a storage medium 1003 for storing video data, and a storage medium 1004 for storing a bitstream. The storage medium 1003 and the storage medium 1004 may be separate storage media, or may be storage areas composed of the same storage medium. A magnetic storage medium such as a hard disk can be used as the storage medium.

In the information processing system shown in FIG. 6, the program memory 1002 stores a program for realizing the function of each block (excluding the buffer block) shown in FIGS. The processor 1001 implements the functions of the video encoding device or the video decoding device shown in FIGS. 1 and 5 by executing processing in accordance with the program stored in the program memory 1002.

FIG. 7 is a block diagram showing the main configuration of the video encoding apparatus according to the present invention. As shown in FIG. 7, the video encoding apparatus according to the present invention is a prediction motion vector calculation means for calculating a prediction motion vector of an encoding target image based on motion vectors of two reference pictures before and after the encoding target image. 11 (implemented by the motion vector prediction unit 112 in the first embodiment).

For example, the predicted motion vector calculation unit 11 calculates a time-axis predicted motion vector based on the weighted sum of the motion vectors of the two preceding and following reference pictures when the feature amounts of the motion vectors of the two preceding and following reference pictures satisfy a predetermined value. The spatial axis prediction motion vector based on the motion vector of the image adjacent to the encoding target image in the screen is calculated when the feature amounts of the motion vectors of the two reference pictures before and after the predetermined value do not satisfy the predetermined value.

The feature quantities of the motion vectors of the two reference pictures before and after used by the predicted motion vector calculation means 11 include, for example, one or more of an inner product, an outer product, and a norm of the motion vectors of the two reference pictures before and after.

FIG. 8 is a block diagram showing the main configuration of the video decoding apparatus according to the present invention. As shown in FIG. 8, the video decoding apparatus according to the present invention is based on the motion vector predictor 21 (first motion vector calculation means 21) for calculating a motion vector predictor of a decoding target image based on motion vectors of two reference pictures before and after the decoding target image. In the second embodiment, it is realized by the motion vector prediction unit 209).

The predicted motion vector calculation means 21 calculates, for example, a time-axis predicted motion vector based on a weighted sum of motion vectors of two reference pictures before and after, as a predicted motion vector of a decoding target image.

The feature quantities of the motion vectors of the two reference pictures before and after that used by the predicted motion vector calculation means 21 include, for example, one or more of the inner product, outer product, and norm of the motion vectors of the two reference pictures before and after.

FIG. 9 is a flowchart showing the main steps of the video encoding method according to the present invention. As shown in FIG. 9, in the video encoding method according to the present invention, a predicted motion vector of an encoding target image is calculated based on the motion vectors of two reference pictures before and after the encoding target image (step S11). including.

FIG. 10 is a flowchart showing the main steps of the video decoding method according to the present invention. As shown in FIG. 10, the video decoding method according to the present invention includes a step (step S21) of calculating a predicted motion vector of a decoding target image based on motion vectors of two reference pictures before and after the decoding target image.

Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2009-277952 filed on Dec. 7, 2009, the entire disclosure of which is incorporated herein.

DESCRIPTION OF SYMBOLS 11 Prediction motion vector calculation means 21 Prediction motion vector calculation means 100 Switch 101 MB buffer 102 Frequency conversion part 103 Quantization part 104 Entropy encoding part 105 Inverse quantization part 106 Inverse frequency conversion part 107 Picture buffer 108 Block distortion removal filter part 109 Decoded picture buffer 110 Intra prediction unit 111 Inter-frame prediction unit 112 Motion vector prediction unit 113 Coding control unit 200 Switch 201 Entropy decoding unit 202 Inverse quantization unit 203 Inverse frequency conversion unit 204 Picture buffer 205 Block distortion removal filter unit 206 Decoded picture Buffer 207 Intra prediction unit 208 Inter-frame prediction unit 209 Motion vector prediction unit 210 Decoding control unit 1001 Processor 1 002 Program memory 1003 Storage medium 1004 Storage medium

Claims

A video encoding device comprising prediction motion vector calculation means for calculating a prediction motion vector of the encoding target image based on motion vectors of two reference pictures before and after the encoding target image.
The video code according to claim 1, wherein the prediction motion vector calculation means calculates a time-axis prediction motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures as a prediction motion vector of the encoding target image. Device.
The predicted motion vector calculation unit calculates a spatial axis predicted motion vector based on a motion vector of an image adjacent to the encoding target image in a screen as a predicted motion vector of the encoding target image. Video encoding device.
The predicted motion vector calculation means calculates a time-axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures when a feature amount of the motion vector of the two preceding and following reference pictures satisfies a predetermined value. The video encoding device according to claim 1, wherein a spatial axis predicted motion vector is calculated based on a motion vector of an image adjacent to the encoding target image in a screen when the predetermined value is not satisfied.
The video encoding device according to claim 4, wherein the feature quantity of the motion vectors of the two preceding and following reference pictures includes at least one of an inner product, an outer product, and a norm of the motion vectors of the two preceding and following reference pictures.
A video decoding device comprising prediction motion vector calculation means for calculating a prediction motion vector of the decoding target image based on motion vectors of two reference pictures before and after the decoding target image.
The video decoding device according to claim 6, wherein the predicted motion vector calculation means calculates a time-axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures as a predicted motion vector of the decoding target image. .
The video decoding according to claim 6, wherein the predicted motion vector calculating unit calculates a spatial axis predicted motion vector based on a motion vector of an image adjacent to the decoding target image in a screen as a predicted motion vector of the decoding target image. apparatus.
The predicted motion vector calculation means calculates a time-axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures when a feature amount of the motion vector of the two preceding and following reference pictures satisfies a predetermined value. The video decoding apparatus according to claim 6, wherein a spatial axis predicted motion vector is calculated based on a motion vector of an image adjacent to the decoding target image in a screen when the predetermined value is not satisfied.
The video decoding apparatus according to claim 9, wherein the feature quantity of the motion vectors of the two preceding and following reference pictures includes at least one of an inner product, an outer product, and a norm of the motion vectors of the two preceding and following reference pictures.
A video encoding method executed by a video encoding device,
A video encoding method for calculating a predicted motion vector of an encoding target image based on motion vectors of two reference pictures before and after the encoding target image.
The video encoding method according to claim 11, wherein a temporal axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures is calculated as a predicted motion vector of the encoding target image.
The video encoding method according to claim 11, wherein a spatial axis predicted motion vector based on a motion vector of an image adjacent to the encoding target image in a screen is calculated as the prediction motion vector of the encoding target image.
When the feature quantities of the motion vectors of the two preceding and following reference pictures satisfy a predetermined value, a temporal axis predicted motion vector is calculated based on a weighted sum of the motion vectors of the two preceding and following reference pictures, and the predetermined value is satisfied. The video encoding method according to claim 11, wherein a spatial axis predicted motion vector based on a motion vector of an image adjacent to the encoding target image in a screen is calculated when there is no image.
The video encoding method according to claim 14, wherein one or more of an inner product, an outer product, and a norm of motion vectors of the two preceding and following reference pictures is used as a feature quantity of the motion vector of the two preceding and following reference pictures.
A video decoding method executed by a video decoding device,
A video decoding method for calculating a predicted motion vector of a decoding target image based on motion vectors of two reference pictures before and after the decoding target image.
The video decoding method according to claim 16, wherein a time-axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures is calculated as a predicted motion vector of the decoding target image.
The video decoding method according to claim 16, wherein a spatial axis predicted motion vector based on a motion vector of an image adjacent to the decoding target image in a screen is calculated as the prediction motion vector of the decoding target image.
When the feature quantities of the motion vectors of the two preceding and following reference pictures satisfy a predetermined value, a temporal axis predicted motion vector is calculated based on a weighted sum of the motion vectors of the two preceding and following reference pictures, and the predetermined value is satisfied. The video decoding method according to claim 16, wherein a spatial axis predicted motion vector based on a motion vector of an image adjacent to the decoding target image in a screen is calculated when there is no image.
The video decoding method according to claim 19, wherein one or more of an inner product, an outer product, and a norm of motion vectors of the two preceding and following reference pictures is used as a feature quantity of the motion vector of the two preceding and following reference pictures.
A video encoding program for causing a computer to calculate a predicted motion vector of the encoding target image based on motion vectors of two reference pictures before and after the encoding target image.
The video coding program according to claim 21, wherein the computer calculates a time-axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures as a predicted motion vector of the encoding target image.
The video encoding program according to claim 21, wherein a computer calculates a spatial axis predicted motion vector based on a motion vector of an image adjacent to the encoding target image in a screen as a prediction motion vector of the encoding target image.
Causing the computer to calculate a time-axis predicted motion vector based on a weighted sum of the motion vectors of the two preceding and following reference pictures when the feature amounts of the motion vectors of the two preceding and following reference pictures satisfy a predetermined value; The video encoding program according to claim 21, wherein when the value is not satisfied, a spatial axis predicted motion vector is calculated based on a motion vector of an image adjacent to the encoding target image in a screen.
The video code according to claim 24, wherein the computer uses one or more of an inner product, an outer product, and a norm of motion vectors of the two preceding and following reference pictures as a feature quantity of the motion vector of the two preceding and following reference pictures. Program.
A video decoding program for causing a computer to calculate a predicted motion vector of the decoding target image based on motion vectors of two reference pictures before and after the decoding target image.
27. The video decoding program according to claim 26, wherein a computer calculates a time-axis predicted motion vector based on a weighted sum of motion vectors of the two preceding and following reference pictures as a predicted motion vector of the decoding target image.
27. The video decoding program according to claim 26, wherein a computer calculates a spatial axis predicted motion vector based on a motion vector of an image adjacent to the decoding target image in a screen as a prediction motion vector of the decoding target image.
Causing the computer to calculate a time-axis predicted motion vector based on a weighted sum of the motion vectors of the two preceding and following reference pictures when the feature amounts of the motion vectors of the two preceding and following reference pictures satisfy a predetermined value; 27. The video decoding program according to claim 26, wherein a spatial axis predicted motion vector is calculated based on a motion vector of an image adjacent to the decoding target image within a screen when the value is not satisfied.
30. The video decoding according to claim 29, wherein the computer uses at least one of an inner product, an outer product, and a norm of motion vectors of the two preceding and following reference pictures as a feature quantity of the motion vector of the two preceding and following reference pictures. program.