JP2020109959A - Inter-prediction device, image coding device, image decoding device, and program - Google Patents

Abstract

To provide an inter-prediction device, an image coding device, an image decoding device, and a program that improve coding efficiency when performing inter-prediction.SOLUTION: An inter-prediction device (inter-prediction unit) includes an adjacent vector acquisition unit that acquires one or more first motion vectors applied to an adjacent decoded block group adjacent to an encoding target block obtained by dividing an original image, a motion vector correction unit that generates a corrected motion vector by correcting a second motion vector applied to the encoding target block using a first motion vector, and a prediction image generation unit that generates a prediction image of the encoding target block by performing inter prediction using the corrected motion vector.SELECTED DRAWING: Figure 3

Description

The present invention relates to an inter prediction device, an image encoding device, an image decoding device, and a program.

In the conventional video coding method represented by HEVC described in Non-Patent Document 1, an image coding apparatus divides an original image into blocks, performs prediction while switching between inter prediction and intra prediction for each block, It is configured to output a stream by performing orthogonal transformation, quantization, and entropy coding on a prediction residual representing an error of a predicted image obtained by prediction.

In inter prediction in such a video encoding method, when a motion vector applied to a block to be coded (Coding Unit: CU) is different from a motion vector applied to an adjacent CU adjacent to the target CU to be coded, The features of the prediction image of the target CU and the prediction image of the adjacent CU are significantly different.

In the example of the conventional video encoding method shown in FIG. 1, it is assumed that the object #1 is moving to the upper left and the object #2 is moving to the upper right. It is highly possible that the same motion vector (MV#3) as the motion of the object #1 is applied in the inter prediction for the adjacent CU#1 located on the left side of the encoding target CU that includes only the object #1. The encoding target CU is located at the object boundary including both the object #1 and the object #2, but since the object #1 occupies a large area, the motion vector (MV# of the adjacent CU#1 adjacent on the left side is It is highly possible that the same motion vector (MV#1) as 2) is applied to the coding target CU.

When the motion vectors of adjacent CUs are the same, the continuity of the motion of the boundary region of the CUs is high and the prediction accuracy is likely to be high. On the other hand, since the CU #2 adjacent to the upper side of the CU to be encoded in FIG. 1 includes many objects #2, there is a high possibility that a motion vector (MV#3) similar to the motion of the object #2 is applied. .. That is, there is a high possibility that the motion vector (MV#1) applied to the encoding target CU and the motion vector (MV#3) applied to the adjacent CU#1 are significantly different. When the motion vectors of such adjacent CUs greatly differ from each other, it is highly possible that at least one of these CUs includes an object boundary.

Recommendation ITU-TH. 265, (12/2016), "High efficiency video coding", International Telecommunication Union

The discontinuity of the motion vector at the object boundary causes a decrease in prediction accuracy, which causes a problem of a decrease in coding efficiency. In HEVC, in order to solve this problem, in a loop filter, block distortion is determined according to whether a difference between a motion vector applied to a coding target CU and a motion vector applied to an adjacent block is larger than a predetermined value. A method of controlling whether or not to apply a deblocking filter that removes is introduced.

However, in prediction (inter prediction), the same processing is applied regardless of the above discontinuity of motion vectors, so the amount of information required to transmit orthogonal transform coefficients increases in blocks near the object boundary. However, there is a problem that the coding efficiency is reduced.

Therefore, an object of the present invention is to provide an inter prediction device, an image coding device, an image decoding device, and a program capable of improving coding efficiency when performing inter prediction.

The inter prediction apparatus according to the first aspect performs inter prediction of an encoding target block obtained by dividing an original image. The inter prediction device includes an adjacent vector acquisition unit that acquires at least one first motion vector applied to an adjacent decoded block group that is adjacent to the current block, and a second motion that is applied to the current block. A motion vector correction unit that generates a corrected motion vector by correcting the vector using the first motion vector, and a predicted image of the encoding target block by performing the inter prediction using the corrected motion vector And a predicted image generation unit that generates

The gist of the image coding apparatus according to the second aspect is to include the inter prediction apparatus according to the first aspect.

The gist of the image decoding apparatus according to the third aspect is to include the inter prediction apparatus according to the first aspect.

The gist of the program according to the fourth aspect is to cause a computer to function as the inter prediction apparatus according to the first aspect.

According to the present invention, it is possible to provide an inter prediction device, an image coding device, an image decoding device, and a program capable of improving coding efficiency when performing inter prediction.

It is a figure for explaining background art. It is a figure which shows the structure of the image coding apparatus which concerns on 1st and 2nd embodiment. It is a figure which shows the structure of the inter prediction part (inter prediction apparatus) which concerns on 1st Embodiment. It is a figure which shows the operation example of the adjacent vector acquisition part which concerns on 1st and 2nd embodiment. It is a figure which shows the operation example of the motion vector correction part which concerns on 1st Embodiment. It is a figure which shows the structure of the image decoding apparatus which concerns on 1st and 2nd embodiment. It is a figure which shows operation|movement of the inter estimation part which concerns on 1st Embodiment. It is a figure which shows the example of a change of the inter prediction part which concerns on 1st Embodiment. It is a figure which shows the structure of the inter prediction part (inter prediction apparatus) which concerns on 2nd Embodiment. It is a figure which shows an example of the small area|region which concerns on 2nd Embodiment. It is a figure which shows the operation example of the motion vector correction part which concerns on 2nd Embodiment. It is a figure which shows operation|movement of the inter estimation part which concerns on 2nd Embodiment. It is a figure which shows the example of a change of the motion vector correction part which concerns on 2nd Embodiment. It is a figure which shows operation|movement of the example of a change of the motion vector correction part which concerns on 2nd Embodiment.

An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. The image encoding device and the image decoding device according to the embodiment respectively perform encoding and decoding of a moving image represented by MPEG. In the following description of the drawings, the same or similar reference numerals are given to the same or similar parts.

<1. First Embodiment>
<Image coding device>
First, the image coding apparatus according to this embodiment will be described. FIG. 2 is a diagram showing the configuration of the image encoding device 1 according to the present embodiment.

As shown in FIG. 2, the image encoding device 1 includes a block dividing unit 100, a subtracting unit 110, a transforming/quantizing unit 120, an entropy encoding unit 130, an inverse quantizing/inverse transforming unit 140, It has a combining unit 150, a memory 160, and a prediction unit 170.

The block division unit 100 divides an input image in units of frames (or pictures) forming a moving image into a plurality of blocks, and outputs the blocks obtained by the division to the subtraction unit 110. The block size is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the block is not limited to a square and may be a rectangle. The block is a unit in which the image encoding device 1 performs encoding and a unit in which the image decoding device performs decoding. Hereinafter, such a block will be referred to as a CU (Coding Unit). Further, a CU processed by the image encoding device 1 is referred to as an encoding target CU, and a CU processed by the image decoding device is referred to as a decoding target CU.

The subtraction unit 110 calculates a prediction residual that represents a difference (error) between the encoding target CU input from the block dividing unit 100 and the prediction image obtained by predicting the encoding target CU by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the prediction image from each pixel value of the CU, and outputs the calculated prediction residual to the conversion/quantization unit 120.

The transform/quantization unit 120 performs orthogonal transform processing and quantization processing in CU units. The conversion/quantization unit 120 includes a conversion unit 121 and a quantization unit 122.

The transform unit 121 performs an orthogonal transform process on the prediction residual input from the subtraction unit 110 to calculate an orthogonal transform coefficient, and outputs the calculated orthogonal transform coefficient to the quantization unit 122. The orthogonal transform refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-leeve transform (KLT), or the like.

The quantization unit 122 quantizes the orthogonal transformation coefficient input from the transformation unit 121 using a quantization parameter (Qp) and a quantization matrix, and the quantized orthogonal transformation coefficient is entropy coding unit 130 and inverse quantization/quantization unit. It outputs to the inverse conversion unit 140. The quantization parameter (Qp) is a parameter commonly applied to each orthogonal transform coefficient in the CU, and is a parameter that determines the roughness of quantization. The quantization matrix is a matrix having as elements the quantized values when quantizing each orthogonal transform coefficient.

The entropy coding unit 130 performs entropy coding on the orthogonal transform coefficient input from the quantization unit 122, performs data compression to generate a coded stream (bit stream), and encodes the coded stream into an image. Output to the outside of the device 1. Huffman coding, CABAC (Context-based Adaptive Binary Mathematical Coding; context adaptive binary arithmetic coding), or the like can be used for entropy coding. In addition, the entropy coding unit 130 receives the control information regarding the prediction from the prediction unit 170, and also performs the entropy coding of the input control information.

The inverse quantization/inverse transform unit 140 performs an inverse quantization process and an inverse orthogonal transform process in CU units. The inverse quantization/inverse transform unit 140 has an inverse quantization unit 141 and an inverse transform unit 142.

The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the dequantization unit 141 dequantizes the orthogonal transformation coefficient input from the quantization unit 122 using the quantization parameter (Qp) and the quantization matrix to restore the orthogonal transformation coefficient. , And outputs the restored orthogonal transform coefficient to the inverse transform unit 142.

The inverse transform unit 142 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121. For example, when the transform unit 121 performs the discrete cosine transform, the inverse transform unit 142 performs the inverse discrete cosine transform. The inverse transform unit 142 restores the prediction residual by performing the inverse orthogonal transform processing on the orthogonal transform coefficient input from the inverse quantization unit 141, and synthesizes the restored prediction residual that is the restored prediction residual. Output to.

The combining unit 150 combines the restored prediction residual input from the inverse transform unit 142 with the predicted image input from the prediction unit 170 on a pixel-by-pixel basis. The synthesizing unit 150 reconstructs (decodes) the coding target CU by adding each pixel value of the restored prediction residual and each pixel value of the predicted image, and outputs the decoded image of the decoded CU unit to the memory 160. Such a decoded image may be referred to as a reconstructed image.

The memory 160 stores the decoded image input from the combining unit 150. The memory 160 stores the decoded image in frame units. The memory 160 outputs the stored decoded image to the prediction unit 170. Further, the memory 160 stores the motion vector calculated by the inter prediction unit 172 for each CU. The memory 160 may be composed of a plurality of memories. Further, a loop filter may be provided between the synthesis unit 150 and the memory 160.

The prediction unit 170 performs prediction in CU units. The prediction unit 170 includes an intra prediction unit 171, an inter prediction unit 172, and a switching unit 173.

The intra prediction unit 171 generates an intra prediction image by referring to the decoded pixel value around the encoding target CU among the decoded images stored in the memory 160, and outputs the generated intra prediction image to the switching unit 173. To do. The intra prediction unit 171 selects the optimum intra prediction mode to be applied to the coding target CU from the plurality of intra prediction modes, and performs intra prediction using the selected intra prediction mode. The intra prediction unit 171 outputs control information regarding the selected intra prediction mode to the entropy coding unit 130.

The inter prediction unit 172 uses the decoded image stored in the memory 160 as a reference image, calculates a motion vector by a method such as block matching, predicts a coding target CU, and generates an inter prediction image. The inter prediction image is output to the switching unit 173. Further, the inter prediction unit 172 outputs the motion vector calculated for each CU to the memory 160, and stores the motion vector in the memory 160.

The inter prediction unit 172 selects an optimal inter prediction method from inter prediction using a plurality of reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), Inter prediction is performed using the selected inter prediction method. The inter prediction unit 172 outputs control information (motion vector and the like) regarding inter prediction to the entropy coding unit 130.

The switching unit 173 switches between the intra prediction image input from the intra prediction unit 171 and the inter prediction image input from the inter prediction unit 172, and outputs one of the prediction images to the subtraction unit 110 and the synthesis unit 150.

Next, the inter prediction unit 172 of the image encoding device 1 according to this embodiment will be described. FIG. 3 is a diagram showing the configuration of the inter prediction unit 172 of the image encoding device 1 according to this embodiment.

As illustrated in FIG. 3, the inter prediction unit 172 according to this embodiment includes an adjacent vector acquisition unit 172a, a motion vector calculation unit 172c, a motion vector correction unit 172d, and a predicted image generation unit 172e.

The adjacent vector acquisition unit 172a acquires, from the memory 160, the motion vector Vp applied to the inter prediction of each adjacent decoded CU located around (upper or left) the CU to be encoded, and acquires the motion vector Vp of the acquired motion vector Vp. The list is output to the motion vector correction unit 172d. Note that the motion vector includes, in addition to the vector values in the horizontal direction (x-axis direction) and the vertical direction (y-axis direction) of the motion vector, the temporal position of the reference picture (reference image) (for example, POC: Picture Order). Count, reference index in reference list, etc.) may be included.

As illustrated in FIG. 4, when intra prediction is applied to some of the adjacent decoded CUs located above or to the left of the CU to be encoded, the adjacent vector acquisition unit 172a The motion vector may be interpolated by substituting the motion vector applied to the CU adjacent to some adjacent decoded CUs, or the motion vector is calculated and interpolated by a weighted average of the surrounding available motion vectors. Alternatively, a motion vector defined in advance by the encoding method may be set. Interpolation is similarly performed when the CU to be encoded is at the screen edge.

The motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate the motion vector V by a method such as block matching, and outputs the calculated motion vector V to the motion vector correction unit 172d.

The motion vector correction unit 172d generates a corrected motion vector V′ by correcting the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a, and the correction is performed. The motion vector V′ is output to the predicted image generation unit 172e.

Specifically, the motion vector correction unit 172d corrects the motion vector V as shown in FIG. The motion vector correction unit 172d corrects the motion vector V by weighted averaging using the adjacent motion vector Vp acquired by the adjacent vector acquisition unit 172a and the motion vector V of the encoding target block.

First, the motion vector correction unit 172d converts the adjacent motion vector Vp into the adjacent motion vector V _L located on the left side of the encoding target CU and the adjacent motion vector V _T located above the encoding target CU as expressed by the following equation (1). Classify into.

V _L = [Vp[-1,0], Vp[-1,1], Vp[-1,2], …, Vp[-1,N-1]]
V _T = [Vp[0,-1], Vp[1,-1], Vp[2,-1], …, Vp[M-1,-1]]
(1)

Secondly, the motion vector correction unit 172d determines the weighting factors W _L , W _T , and W _V used to correct the motion vector V. The motion vector correction unit 172d calculates the weighting factors W _L , W _T , and W _V by, for example, the following equation (2).

W _L = 32 >> log2(height)
W _T = 32 >> log2(width)
W _V = 32-W _L -W _T
(2)

However, the width and the height respectively represent the width and the height of the encoding target CU. The weighting factors W _L and W _T may be determined according to the width or height of the block or the ratio thereof, and are not limited to the calculation method of the equation (2).

Thirdly, the motion vector correction unit 172d corrects the uncorrected motion vector V[x, y] based on the weighting factors W _L and W _T and the adjacent motion vectors V _L and V _T. The corrected motion vector V′ is calculated by the equation (3).

V'= W _L・median(V _L )+W _T・median(V _T )+W _V・V)
(3)

The predicted image generation unit 172e generates a predicted image of the coding target CU by performing inter prediction of the coding target CU using the corrected motion vector V′ generated by the motion vector correction unit 172d, and the generated prediction The image (inter prediction image) is output to the switching unit 173.

The prediction image is input to the subtraction unit 110 via the switching unit 173, and the subtraction unit 110 outputs the prediction residual representing the difference between the CU to be encoded and the prediction image to the conversion/quantization unit 120. .. The transform/quantization unit 120 generates a quantized orthogonal transform coefficient from the prediction residual, and outputs the generated orthogonal transform coefficient to the entropy coding unit 130.

<Image decoding device>
Next, the image decoding apparatus according to this embodiment will be described. However, with regard to the same operation as that of the image encoding device 1 described above, redundant description will be omitted. FIG. 6 is a diagram showing the configuration of the image decoding device 2 according to the present embodiment.

As illustrated in FIG. 6, the image decoding device 2 includes an entropy decoding unit 200, an inverse quantization/inverse conversion unit 210, a combining unit 220, a memory 230, and a prediction unit 240.

The entropy decoding unit 200 decodes the coded stream generated by the image coding device 1, and outputs the quantized orthogonal transform coefficient to the inverse quantization/inverse transform unit 210. Further, the entropy decoding unit 200 acquires control information regarding prediction (intra prediction and inter prediction), and outputs the acquired control information to the prediction unit 240.

The inverse quantization/inverse transform unit 210 performs an inverse quantization process and an inverse orthogonal transform process in CU units. The inverse quantization/inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the image encoding device 1. The inverse quantization unit 211 inversely quantizes the quantized orthogonal transformation coefficient input from the entropy decoding unit 200 using the quantization parameter (Qp) and the quantization matrix to obtain the orthogonal transformation coefficient of the decoding target CU. The restored orthogonal transform coefficient is output to the inverse transform unit 212.

The inverse transform unit 212 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 restores the prediction residual by performing the inverse orthogonal transform process on the orthogonal transform coefficient input from the inverse quantization unit 211, and synthesizes the restored prediction residual (restored prediction residual). Output to.

The combining unit 220 reconstructs (decodes) the original CU by combining the prediction residual input from the inverse transform unit 212 and the prediction image input from the prediction unit 240 in pixel units, and in CU units. The decoded image of is output to the memory 230.

The memory 230 stores the decoded image input from the combining unit 220. The memory 230 stores the decoded image in frame units. Further, the memory 230 stores the motion vector calculated by the inter prediction unit 242 for each CU. The memory 230 outputs the decoded image in frame units to the outside of the image decoding device 2. The memory 230 may be composed of a plurality of memories. A loop filter may be provided between the combining unit 220 and the memory 230.

The prediction unit 240 performs prediction in CU units. The prediction unit 240 includes an intra prediction unit 241, an inter prediction unit 242, and a switching unit 243.

The intra prediction unit 241 refers to the decoded image stored in the memory 230, generates the intra-predicted image by predicting the decoding target CU by the intra prediction according to the control information input from the entropy decoding unit 200, and generates the intra-predicted image. The intra prediction image is output to the switching unit 243.

The inter prediction unit 242 predicts the decoding target CU by inter prediction using the decoded image stored in the memory 230 as a reference image. The inter prediction unit 242 generates an inter prediction image by performing inter prediction according to the control information input from the entropy decoding unit 200, and outputs the generated inter prediction image to the switching unit 243. In the present embodiment, the inter prediction unit 242 has the same configuration as the inter prediction unit 172 shown in FIG.

The switching unit 243 switches between the intra-prediction image input from the intra-prediction unit 241 and the inter-prediction image input from the inter-prediction unit 242, and outputs one of the prediction images to the synthesis unit 220.

<Operation of inter prediction unit>
FIG. 7: is a figure which shows operation|movement of the inter estimation part 172 of the image coding apparatus 1 which concerns on this embodiment. The inter prediction unit 242 of the image decoding device 2 also operates similarly to the inter prediction unit 172 of the image encoding device 1. However, in the operation of the inter prediction unit 242, the “encoding target CU” in the operation of the inter prediction unit 172 is replaced with the “decoding target CU”.

As shown in FIG. 7, in step S1, the adjacent vector acquisition unit 172a causes the motion vector applied from the memory 160 to the inter prediction of each adjacent decoded CU located around (upper or left) the CU to be encoded. Get Vp.

In step S2, the motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate the motion vector V by a method such as block matching.

In step S3, the motion vector correction unit 172d corrects the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a to obtain the corrected motion vector V′. To generate.

In step S4, the predicted image generation unit 172e performs inter prediction using the corrected motion vector V′ to generate a predicted image of the CU to be coded.

As described above, the inter prediction unit 172 according to the present embodiment can reduce the discontinuity of the motion vector by using the motion vector corrected using the adjacent motion vector Vp, and thus the prediction residual Can be suppressed from increasing in blocks near the object boundary, and the coding efficiency can be improved.

<Example of changing the inter prediction unit>
FIG. 8 is a diagram illustrating a modification example of the inter prediction unit 172 according to the first embodiment.

As illustrated in FIG. 8, the inter prediction unit 172 includes a continuity evaluation unit 172f and a correction adjacent vector determination unit 172g in addition to the configuration according to the above-described embodiment.

The continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the adjacent motion vector Vp to determine the boundary region between each decoded block included in the adjacent decoded block group and the encoding target CU. Evaluate the continuity of the movement of.

The continuity evaluation unit 172f acquires, from the memory 160, the motion vector Vp applied to the decoded one or more adjacent CUs adjacent to the encoding target CU. Then, the continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the motion vector Vp applied to the adjacent decoded CU group, and thereby the adjacent decoded CU group is encoded. The continuity of motion with the CU is evaluated, and the evaluation result is output to the correction adjacent vector determination unit 172g.

For example, if the L1 norm representing the difference (distance) between the motion vector V applied to the encoding target CU and the motion vector Vp of the decoded CU is less than a predetermined threshold value, the continuity evaluating unit 172f performs continuity. It is evaluated as highly effective. On the other hand, if the L1 norm is equal to or more than the threshold value, it is evaluated that the continuity is low. However, the index value of continuity is not limited to the L1 norm, and other index values may be used.

Note that the motion vector includes vector values in the X-axis and Y-axis directions, as well as flags indicating the decoded frame to be referred to (for example, inter_pred_idc, ref_idx in HEVC). When evaluating the continuity of the motion vector with the adjacent block, the continuity evaluating unit 172f temporally separates the frame containing the encoding target CU (current frame) from the decoded frame referenced by the encoding target CU. The vector value of the motion vector applied to the adjacent decoded CU is scaled according to (POC distance) and the POC distance between the current frame and the decoded frame that is the reference destination of the adjacent decoded CU, and then the motion vector is continuously scaled. You may evaluate sex.

The correction adjacent vector determination unit 172g determines the motion vector used for the correction of the motion vector V based on the motion vector Vp, based on the motion continuity evaluated by the continuity evaluation unit 172f.

For example, the correction adjacent vector determination unit 172g
V _L = [Vp[-1,0], Vp[-1,1], Vp[-1,2], …, Vp[-1,N-1]]
V _T = [Vp[0,-1], Vp[1,-1], Vp[2,-1], …, Vp[M-1,-1]]
When
_{V 'L = [v L ∈V} L | || v L - V || 2 <Th]
_{V 'T = [v T ∈V} T | || v T - V || 2 <Th]
In this way, the motion vector used for correcting the motion vector V is determined.

That, V _'L Of left adjacent vectors Vp of the encoding target CU, a set composed of vectors of L2 norm is equal to or smaller than the threshold Th of the motion vector V, V' _T above the encoding target CU Of the adjacent vectors Vp of L2 norm with respect to the motion vector V is a set constituted by vectors having a threshold value Th or less.

The motion vector correction unit 172d uses these V′ _L and V′ _T (third motion vector) to calculate the corrected motion vector V′ according to the following equation (4).

V'= W _L・median(V' _L )+W _T・median(V' _T )+W _V・V)
(4)

<2. Second Embodiment>
The differences between the second embodiment and the first embodiment will be mainly described.

FIG. 9 is a diagram showing the configuration of the inter prediction unit 172 of the image encoding device 1 according to the second embodiment.

As shown in FIG. 9, the inter prediction unit 172 according to the second embodiment has the configuration (adjacent vector acquisition unit 172a, motion vector calculation unit 172c, motion vector correction unit 172d, predicted image generation unit) described in the first embodiment. In addition to 172e), it has a small area dividing section 172b.

The small area dividing unit 172b divides the coding target CU into a plurality of small areas having a predetermined size, and outputs information on each small area obtained by the division to the motion vector correcting unit 172d. Such small areas may be referred to as sub-blocks.

FIG. 10 is a diagram showing an example of a small area. As shown in FIG. 10A, the small areas may have the same size, and each small area may be an area of 4×4 pixels. Alternatively, if common processing is defined in advance between the image decoding device and the image decoding device, the size of each small area may be different as shown in FIG. For example, the area near the boundary of the CU to be encoded may be finely divided, and the vicinity of the center may be roughly divided into small areas.

The motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate the motion vector V by a method such as block matching, and outputs the calculated motion vector V to the motion vector correction unit 172d. In the present embodiment, the motion vector calculation unit 172c may calculate one motion vector V for the encoding target CU, or for each small region obtained by dividing the encoding target CU by the small region dividing unit 172b. The motion vector V may be calculated.

The motion vector correction unit 172d generates a corrected motion vector V′ by correcting the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a, and the correction is performed. The motion vector V′ is output to the predicted image generation unit 172e.

In the present embodiment, the motion vector correction unit 172d corrects the motion vector V for each small area, as shown in FIG. The motion vector correction unit 172d weights the motion vector Vp of the adjacent decoded CU corresponding to the position of the small area using the weight determined according to the position of the small area, and uses the weighted motion vector Vp, The motion vector V applied to the small area is corrected.

For example, the motion vector correction unit 172d substitutes the uncorrected motion vector V into the motion vector array V[x, y] for each small region of the CU to be encoded. However, when the uncorrected motion vector V is a motion vector that differs for each small block in the encoding target CU, a motion vector array that is different for each small region based on the motion vector set for each small block. Substitute in V[x, y]. When the size of the small block is different from the size of the small area, the size is substituted by a method defined in advance. For example, when the size of the small block is larger than the small area, the motion vector of the small block is substituted into the motion vector array V[x, y] of all the small areas included in the small block. On the other hand, when the size of the small block is smaller than the small area, the vector calculated by the average or median of the motion vectors of all the small blocks included in the small area is used as the motion vector V[x, y of the small area. ]]

First, the motion vector correction unit 172d determines the weighting factors W _L , W _T , W _LT , W _V according to the position of each small area. The motion vector correction unit 172d calculates the weighting factors W _L , W _T , W _LT , and W _V by, for example, the following equation (5).

（５）

(5)

However, x and y represent the position of the small area, and width and height represent the width and height of the encoding target CU, respectively. Note that the weighting factors W _L , W _T , W _LT , and W _V may be determined according to the position of the small area, the width and height of the CU, and are not limited to the calculation method of Expression (5).

Secondly, the motion vector correction unit 172d corrects the uncorrected motion vector V[x, y] based on the weighting factors W _L , W _T , W _LT , W _V and the adjacent motion vector Vp. The corrected motion vector V′[x, y] is calculated by the following equation (6).

（６）

(6)

The predicted image generation unit 172e uses the corrected motion vector V′[x, y] generated by the motion vector correction unit 172d to perform inter prediction for each small area of the coding target CU, thereby The predicted image is generated, and the generated predicted image (inter predicted image) is output to the switching unit 173.

Specifically, the predicted image generation unit 172e generates a predicted image for each small area by performing inter prediction for each small area using the motion vector V'[x, y] for each small area, and The predicted image of the encoding target CU is generated by synthesizing the predicted image for each region.

The prediction image is input to the subtraction unit 110 via the switching unit 173, and the subtraction unit 110 outputs the prediction residual representing the difference between the CU to be encoded and the prediction image to the conversion/quantization unit 120. .. The transform/quantization unit 120 generates a quantized orthogonal transform coefficient from the prediction residual, and outputs the generated orthogonal transform coefficient to the entropy coding unit 130. Note that the entropy coding unit 130 may code one motion vector for each CU to be coded, without coding (signaling) the motion vector for each small area.

Further, in the present embodiment, the inter prediction unit 242 of the image decoding device 2 has the same configuration as the inter prediction unit 172 shown in FIG. 9.

<Operation of inter prediction unit>
FIG. 12 is a diagram showing the operation of the inter prediction unit 172 of the image encoding device 1 according to this embodiment. The inter prediction unit 242 of the image decoding device 2 also operates similarly to the inter prediction unit 172 of the image encoding device 1. However, in the operation of the inter prediction unit 242, the “encoding target CU” in the operation of the inter prediction unit 172 is replaced with the “decoding target CU”.

As shown in FIG. 12, in step S11, the adjacent vector acquisition unit 172a uses the memory 160 to apply the motion vector applied to the inter prediction of each adjacent decoded CU located around (upper or left) the CU to be encoded. Get Vp.

In step S12, the small area dividing unit 172b divides the encoding target CU into a plurality of small areas having a predetermined size.

In step S13, the motion vector calculation unit 172c calculates the motion vector V by a method such as block matching using the decoded image stored in the memory 160 as a reference image.

In step S14, the motion vector correction unit 172d corrects the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a to obtain the corrected motion vector V′. To generate.

In step S15, the predicted image generation unit 172e performs inter prediction for each small area using the corrected motion vector V', and generates a predicted image of the encoding target CU.

As described above, the inter prediction unit 172 according to the present embodiment can perform fine inter prediction using the motion vector corrected using the adjacent motion vector Vp for each small area in the decoding target CU. Therefore, it is possible to prevent the prediction residual from increasing in blocks near the object boundary, and improve the coding efficiency.

<Example of changing the inter prediction unit>
FIG. 13 is a diagram illustrating a modification example of the inter prediction unit 172 according to the second embodiment.

As illustrated in FIG. 13, the inter prediction unit 172 according to the present modification includes a continuity evaluation unit 172f and a corrected small area determination unit 172h in addition to the configuration according to the above-described embodiment.

The continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the adjacent motion vector Vp to determine the boundary region between each decoded block included in the adjacent decoded block group and the encoding target CU. Evaluate the continuity of the movement of.

The continuity evaluation unit 172f acquires, from the memory 160, the motion vector Vp applied to the decoded one or more adjacent CUs adjacent to the encoding target CU. Then, the continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the motion vector Vp applied to the adjacent decoded CU group, and thereby the adjacent decoded CU group is encoded. The continuity of the movement with the CU is evaluated, and the evaluation result is output to the correction small area determination unit 172h.

For example, as illustrated in FIG. 14A, the continuity evaluation unit 172f previously sets an L1 norm representing the difference (distance) between the motion vector V applied to the encoding target CU and the motion vector Vp of the decoded CU. If it is less than the specified threshold, it is evaluated that the continuity is high. On the other hand, when the L1 norm is equal to or more than the threshold value, it is evaluated that the continuity is low. However, the index value of continuity is not limited to the L1 norm, and other index values may be used.

Note that the motion vector includes vector values in the X-axis and Y-axis directions, as well as flags indicating the decoded frame to be referred to (for example, inter_pred_idc, ref_idx in HEVC). When evaluating the continuity of the motion vector with the adjacent block, the continuity evaluating unit 172f temporally separates the frame containing the encoding target CU (current frame) from the decoded frame referenced by the encoding target CU. The vector value of the motion vector applied to the adjacent decoded CU is scaled according to (POC distance) and the POC distance between the current frame and the decoded frame that is the reference destination of the adjacent decoded CU, and then the motion vector is continuously scaled. You may evaluate sex.

The correction small area determination unit 172h determines at least one small area in which the motion vector V should be corrected based on the continuity of the motion evaluated by the continuity evaluation unit 172f. For example, as illustrated in FIG. 14B, the correction small area determination unit 172h determines only each small area corresponding to an area having low motion continuity as a small area in which the motion vector V should be corrected. However, the correction small area determination unit 172h may determine all the small areas of the encoding target CU as the small areas in which the motion vector V should be corrected.

In the present modification, the motion vector correction unit 172d generates a corrected motion vector V'for the small area determined by the corrected small area determination unit 172h. Further, the motion vector correction unit 172d determines a weighting coefficient according to the position of the small area and the continuity of the motion evaluated by the continuity evaluation unit 172f, and uses the determined weighting coefficient to determine the weighting coefficient of the small area. The adjacent motion vector Vp corresponding to the position may be weighted.

The motion vector correction unit 172d may be set so as to increase the weight of the adjacent motion vector Vp corresponding to the region with low motion continuity. For example, by introducing the value c _x, c _y corresponding to continuity of motion for formula (5), calculates weight coefficients _{W L, W T, W LT} , the W _V by the following equation (7).

（７）

(7)

As described above, according to the present modification, the coding efficiency in the case of performing inter prediction by correcting the motion vector V of the coding target CU (small area) in consideration of the continuity of motion in the block boundary area. Can be further improved.

<3. Other Embodiments>
The program may be provided by a program that causes a computer to execute each process performed by the image encoding device 1 and a program that causes a computer to perform each process performed by the image decoding device 2. Further, the program may be recorded in a computer-readable medium. A computer readable medium can be used to install the program on a computer. Here, the computer-readable medium in which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

Further, circuits for executing the processes performed by the image coding apparatus 1 may be integrated to configure the image coding apparatus 1 as a semiconductor integrated circuit (chip set, SoC). Similarly, a circuit that executes each process performed by the image decoding device 2 may be integrated to configure the image decoding device 2 as a semiconductor integrated circuit (chip set, SoC).

Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the spirit of the invention.

1: image encoding device 2: image decoding device 100: block division unit 110: subtraction unit 120: conversion/quantization unit 121: conversion unit 122: quantization unit 130: entropy encoding unit 140: dequantization/inverse conversion Unit 141: inverse quantization unit 142: inverse transform unit 150: synthesis unit 160: memory 170: prediction unit 171: intra prediction unit 172: inter prediction unit 172a: adjacent vector acquisition unit 172b: small area division unit 172c: motion vector calculation Unit 172d: motion vector correction unit 172e: predicted image generation unit 172f: continuity evaluation unit 172g: correction adjacent vector determination unit 172h: corrected small region determination unit 173: switching unit 200: entropy decoding unit 210: dequantization/inverse Transform unit 211: Inverse quantization unit 212: Inverse transform unit 220: Compositing unit 230: Memory 240: Prediction unit 241: Intra prediction unit 242: Inter prediction unit 243: Switching unit

Claims (10)

An inter-prediction device that performs inter-prediction of an encoding target block obtained by dividing an original image,
An adjacent vector acquisition unit that acquires one or more first motion vectors applied to an adjacent decoded block group adjacent to the target block;
A motion vector correction unit that generates a corrected motion vector by correcting the second motion vector applied to the encoding target block using the first motion vector;
An inter prediction device, comprising: a prediction image generation unit that generates a prediction image of the encoding target block by performing the inter prediction using the corrected motion vector. The motion vector correction unit,
Weighting the first motion vector using a weight determined according to the width and height of the encoding target block;
The inter prediction apparatus according to claim 1, wherein the second motion vector is corrected using the weighted first motion vector. By comparing the second motion vector applied to the encoding target block with the one or more first motion vectors, each decoded block included in the adjacent decoded block group and the encoding target block A continuity evaluation unit that evaluates the continuity of the movement of the boundary area,
A correction adjoining vector determination unit that determines a third motion vector used for correction of the second motion vector based on the evaluated motion continuity based on the first motion vector,
The inter prediction device according to claim 1 or 2, wherein the motion vector correction unit generates the corrected motion vector using the third motion vector determined by the correction adjacent vector determination unit. Further comprising a small area dividing unit that divides the encoding target block into a plurality of small areas,
The motion vector correction unit generates a corrected motion vector by correcting the second motion vector applied to each small area of the encoding target block using the first motion vector corresponding to the position of the small area. The inter prediction apparatus according to claim 1, wherein: The motion vector correction unit,
Using the weight determined according to the position of the small area, weighting the first motion vector corresponding to the position of the small area,
The inter prediction apparatus according to claim 4, wherein the weighted first motion vector is used to correct the second motion vector applied to the small region. By comparing the second motion vector applied to the encoding target block with the one or more first motion vectors, each decoded block included in the adjacent decoded block group and the encoding target block A continuity evaluation unit that evaluates the continuity of the movement of the boundary area,
A correction small area determination unit that determines at least one small area in which the second motion vector is to be corrected based on the evaluated continuity of the movement,
The inter prediction device according to claim 5, wherein the motion vector correction unit generates the corrected motion vector for the small area determined by the corrected small area determination unit. The motion vector correction unit may weight the first motion vector corresponding to the position of the small area using a weight determined according to the position of the small area and the continuity of the motion. The inter prediction apparatus according to claim 6. An image coding device comprising the inter prediction device according to claim 1. An image decoding apparatus comprising the inter prediction apparatus according to any one of claims 1 to 7. A program that causes a computer to function as the inter prediction device according to claim 1.

Images