JP7449690B2 - Inter prediction device, image encoding device, image decoding device, and program - Google Patents

Description

The present invention relates to an inter prediction device, an image encoding device, an image decoding device, and a program.

In conventional video encoding methods represented by HEVC described in Non-Patent Document 1, an image encoding device divides an original image into blocks, performs prediction while switching between inter prediction and intra prediction for each block, and performs prediction while switching between inter prediction and intra prediction for each block. It is configured to output a stream by performing orthogonal transformation, quantization, and entropy coding on a prediction residual representing an error in a predicted image obtained by prediction.

In inter prediction in such a video encoding method, if a motion vector applied to a block to be encoded (Coding Unit: CU) is different from a motion vector applied to an adjacent CU adjacent to the CU to be encoded, the code is The predicted image of the CU to be converted and the predicted image of the adjacent CU have significantly different characteristics.

In the example of the conventional video encoding method shown in FIG. 1, it is assumed that object #1 is moving to the upper left and object #2 is moving to the upper right. In inter prediction for adjacent CU #1 located on the left side of the encoding target CU that includes only object #1, it is highly likely that a motion vector (MV #3) similar to the motion of object #1 will be applied. The encoding target CU is located at an object boundary that includes both object #1 and object #2, but since object #1 occupies a large area, the motion vector (MV # The motion vector (MV#1) similar to 2) is likely to be applied to the encoding target CU.

When the motion vectors of adjacent CUs are similar, the continuity of motion in the boundary area of the CUs is high, and the prediction accuracy is likely to be high. On the other hand, since CU #2 adjacent to the upper side of the encoding target CU in FIG. 1 includes many objects #2, there is a high possibility that a motion vector (MV #3) similar to the motion of object #2 will be applied. . That is, there is a high possibility that the motion vector (MV#1) applied to the encoding target CU and the motion vector (MV#3) applied to the adjacent CU#1 are significantly different. If the motion vectors of such adjacent CUs are significantly different, there is a high possibility that at least one of these CUs includes an object boundary.

Recommendation ITU-T H. 265, (12/2016), "High efficiency video coding", International Telecommunication Union

Discontinuity of motion vectors at object boundaries causes a decrease in prediction accuracy, which poses a problem in that encoding efficiency decreases. In HEVC, in order to solve this problem, the loop filter calculates block distortion depending on whether the difference between the motion vector applied to the CU to be encoded and the motion vector applied to the adjacent block is larger than a predefined value. A method has been introduced to control whether or not to apply a deblocking filter that removes the deblocking filter.

However, in prediction (inter prediction), the same processing is applied regardless of the above-mentioned discontinuity of motion vectors, so the amount of information required to transmit orthogonal transform coefficients increases in blocks near object boundaries. , there is a problem that encoding efficiency decreases.

Therefore, an object of the present invention is to provide an inter prediction device, an image coding device, an image decoding device, and a program that can improve coding efficiency when performing inter prediction.

The inter prediction device according to the first aspect performs inter prediction of a block to be coded obtained by dividing an original image. The inter prediction device includes an adjacent vector acquisition unit that acquires one or more first motion vectors applied to a group of adjacent decoded blocks adjacent to the current block to be encoded, and a second motion vector applied to the current block to be encoded. a motion vector correction unit that generates a corrected motion vector by correcting a vector using the first motion vector; and a predicted image of the block to be coded by performing the inter prediction using the corrected motion vector. and a predicted image generation unit that generates a predicted image.

The gist of the image encoding device according to the second aspect is that it includes the inter prediction device according to the first aspect.

The gist of the image decoding device according to the third aspect is that it includes the inter prediction device according to the first aspect.

The gist of the program according to the fourth aspect is to cause a computer to function as the inter prediction device according to the first aspect.

According to the present invention, it is possible to provide an inter prediction device, an image coding device, an image decoding device, and a program that can improve coding efficiency when performing inter prediction.

FIG. 2 is a diagram for explaining background technology. FIG. 1 is a diagram showing the configuration of an image encoding device according to first and second embodiments. It is a diagram showing the configuration of an inter prediction unit (inter prediction device) according to the first embodiment. FIG. 6 is a diagram illustrating an example of the operation of the adjacent vector acquisition unit according to the first and second embodiments. FIG. 3 is a diagram showing an example of the operation of the motion vector correction section according to the first embodiment. FIG. 1 is a diagram showing the configuration of an image decoding device according to first and second embodiments. It is a figure showing operation of an inter prediction part concerning a 1st embodiment. It is a figure which shows the example of a change of the inter prediction part based on 1st Embodiment. It is a figure showing the composition of the inter prediction part (inter prediction device) concerning a 2nd embodiment. It is a figure showing an example of a small area concerning a 2nd embodiment. FIG. 7 is a diagram illustrating an example of the operation of a motion vector correction unit according to a second embodiment. It is a figure which shows the operation|movement of the inter prediction part based on 2nd Embodiment. FIG. 7 is a diagram showing a modification example of the motion vector correction unit according to the second embodiment. FIG. 7 is a diagram showing the operation of a modified example of the motion vector correction section according to the second embodiment.

An image encoding device and an image decoding device according to embodiments will be described with reference to the drawings. The image encoding device and the image decoding device according to the embodiment encode and decode moving images represented by MPEG, respectively. In the description of the drawings below, the same or similar parts are designated by the same or similar symbols.

<1. First embodiment>
<Image encoding device>
First, an image encoding device according to this embodiment will be explained. FIG. 2 is a diagram showing the configuration of the image encoding device 1 according to this embodiment.

As shown in FIG. 2, the image encoding device 1 includes a block division section 100, a subtraction section 110, a transformation/quantization section 120, an entropy encoding section 130, an inverse quantization/inverse transformation section 140, It has a synthesis section 150, a memory 160, and a prediction section 170.

The block dividing unit 100 divides an input image in units of frames (or pictures) constituting a moving image into a plurality of blocks, and outputs the blocks obtained by the division to the subtracting unit 110. The size of the block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the block is not limited to square, but may be rectangular. A block is a unit for encoding by the image encoding device 1 and a unit for decoding by the image decoding device. In the following, such a block will be referred to as a CU (Coding Unit). Moreover, the CU processed by the image encoding device 1 is called a coding target CU, and the CU processed by the image decoding device is called a decoding target CU.

The subtraction unit 110 calculates a prediction residual representing the difference (error) between the encoding target CU input from the block dividing unit 100 and the predicted image obtained by predicting the encoding target CU by the prediction unit 170. Specifically, the subtraction unit 110 calculates a prediction residual by subtracting each pixel value of the predicted image from each pixel value of the CU, and outputs the calculated prediction residual to the conversion/quantization unit 120.

The transformation/quantization unit 120 performs orthogonal transformation processing and quantization processing on a CU basis. The conversion/quantization unit 120 includes a conversion unit 121 and a quantization unit 122.

The transform unit 121 performs orthogonal transform processing on the prediction residual input from the subtracter 110 to calculate orthogonal transform coefficients, and outputs the calculated orthogonal transform coefficients to the quantizer 122 . The orthogonal transformation refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), or the like.

The quantization unit 122 quantizes the orthogonal transform coefficients input from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix, and sends the quantized orthogonal transform coefficients to an entropy encoder 130 and inverse quantization/dequantization. It is output to the inverse transformer 140. Note that the quantization parameter (Qp) is a parameter that is commonly applied to each orthogonal transform coefficient within the CU, and is a parameter that determines the roughness of quantization. The quantization matrix is a matrix having as elements quantization values when quantizing each orthogonal transform coefficient.

The entropy encoding unit 130 performs entropy encoding on the orthogonal transform coefficients input from the quantization unit 122, performs data compression to generate an encoded stream (bitstream), and performs image encoding on the encoded stream. Output to the outside of the device 1. For entropy encoding, a Huffman code, CABAC (Context-based Adaptive Binary Arithmetic Coding), or the like can be used. Note that the entropy encoding unit 130 receives control information related to prediction from the prediction unit 170, and also performs entropy encoding of the input control information.

The inverse quantization/inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing in units of CUs. The inverse quantization/inverse transformation section 140 includes an inverse quantization section 141 and an inverse transformation section 142.

The dequantization unit 141 performs dequantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the dequantization unit 141 restores orthogonal transformation coefficients by dequantizing the orthogonal transformation coefficients input from the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix. , the restored orthogonal transform coefficients are output to the inverse transform unit 142.

The inverse transform unit 142 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs inverse orthogonal transform processing on the orthogonal transform coefficients input from the inverse quantization unit 141 to restore the prediction residual, and combines the restored prediction residual, which is the restored prediction residual, with the synthesis unit 150 Output to.

The combination unit 150 combines the restored prediction residual input from the inverse transformation unit 142 with the predicted image input from the prediction unit 170 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the restored prediction residual and each pixel value of the predicted image to reconstruct (decode) the encoding target CU, and outputs the decoded image in units of CU to the memory 160. Such a decoded image is sometimes referred to as a reconstructed image.

Memory 160 stores the decoded image input from synthesis section 150. Memory 160 stores decoded images in units of frames. The memory 160 outputs the stored decoded image to the prediction unit 170. Furthermore, the memory 160 stores the motion vector calculated by the inter prediction unit 172 for each CU. Note that the memory 160 may be composed of a plurality of memories. Further, a loop filter may be provided between the synthesis section 150 and the memory 160.

The prediction unit 170 performs prediction on a CU basis. The prediction unit 170 includes an intra prediction unit 171, an inter prediction unit 172, and a switching unit 173.

The intra prediction unit 171 generates an intra prediction image by referring to the decoded pixel values around the encoding target CU among the decoded images stored in the memory 160, and outputs the generated intra prediction image to the switching unit 173. do. The intra prediction unit 171 selects the optimal intra prediction mode to be applied to the encoding target CU from among the plurality of intra prediction modes, and performs intra prediction using the selected intra prediction mode. Intra prediction section 171 outputs control information regarding the selected intra prediction mode to entropy encoding section 130.

The inter prediction unit 172 uses the decoded image stored in the memory 160 as a reference image, calculates a motion vector by a method such as block matching, predicts the CU to be encoded, and generates an inter predicted image. The inter predicted image is output to the switching unit 173. Further, the inter prediction unit 172 outputs the motion vector calculated for each CU to the memory 160, and causes the memory 160 to store the motion vector.

The inter prediction unit 172 selects the optimal inter prediction method from among inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), Perform inter prediction using the selected inter prediction method. The inter prediction unit 172 outputs control information (motion vectors, etc.) related to inter prediction to the entropy encoding unit 130.

The switching unit 173 switches between the intra-predicted image input from the intra-predicting unit 171 and the inter-predicted image input from the inter-predicting unit 172, and outputs one of the predicted images to the subtracting unit 110 and the combining unit 150.

Next, the inter prediction unit 172 of the image encoding device 1 according to this embodiment will be explained. FIG. 3 is a diagram showing the configuration of the inter prediction unit 172 of the image encoding device 1 according to the present embodiment.

As shown in FIG. 3, the inter prediction unit 172 according to this embodiment includes an adjacent vector acquisition unit 172a, a motion vector calculation unit 172c, a motion vector correction unit 172d, and a predicted image generation unit 172e.

The adjacent vector acquisition unit 172a acquires from the memory 160 the motion vectors Vp applied to the inter prediction of each adjacent decoded CU located around (above or to the left of) the encoding target CU, and The list is output to the motion vector correction section 172d. Note that the motion vector includes the horizontal direction (x-axis direction) and vertical direction (y-axis direction) vector values of the motion vector, as well as the temporal position of a reference picture (reference image) (for example, POC: Picture Order Count, reference index in the reference list), etc.

As shown in FIG. 4, when intra prediction is applied to some adjacent decoded CUs among the adjacent decoded CUs located above or to the left of the encoding target CU, the adjacent vector acquisition unit 172a A motion vector may be interpolated by substituting a motion vector applied to a CU close to some adjacent decoded CUs, or a motion vector may be calculated by a weighted average of surrounding available motion vectors and interpolated. Alternatively, a motion vector predefined by the encoding method may be set. Interpolation is performed in the same manner when the encoding target CU is at the edge of the screen.

The motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a method such as block matching, and outputs the calculated motion vector V to the motion vector correction unit 172d.

The motion vector correction unit 172d generates a corrected motion vector V′ by correcting the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a, and performs the correction. The motion vector V' is output to the predicted image generation section 172e.

Specifically, the motion vector correction unit 172d corrects the motion vector V as shown in FIG. The motion vector correction unit 172d corrects the motion vector V by weighted averaging using the adjacent motion vector Vp acquired by the adjacent vector acquisition unit 172a and the motion vector V of the current block to be encoded.

First, the motion vector correction unit 172d converts the adjacent motion vector Vp into an adjacent motion vector V _L located on the left side of the encoding target CU and an adjacent motion vector V _T located above the encoding target CU as shown in equation (1) below. Classify into.

V _L = [Vp[-1,0], Vp[-1,1], Vp[-1,2], …, Vp[-1,N-1]]
V _T = [Vp[0,-1], Vp[1,-1], Vp[2,-1], …, Vp[M-1,-1]]
(1)

Second, the motion vector correction unit 172d determines weighting coefficients W _L , W _T , and W _V used for correcting the motion vector V. The motion vector correction unit 172d calculates the weighting coefficients W _L , W _T , and W _V using the following equation (2), for example.

W _L = 32 >> log2(height)
W _T = 32 >> log2(width)
W _V = 32 - W _L - W _T
(2)

However, width and height represent the width and height of the encoding target CU, respectively. Note that the weighting coefficients W _L and W _T may be determined according to the width and height of the block, or the ratio thereof, and are not limited to the calculation method of equation (2).

Thirdly, the motion vector correction unit 172d corrects the motion vector V[x,y] before correction based on the weighting coefficients W _L , W _T and the adjacent motion vectors V _L and V _T , for example, as follows. The corrected motion vector V' is calculated using equation (3).

V'= W _L・median(V _L ) + W _T・median(V _T ) + W _V・V)
(3)

The predicted image generation unit 172e generates a predicted image of the CU to be encoded by performing inter prediction of the CU to be encoded using the corrected motion vector V' generated by the motion vector correction unit 172d, and uses the generated prediction. The image (inter predicted image) is output to the switching unit 173.

Note that this predicted image is input to the subtraction unit 110 via the switching unit 173, and the subtraction unit 110 outputs a prediction residual representing the difference between the encoding target CU and the predicted image to the conversion/quantization unit 120. . The transform/quantizer 120 generates quantized orthogonal transform coefficients from the prediction residual, and outputs the generated orthogonal transform coefficients to the entropy encoder 130.

<Image decoding device>
Next, an image decoding device according to this embodiment will be described. However, redundant explanation of operations similar to those of the image encoding device 1 described above will be omitted. FIG. 6 is a diagram showing the configuration of the image decoding device 2 according to this embodiment.

As shown in FIG. 6, the image decoding device 2 includes an entropy decoding section 200, an inverse quantization/inverse transformation section 210, a combining section 220, a memory 230, and a prediction section 240.

The entropy decoding unit 200 decodes the encoded stream generated by the image encoding device 1 and outputs quantized orthogonal transform coefficients to the inverse quantization/inverse transformation unit 210. The entropy decoding unit 200 also acquires control information regarding prediction (intra prediction and inter prediction), and outputs the acquired control information to the prediction unit 240.

The inverse quantization/inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing in units of CUs. The inverse quantization/inverse transformation section 210 includes an inverse quantization section 211 and an inverse transformation section 212.

The dequantization unit 211 performs dequantization processing corresponding to the quantization processing performed by the quantization unit 122 of the image encoding device 1. The dequantization unit 211 dequantizes the quantized orthogonal transform coefficients input from the entropy decoding unit 200 using the quantization parameter (Qp) and the quantization matrix, thereby dequantizing the orthogonal transform coefficients of the CU to be decoded. The restored orthogonal transform coefficients are output to the inverse transform unit 212.

The inverse transform unit 212 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 performs inverse orthogonal transform processing on the orthogonal transform coefficients input from the inverse quantization unit 211 to restore the prediction residual, and combines the restored prediction residual (restored prediction residual) to the synthesis unit 220 Output to.

The synthesis unit 220 reconstructs (decodes) the original CU by synthesizing the prediction residual input from the inverse transformation unit 212 and the predicted image input from the prediction unit 240 in units of pixels. The decoded image is output to the memory 230.

Memory 230 stores the decoded image input from synthesis section 220. The memory 230 stores decoded images in units of frames. Furthermore, the memory 230 stores the motion vector calculated by the inter prediction unit 242 for each CU. The memory 230 outputs the decoded image in units of frames to the outside of the image decoding device 2 . Note that the memory 230 may be composed of a plurality of memories. A loop filter may be provided between the synthesis section 220 and the memory 230.

The prediction unit 240 performs prediction on a CU basis. The prediction unit 240 includes an intra prediction unit 241, an inter prediction unit 242, and a switching unit 243.

The intra prediction unit 241 refers to the decoded image stored in the memory 230 and generates an intra predicted image by predicting the decoding target CU by intra prediction according to the control information input from the entropy decoding unit 200. The intra predicted image is output to the switching unit 243.

The inter prediction unit 242 uses the decoded image stored in the memory 230 as a reference image to predict the decoding target CU by inter prediction. The inter prediction unit 242 generates an inter prediction image by performing inter prediction according to the control information input from the entropy decoding unit 200, and outputs the generated inter prediction image to the switching unit 243. In this embodiment, the inter prediction unit 242 has the same configuration as the inter prediction unit 172 shown in FIG. 3.

The switching unit 243 switches between the intra-predicted image input from the intra-predicting unit 241 and the inter-predicted image input from the inter-predicting unit 242, and outputs one of the predicted images to the combining unit 220.

<Operation of inter prediction unit>
FIG. 7 is a diagram showing the operation of the inter prediction unit 172 of the image encoding device 1 according to the present embodiment. The inter prediction unit 242 of the image decoding device 2 also operates in the same manner as the inter prediction unit 172 of the image encoding device 1. However, in the operation of the inter prediction unit 242, the “CU to be encoded” in the operation of the inter prediction unit 172 is read as the “CU to be decoded”.

As shown in FIG. 7, in step S1, the adjacent vector acquisition unit 172a acquires motion vectors applied to inter prediction of each adjacent decoded CU located around (above or to the left of) the encoding target CU from the memory 160. Get Vp.

In step S2, the motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a technique such as block matching.

In step S3, the motion vector correction section 172d corrects the motion vector V calculated by the motion vector calculation section 172c using the motion vector Vp acquired by the adjacent vector acquisition section 172a, thereby obtaining a corrected motion vector V'. generate.

In step S4, the predicted image generation unit 172e performs inter prediction using the corrected motion vector V' to generate a predicted image of the encoding target CU.

In this way, according to the inter prediction unit 172 according to the present embodiment, discontinuity of the motion vector can be reduced using the motion vector corrected using the adjacent motion vector Vp, so that the prediction residual It is possible to suppress an increase in blocks near object boundaries and improve encoding efficiency.

<Example of changing the inter prediction unit>
FIG. 8 is a diagram showing a modification example of the inter prediction unit 172 according to the first embodiment.

As shown in FIG. 8, the inter prediction unit 172 includes a continuity evaluation unit 172f and a correction adjacent vector determination unit 172g in addition to the configuration according to the embodiment described above.

The continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the adjacent motion vector Vp, thereby determining the boundary area between each decoded block included in the adjacent decoded block group and the encoding target CU. Evaluate the continuity of movement.

The continuity evaluation unit 172f acquires from the memory 160 the motion vector Vp applied to one or more decoded adjacent CUs adjacent to the encoding target CU. Then, the continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the motion vector Vp applied to the adjacent decoded CU group, and determines the encoding target for the adjacent decoded CU group. The continuity of motion with the CU is evaluated, and the evaluation result is output to the correction adjacent vector determination unit 172g.

For example, if the L1 norm representing the difference (distance) between the motion vector V applied to the encoding target CU and the motion vector Vp of the decoded CU is less than a predetermined threshold, the continuity evaluation unit 172f Evaluated as having high quality. On the other hand, if the L1 norm is equal to or greater than the threshold value, it is evaluated that the continuity is low. However, the continuity index value is not limited to the L1 norm, and other index values may be used.

Note that the motion vector includes vector values in the X-axis and Y-axis directions as well as a flag indicating a decoded frame to be referenced (for example, inter_pred_idc, ref_idx in HEVC). In evaluating the continuity of motion vectors with adjacent blocks, the continuity evaluation unit 172f evaluates the temporal distance between the frame containing the encoding target CU (current frame) and the decoded frame referenced by the encoding target CU. (POC distance) and the POC distance between the current frame and the reference decoded frame of the adjacent decoded CU, the vector value of the motion vector applied to the adjacent decoded CU is scaled, and then You can also evaluate gender.

The correction adjacent vector determining unit 172g determines a motion vector to be used for correcting the motion vector V, based on the motion vector Vp, based on the continuity of motion evaluated by the continuity evaluation unit 172f.

For example, the correction adjacent vector determining unit 172g
V _L = [Vp[-1,0], Vp[-1,1], Vp[-1,2], …, Vp[-1,N-1]]
V _T = [Vp[0,-1], Vp[1,-1], Vp[2,-1], …, Vp[M-1,-1]]
When
V' _L =[ v _L ∈V _L | || v _L - V || ₂ < Th ]
V' _T =[ v _T ∈V _T | || v _T - V || ₂ < Th ]
A motion vector to be used for correcting the motion vector V is determined as follows.

That is, V' _L is a set of vectors whose L2 norm with the motion vector V is less than or equal to the threshold Th among the left adjacent vectors Vp of the CU to be encoded, and V' _T is the set of vectors on the left side of the CU to be encoded. This is a set made up of vectors among adjacent vectors Vp whose L2 norm with the motion vector V is less than or equal to the threshold Th.

The motion vector correction unit 172d uses these V' _L and V' _T (third motion vector) to calculate a corrected motion vector V' according to equation (4) below.

V'= W _L・median(V' _L ) + W _T・median(V' _T ) + W _V・V)
(4)

<2. Second embodiment>
Regarding the second embodiment, differences from the first embodiment will be mainly described.

FIG. 9 is a diagram showing the configuration of the inter prediction unit 172 of the image encoding device 1 according to the second embodiment.

As shown in FIG. 9, the inter prediction unit 172 according to the second embodiment has the configuration described in the first embodiment (adjacent vector acquisition unit 172a, motion vector calculation unit 172c, motion vector correction unit 172d, predicted image generation unit). 172e), it has a small region dividing section 172b.

The small region dividing unit 172b divides the encoding target CU into a plurality of small regions of a predefined size, and outputs information about each small region obtained by the division to the motion vector correction unit 172d. Such a small area may be referred to as a subblock.

FIG. 10 is a diagram showing an example of a small area. As shown in FIG. 10(a), the size of each small area may be the same, and each small area may be an area of 4×4 pixels, for example. Alternatively, if common processing is defined in advance for the image decoding device and the image decoding device, the size of each small area may be made different, as shown in FIG. 10(b). For example, the CU to be encoded may be divided into small regions by a method of finely dividing the area near the boundary and coarsely dividing the area near the center.

The motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a method such as block matching, and outputs the calculated motion vector V to the motion vector correction unit 172d. In this embodiment, the motion vector calculation unit 172c may calculate one motion vector V for each CU to be encoded, or for each small region obtained by dividing the CU to be encoded by the small region dividing unit 172b. A motion vector V may also be calculated.

The motion vector correction unit 172d generates a corrected motion vector V′ by correcting the motion vector V calculated by the motion vector calculation unit 172c using the motion vector Vp acquired by the adjacent vector acquisition unit 172a, and performs the correction. The motion vector V' is output to the predicted image generation section 172e.

In this embodiment, the motion vector correction unit 172d corrects the motion vector V for each small region, as shown in FIG. The motion vector correction unit 172d weights the motion vector Vp of the adjacent decoded CU corresponding to the position of the small area using the weight determined according to the position of the small area, and uses the weighted motion vector Vp, The motion vector V applied to the small area is corrected.

For example, the motion vector correction unit 172d assigns the motion vector V before correction to the motion vector array V[x,y] for each small region of the encoding target CU. However, if the motion vector V before correction is a motion vector that differs for each small block within the encoding target CU, a motion vector array that differs for each small area based on the motion vector set for each small block is performed. Assign to V[x,y]. If the size of the small block is different from the small area described above, it is substituted using a predefined method. For example, if the size of the small block is larger than the small area, the motion vector of the small block is substituted into the motion vector array V[x,y] of all the small areas included in the small block. On the other hand, if the size of the small block is smaller than the small area, the motion vector of the small area V[x, y ].

First, the motion vector correction unit 172d determines weighting coefficients W _L , W _T , W _LT , and W _V according to the position of each small region. The motion vector correction unit 172d calculates the weighting coefficients W _L , W _T , W _LT , and W _V using the following equation (5), for example.

（５）

(5)

However, x and y represent the position of the small area, and width and height represent the width and height of the encoding target CU, respectively. Note that the weighting coefficients W _L , W _T , W _LT , and W _V may be determined according to the position of the small region and the width and height of the CU, and are not limited to the calculation method of equation (5).

Second, the motion vector correction unit 172d corrects the motion vector V[x,y] before correction based on the weighting coefficients W _L , W _T , W _LT , W _V and the adjacent motion vector Vp, for example. A corrected motion vector V'[x,y] is calculated using the following equation (6).

（６）

(6)

The predicted image generation unit 172e performs inter prediction for each small region of the encoding target CU using the corrected motion vector V′[x,y] generated by the motion vector correction unit 172d, thereby predicting the encoding target CU. A predicted image is generated, and the generated predicted image (inter predicted image) is output to the switching unit 173.

Specifically, the predicted image generation unit 172e generates a predicted image for each small area by performing inter prediction for each small area using the motion vector V'[x,y] for each small area. A predicted image of the encoding target CU is generated by combining predicted images for each region.

Note that this predicted image is input to the subtraction unit 110 via the switching unit 173, and the subtraction unit 110 outputs a prediction residual representing the difference between the encoding target CU and the predicted image to the conversion/quantization unit 120. . The transform/quantizer 120 generates quantized orthogonal transform coefficients from the prediction residual, and outputs the generated orthogonal transform coefficients to the entropy encoder 130. Note that the entropy encoding unit 130 only needs to encode one motion vector for each CU to be encoded, without encoding (signaling) the motion vector for each small region.

Furthermore, in this embodiment, the inter prediction unit 242 of the image decoding device 2 has the same configuration as the inter prediction unit 172 shown in FIG. 9 .

<Operation of inter prediction unit>
FIG. 12 is a diagram showing the operation of the inter prediction unit 172 of the image encoding device 1 according to this embodiment. The inter prediction unit 242 of the image decoding device 2 also operates in the same manner as the inter prediction unit 172 of the image coding device 1. However, in the operation of the inter prediction unit 242, the “CU to be encoded” in the operation of the inter prediction unit 172 is read as the “CU to be decoded”.

As shown in FIG. 12, in step S11, the adjacent vector acquisition unit 172a acquires from the memory 160 motion vectors applied to inter prediction of each adjacent decoded CU located around (above or to the left of) the encoding target CU. Get Vp.

In step S12, the small region dividing unit 172b divides the encoding target CU into a plurality of small regions of a predefined size.

In step S13, the motion vector calculation unit 172c uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector V by a technique such as block matching.

In step S14, the motion vector correction section 172d corrects the motion vector V calculated by the motion vector calculation section 172c using the motion vector Vp acquired by the adjacent vector acquisition section 172a, thereby obtaining a corrected motion vector V'. generate.

In step S15, the predicted image generation unit 172e performs inter prediction for each small region using the corrected motion vector V', and generates a predicted image of the encoding target CU.

In this way, the inter prediction unit 172 according to the present embodiment can perform detailed inter prediction using a motion vector corrected using the adjacent motion vector Vp for each small region within the CU to be decoded. Therefore, it is possible to suppress the prediction residual from increasing in blocks near the object boundary and improve encoding efficiency.

<Example of changing the inter prediction unit>
FIG. 13 is a diagram showing a modification example of the inter prediction unit 172 according to the second embodiment.

As shown in FIG. 13, the inter prediction unit 172 according to the present modification includes a continuity evaluation unit 172f and a correction small region determination unit 172h in addition to the configuration according to the embodiment described above.

The continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the adjacent motion vector Vp, thereby determining the boundary area between each decoded block included in the adjacent decoded block group and the encoding target CU. Evaluate the continuity of movement.

The continuity evaluation unit 172f acquires from the memory 160 the motion vector Vp applied to one or more decoded adjacent CUs adjacent to the encoding target CU. Then, the continuity evaluation unit 172f compares the motion vector V applied to the encoding target CU with the motion vector Vp applied to the adjacent decoded CU group, and determines the encoding target for the adjacent decoded CU group. The continuity of the motion with the CU is evaluated, and the evaluation result is output to the correction small area determination unit 172h.

For example, as shown in FIG. 14A, the continuity evaluation unit 172f determines in advance the L1 norm representing the difference (distance) between the motion vector V applied to the encoding target CU and the motion vector Vp of the decoded CU. If it is less than the specified threshold, it is evaluated that continuity is high. On the other hand, if the L1 norm is equal to or greater than the threshold value, it is evaluated that the continuity is low. However, the continuity index value is not limited to the L1 norm, and other index values may be used.

Note that the motion vector includes vector values in the X-axis and Y-axis directions as well as a flag indicating a decoded frame to be referenced (for example, inter_pred_idc, ref_idx in HEVC). In evaluating the continuity of motion vectors with adjacent blocks, the continuity evaluation unit 172f evaluates the temporal distance between the frame containing the encoding target CU (current frame) and the decoded frame referenced by the encoding target CU. (POC distance) and the POC distance between the current frame and the reference decoded frame of the adjacent decoded CU, the vector value of the motion vector applied to the adjacent decoded CU is scaled, and then You can also evaluate gender.

The correction small area determination unit 172h determines at least one small area in which the motion vector V should be corrected based on the continuity of motion evaluated by the continuity evaluation unit 172f. For example, as shown in FIG. 14(b), the correction small area determining unit 172h determines only each small area corresponding to an area with low motion continuity as a small area whose motion vector V should be corrected. However, the correction small area determining unit 172h may determine all the small areas of the encoding target CU as small areas in which the motion vector V should be corrected.

In this modified example, the motion vector correction section 172d generates a corrected motion vector V' for the small region determined by the corrected small region determining section 172h. Further, the motion vector correction unit 172d determines a weighting coefficient according to the position of the small area and the continuity of the motion evaluated by the continuity evaluation unit 172f, and uses the determined weighting coefficient to calculate the weighting coefficient of the small area. Adjacent motion vectors Vp corresponding to positions may be weighted.

The motion vector correction unit 172d may be set to increase the weight of the adjacent motion vector Vp corresponding to an area with low motion continuity. For example, values c _x and c _y corresponding to continuity of motion are introduced into equation (5), and weighting coefficients W _L , W _T , W _LT , and W _V are calculated using equation (7) below.

（７）

(7)

In this way, according to this modification example, by correcting the motion vector V of the encoding target CU (small area) in consideration of the continuity of motion in the block boundary area, the encoding efficiency when performing inter prediction is improved. can be further improved.

<3. Other embodiments>
It may be provided by a program that causes a computer to execute each process performed by the image encoding device 1 and a program that causes a computer to execute each process performed by the image decoding device 2. Moreover, the program may be recorded on a computer-readable medium. Computer-readable media allow programs to be installed on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

Further, the image encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC) by integrating circuits that execute each process performed by the image encoding device 1. Similarly, the image decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC) by integrating circuits that execute each process performed by the image decoding device 2.

Although the embodiments have been described above in detail with reference to the drawings, the specific configuration is not limited to that described above, and various design changes can be made without departing from the scope of the invention.

1: Image encoding device 2: Image decoding device 100: Block division unit 110: Subtraction unit 120: Transformation/quantization unit 121: Transformation unit 122: Quantization unit 130: Entropy encoding unit 140: Inverse quantization/inverse transformation Unit 141: Inverse quantization unit 142: Inverse transformation unit 150: Combination unit 160: Memory 170: Prediction unit 171: Intra prediction unit 172: Inter prediction unit 172a: Adjacent vector acquisition unit 172b: Small area division unit 172c: Motion vector calculation Section 172d: Motion vector correction section 172e: Predicted image generation section 172f: Continuity evaluation section 172g: Correction adjacent vector determination section 172h: Correction small area determination section 173: Switching section 200: Entropy decoding section 210: Inverse quantization/inverse Transforming unit 211: Inverse quantizing unit 212: Inverse transforming unit 220: Combining unit 230: Memory 240: Predicting unit 241: Intra predicting unit 242: Inter predicting unit 243: Switching unit

Claims (6)

An inter prediction device that performs inter prediction of a block to be encoded obtained by dividing an original image,
an adjacent vector acquisition unit that acquires a first motion vector applied to blocks surrounding the encoding target block;
a motion vector correction unit that generates a corrected motion vector by correcting a second motion vector applied to the encoding target block using the first motion vector;
a predicted image generation unit that generates a predicted image of the encoding target block by performing the inter prediction using the corrected motion vector ,
The motion vector correction section includes:
An inter prediction device, characterized in that the second motion vector is corrected using the first motion vector and a parameter determined according to the width and height of the encoding target block . further comprising a small area dividing unit that divides the encoding target block into a plurality of small areas,
The motion vector correction unit generates a corrected motion vector by correcting a second motion vector applied to each small region of the encoding target block using the first motion vector corresponding to the position of the small region. The inter prediction device according to claim 1, characterized in that: The motion vector correction section includes:
weighting the first motion vector corresponding to the position of the small area using a weight determined according to the position of the small area;
The inter prediction device according to claim 2 , wherein the weighted first motion vector is used to correct a second motion vector applied to the small area. An image encoding device comprising the inter prediction device according to any one of claims 1 to 3 . An image decoding device comprising the inter prediction device according to any one of claims 1 to 3 . A program that causes a computer to function as the inter prediction device according to any one of claims 1 to 3 .

Images