JP2020053725A - Predictive image correction device, image encoding device, image decoding device, and program - Google Patents

Abstract

To improve encoding efficiency when performing inter-prediction.SOLUTION: A predictive image correction device which corrects a prediction image obtained through inter-prediction for each of blocks constituting an image comprises: a continuity evaluation part which compares a motion vector applied to an object block for which the inter-prediction is performed with a motion vector applied to one or a plurality of contiguous blocks adjoining the object block and having been decoded so as to evaluate continuity of motion of the one or plurality of adjacent blocks to the object block; and a correction part which uses, when there is an adjacent block whose continuity evaluated by the continuity evaluation part is less than a predetermined value is present, a decoded image of a border region on the object block in a decoded image of the adjacent block whose continuity is less than the predetermined value so as to correct a prediction image of the border region on the adjacent block whose continuity is less than the predetermined value among prediction images of the object block.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a predicted image correction device, an image encoding device, an image decoding device, and a program.

  In a conventional video coding method represented by HEVC described in Non-Patent Document 1, an image coding device divides an original image into blocks, performs prediction while switching between inter prediction and intra prediction for each block, It is configured to output a stream by performing orthogonal transformation, quantization, and entropy coding on a prediction residual representing an error of a prediction image obtained by prediction.

  In the inter prediction in such a video coding method, if a motion vector applied to a coding target block (Coding Unit: CU) is different from a motion vector applied to an adjacent CU adjacent to the coding target CU, the coding is performed. The predicted image of the target CU and the predicted image of the adjacent CU have significantly different features.

  In the example of the conventional video encoding method shown in FIG. 1, it is assumed that object # 1 is moving to the upper left and object # 2 is moving to the upper right. In the inter prediction for the adjacent CU # 1 located on the left side of the encoding target CU including only the object # 1, it is highly likely that the same motion vector (MV # 3) as the motion of the object # 1 is applied. The encoding target CU is located at the object boundary including both the object # 1 and the object # 2, but since the object # 1 occupies a large area, the motion vector (MV #) of the adjacent CU # 1 adjacent on the left side It is highly likely that the same motion vector (MV # 1) as in 2) is applied to the coding target CU.

  When the motion vectors of adjacent CUs are the same, the continuity of motion in the boundary area of the CU is high, and the prediction accuracy is likely to be high. On the other hand, since CU # 2 adjacent to the encoding target CU in FIG. 1 includes many objects # 2, there is a high possibility that the same motion vector (MV # 3) as the motion of object # 2 will be applied. . That is, there is a high possibility that the motion vector (MV # 1) applied to the coding target CU and the motion vector (MV # 3) applied to CU # 1 are significantly different. When the motion vectors of adjacent CUs are significantly different, it is highly possible that at least one of these CUs includes an object boundary.

Recommendation ITU-TH. 265, (12/2016), “High efficiency video coding”, International Telecommunication Union

  The discontinuity of the motion vector at the object boundary causes a problem that the prediction accuracy is reduced and the coding efficiency is reduced. In HEVC, in order to solve this problem, in a loop filter, block distortion is determined according to whether a difference between a motion vector applied to an encoding target CU and a motion vector applied to an adjacent block is larger than a predetermined value. A technique has been introduced to control whether or not to apply a deblocking filter for removing.

  However, in prediction (inter prediction), since the same processing is applied regardless of the discontinuity of the motion vector, the amount of information necessary for transmitting orthogonal transform coefficients increases in blocks near the object boundary. However, there is a problem that the coding efficiency is reduced.

  Therefore, an object of the present invention is to provide a predicted image correction device, an image encoding device, an image decoding device, and a program that can improve encoding efficiency when performing inter prediction.

  A predicted image correction device according to a first feature is a predicted image correction device that corrects a predicted image obtained by performing inter prediction for each block that forms an image, wherein a target block on which the inter prediction is performed is By comparing the applied motion vector with a motion vector applied to one or more decoded neighboring blocks adjacent to the current block, the continuity of motion of the one or more neighboring blocks with the current block is compared. And a continuity evaluation unit that evaluates, when there is an adjacent block in which the continuity evaluated by the continuity evaluation unit is lower than a predetermined value, the adjacent block in which the continuity is lower than a predetermined value The continuity of the predicted image of the target block is lower than a predetermined value using the decoded image of the boundary region with the target block in the decoded images Prediction image of the boundary region between the contact block and summarized in that and a correcting unit for correcting the.

  The gist of the image encoding device according to the second feature is to include an inter prediction unit that performs inter prediction and a predicted image correction device according to the first feature.

  The gist of the image decoding device according to the third feature is to include an inter prediction unit that performs inter prediction and a predicted image correction device according to the first feature.

  The gist of the program according to the fourth feature is to cause a computer to function as the predicted image correction device according to the first feature.

  According to the present invention, it is possible to provide a predicted image correction device, an image encoding device, an image decoding device, and a program capable of improving encoding efficiency when performing inter prediction.

It is a figure for explaining background art. It is a figure showing the composition of the picture coding device concerning an embodiment. It is a figure showing the composition of the prediction picture amendment part in the picture coding device concerning an embodiment. FIG. 5 is a diagram for explaining an operation of a predicted image correction unit according to the embodiment. It is a figure for explaining operation of a continuity evaluation part in a prediction picture amendment part concerning an embodiment. FIG. 5 is a diagram for explaining an operation of a correction unit in the predicted image correction unit according to the embodiment. FIG. 7 is a diagram for describing a specific example of the operation of the correction unit in the predicted image correction unit according to the embodiment. It is a figure showing operation of a prediction picture amendment part in an image coding device concerning an embodiment. It is a figure showing the composition of the picture decoding device concerning an embodiment. It is a figure showing the composition of the prediction picture amendment part of the picture decoding device concerning an embodiment. FIG. 11 is a diagram for explaining an operation according to a modification. It is a figure showing composition of a prediction picture amendment part concerning a modification. It is a figure for explaining operation of a residual energy calculation part in a prediction picture amendment part concerning a modification. It is a figure for explaining operation of a correction part in a prediction picture correction part concerning a modification. FIG. 11 is a diagram for explaining an operation according to another embodiment.

  An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. The image encoding device and the image decoding device according to the embodiment respectively encode and decode a moving image represented by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

<Image coding device>
First, an image encoding device according to the present embodiment will be described. FIG. 2 is a diagram illustrating a configuration of the image encoding device 1 according to the present embodiment.

  As illustrated in FIG. 2, the image encoding device 1 includes a block division unit 100, a subtraction unit 110, a transformation / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transformation unit 140, It includes a synthesizing unit 150, a memory unit 160, and a prediction unit 170.

  The block division unit 100 divides an input image in units of frames (or pictures) constituting a moving image into a plurality of blocks, and outputs the blocks obtained by the division to the subtraction unit 110. The size of the block is, for example, 32 × 32 pixels, 16 × 16 pixels, 8 × 8 pixels, or 4 × 4 pixels. The shape of the block is not limited to a square, but may be a rectangle. A block is a unit in which the image encoding device 1 performs encoding and a unit in which the image decoding device performs decoding. Hereinafter, such a block is referred to as a CU (Coding Unit).

  The subtraction unit 110 calculates a prediction residual indicating a difference (error) between the coding target CU input from the block dividing unit 100 and a prediction image obtained by the prediction unit 170 predicting the coding target CU. Specifically, the subtraction unit 110 calculates a prediction residual by subtracting each pixel value of the prediction image from each pixel value of the CU, and outputs the calculated prediction residual to the transform / quantization unit 120.

  The transform / quantization unit 120 performs an orthogonal transform process and a quantization process on a CU basis. The conversion / quantization unit 120 includes a conversion unit 121 and a quantization unit 122.

  Transform section 121 performs orthogonal transform processing on the prediction residual input from subtraction section 110 to calculate an orthogonal transform coefficient, and outputs the calculated orthogonal transform coefficient to quantization section 122. The orthogonal transform refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen Lobe transform (KLT), or the like.

  The quantization unit 122 quantizes the orthogonal transform coefficient input from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix, and quantizes the quantized orthogonal transform coefficient into the entropy encoding unit 130 and the inverse quantization / Output to the inverse transform unit 140. Note that the quantization parameter (Qp) is a parameter commonly applied to each orthogonal transform coefficient in the CU, and is a parameter that determines the roughness of quantization. The quantization matrix is a matrix having, as elements, a quantization value when quantizing each orthogonal transform coefficient.

  The entropy coding unit 130 performs entropy coding on the orthogonal transform coefficients input from the quantization unit 122, performs data compression to generate a coded stream (bit stream), and performs image coding on the coded stream. Output to the outside of the device 1. For the entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding; context adaptive binary arithmetic coding), or the like can be used. Note that the entropy coding unit 130 receives control information related to prediction from the prediction unit 170, and also performs entropy coding of the input control information.

  The inverse quantization / inverse transforming unit 140 performs an inverse quantization process and an inverse orthogonal transform process on a CU basis. The inverse quantization / inverse transformation unit 140 includes an inverse quantization unit 141 and an inverse transformation unit 142.

  The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the orthogonal transform coefficient by inversely quantizing the orthogonal transform coefficient input from the quantization unit 122 using the quantization parameter (Qp) and the quantization matrix. , And outputs the restored orthogonal transform coefficients to the inverse transform unit 142.

  The inverse transform unit 142 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121. For example, when the transform unit 121 performs the discrete cosine transform, the inverse transform unit 142 performs the inverse discrete cosine transform. The inverse transform unit 142 performs an inverse orthogonal transform process on the orthogonal transform coefficient input from the inverse quantization unit 141 to restore the prediction residual, and combines the restored prediction residual that is the restored prediction residual with the synthesis unit 150. Output to

  The combining unit 150 combines the restored prediction residual input from the inverse transform unit 142 with the predicted image input from the prediction unit 170 in pixel units. The combining unit 150 reconstructs (decodes) the CU to be encoded by adding each pixel value of the restored prediction residual and each pixel value of the predicted image, and outputs the decoded CU unit decoded image to the memory unit 160. . Such a decoded image may be called a reconstructed image.

  The memory unit 160 stores the decoded image input from the combining unit 150. The memory unit 160 stores the decoded image in frame units. The memory unit 160 outputs the stored decoded image to the prediction unit 170. Further, the memory unit 160 stores the motion vector calculated by the inter prediction unit 172 for each CU. Note that the memory unit 160 may be configured by a plurality of memories. Further, a loop filter may be provided between the combining unit 150 and the memory unit 160.

  The prediction unit 170 performs prediction on a CU basis. The prediction unit 170 includes an intra prediction unit 171, an inter prediction unit 172, a predicted image correction unit 173, and a switching unit 174. The predicted image correction unit 173 corresponds to a predicted image correction device.

  The intra prediction unit 171 generates an intra prediction image by referring to decoded pixel values around the encoding target CU among the decoded images stored in the memory unit 160, and sends the generated intra prediction image to the switching unit 174. Output. In addition, the intra prediction unit 171 selects an optimal intra prediction mode to be applied to the target CU from a plurality of intra prediction modes, and performs intra prediction using the selected intra prediction mode. Intra prediction section 171 outputs control information on the selected intra prediction mode to entropy encoding section 130.

  The inter prediction unit 172 calculates a motion vector by a method such as block matching using the decoded image stored in the memory unit 160 as a reference image, generates an inter prediction image by predicting a coding target CU, and generates an inter prediction image. The predicted inter prediction image is output to the predicted image correction unit 173 together with the motion vector. Further, the inter prediction unit 172 outputs the motion vector calculated for each CU to the memory unit 160, and stores the motion vector in the memory unit 160. The inter prediction unit 172 selects an optimal inter prediction method from inter prediction (typically, bi-prediction) using a plurality of reference images and inter prediction (unidirectional prediction) using one reference image, Inter prediction is performed using the selected inter prediction method. The inter prediction unit 172 outputs control information (such as a motion vector) related to inter prediction to the entropy coding unit 130.

  The predicted image correction unit 173 corrects the inter predicted image input from the inter prediction unit 172, and outputs the corrected inter predicted image to the switching unit 174. Specifically, the predicted image correction unit 173 acquires the motion vector applied to the encoding target CU from the inter prediction unit 172, and calculates the motion vector of the decoded block group adjacent to the encoding target CU and the decoded vector. The decoded image of the boundary area between the block group and the encoding target CU is acquired from the memory unit 160, and the inter prediction image of the encoding target CU is corrected based on the acquired information. Details of the predicted image correction unit 173 will be described later.

  The switching unit 174 switches between the intra prediction image input from the intra prediction unit 171 and the inter prediction image input from the prediction image correction unit 173, and outputs one of the prediction images to the subtraction unit 110 and the synthesis unit 150. .

  Next, the predicted image correction unit 173 will be described. FIG. 3 is a diagram illustrating a configuration of the predicted image correction unit 173 of the image encoding device 1.

  As shown in FIG. 3, the predicted image correction unit 173 includes a continuity evaluation unit 173a and a correction unit 173b.

  The continuity evaluation unit 173a acquires, from the memory unit 160, a motion vector applied to one or a plurality of decoded adjacent CUs adjacent to the encoding target CU. For example, as illustrated in FIG. 4, when the decoded CU # 1 to CU # 7 adjacent to the encoding target CU are included in the decoded CU group, the continuity evaluation unit 173a sets the CU # 1 to CU # 7 , And obtain a motion vector applied to inter prediction.

  In FIG. 4, the continuity evaluation unit 173a obtains the motion vectors of the decoded CUs adjacent to the upper side and the left side of the encoding target CU, but when the right and lower CUs are already decoded. May further acquire a motion vector applied to the right or lower CU.

  Then, the continuity evaluation unit 173a compares the motion vector applied to the encoding target CU with the motion vector applied to the adjacent decoded CU group, and determines the adjacent decoded CU group as the encoding target CU. Is evaluated, and the evaluation result is output to the correction unit 173b.

  Specifically, the continuity evaluation unit 173a evaluates that continuity is high when the L1 norm between the vector applied to the encoding target CU and the motion vector of the decoded CU is less than a predetermined threshold. . On the other hand, when the L1 norm is equal to or larger than the threshold, it is evaluated that the continuity is low. However, the index value of continuity is not limited to the L1 norm, and another index value may be used.

  For example, as illustrated in FIG. 5, the continuity evaluation unit 173a determines the difference between the motion vector of each of the decoded CU # 1 to CU # 7 adjacent to the current CU and the motion vector of the current CU. An L1 norm representing (distance) is calculated, and the L1 norm calculated for each of CU # 1 to CU # 7 is compared with a threshold.

  In the example shown in FIG. 5, the L1 norm corresponding to CU # 1, the L1 norm corresponding to CU # 2, the L1 norm corresponding to CU # 3, the L1 norm corresponding to CU # 4, and CU # 5 Since the L1 norm is equal to or larger than the threshold value, it means that these adjacent CUs have low continuity with the coding target CU. On the other hand, since the L1 norm corresponding to CU # 6 and the L1 norm corresponding to CU # 7 are smaller than the threshold value, these adjacent CUs have high continuity with the CU to be encoded. When all of CU # 1 to CU # 7 are evaluated as having high continuity, the correction of the inter prediction image of the encoding target CU by the correction unit 173b may be omitted.

  In the present embodiment, it is assumed that the motion vector includes, in addition to the vector values in the X-axis and Y-axis directions, flags (for example, inter_pred_idc and ref_idx in HEVC) indicating the decoded frame to be referred to. When evaluating the continuity of the motion vector with the adjacent block, the continuity evaluation unit 173a sets a temporal distance between the frame including the encoding target CU (current frame) and the decoded frame referred to by the encoding target CU. (POC distance) and the POC distance between the current frame and the decoded frame of the reference destination of the adjacent decoded CU, and then successively scales the vector value of the motion vector applied to the adjacent decoded CU. Sex may be evaluated.

  The correction unit 173b stores at least a decoded image of a boundary area between the CU to be encoded and a decoded image of a group of decoded CUs adjacent to the CU to be encoded, in order to correct the inter prediction image of the CU to be encoded. Obtained from the unit 160.

  For example, as illustrated in FIG. 4, among the decoded images of the CU # 1 to CU # 7 that have been decoded and are adjacent to the coding target CU, the correction unit 173b decodes the decoded image of the boundary area with the coding target CU. From the memory unit 160.

  Note that the boundary area with the encoding target CU refers to an area including decoded pixels adjacent to the encoding target CU, and an area including decoded pixels from the encoding target CU to N pixels (N> 1). May be used as the boundary region.

  Further, in the example illustrated in FIG. 4, the correction unit 173b does not acquire the decoded images of the boundary areas for all of the decoded CUs # 1 to CU # 7 adjacent to the coding target CU, and the CU # 1 to the CU # 1. The decoded image of the boundary area may be obtained only for CU # 1 to CU # 5 (see FIG. 5) evaluated as having low continuity by the continuity evaluation unit 173a among # 7.

  Then, when there is an adjacent CU (predetermined adjacent block) evaluated as having low continuity by the continuity evaluation unit 173a, the correction unit 173b determines at least a boundary between the decoded image of the adjacent CU and the coding target CU. Using the decoded image of the area, at least the predicted image of the boundary area with the adjacent CU among the inter-predicted images of the coding target CU is corrected.

  Specifically, the correction unit 173b decodes the predicted image of the coding target CU with respect to the block boundary between the neighboring CU evaluated as having low continuity and the coding target CU by the continuity evaluating unit 173a. Correction is made using the value of the image. For example, as illustrated in FIG. 6, the correction unit 173b determines the encoding target CU with respect to the block boundary between the adjacent CU # 1 to CU # 5 evaluated as having low continuity by the continuity evaluation unit 173a and the encoding target CU. The inter prediction image of the CU is corrected using the values of the decoded images of the adjacent CU # 1 to CU # 5.

  Here, a specific example of the correction method by the correction unit 173b will be described. The inter prediction image generated using the motion vector applied to the encoding target CU (width M, height N) is pred [i, j] (i = 0... M-1, j = 0... N-1). When the decoded image of the adjacent decoded CU is rec [i, j] (i = -1, j = −1... N and i = −1... M, j = −1), the correction unit 173b The corrected inter prediction image pred '[i, j] is calculated by the equation shown in FIG.

  FIG. 7A illustrates an example of correcting the left block boundary region of the inter prediction image of the encoding target CU, and FIG. 7B illustrates correcting the upper block boundary region of the inter prediction image of the encoding target CU. FIG. 7C shows an example in which the block boundary area at the upper left corner of the inter prediction image of the coding target CU is corrected. In the inter prediction image, up to two pixels from the block boundary are corrected as a block boundary region, but up to M pixels (M ≧ 3) from the block boundary may be corrected as a block boundary region.

  When the inter prediction image of the encoding target CU is corrected in this way, the correction unit 173b outputs the corrected inter prediction image to the subtraction unit 110 and the synthesis unit 150 via the switching unit 174.

  Next, the operation of the predicted image correction unit 173 will be described. FIG. 8 is a diagram illustrating the operation of the predicted image correction unit 173 of the image encoding device 1.

  As shown in FIG. 8, in step S101, the continuity evaluation unit 173a compares the motion vector applied to the coding target CU on which inter prediction has been performed with the motion vector applied to the decoded adjacent block. Evaluates the continuity of the motion between the neighboring block and the coding target CU.

  When there is a predetermined adjacent block whose continuity evaluated by the continuity evaluation unit 173a is lower than the predetermined value (step S102: YES), in step S103, the correction unit 173b determines at least the code of the decoded image of the predetermined adjacent block. Using the decoded image of the boundary region with the coding target CU, the prediction image of the boundary region with at least a predetermined adjacent block in the prediction image of the coding target CU is corrected.

  As described above, according to the predicted image correction unit 173, by correcting the predicted image in consideration of the discontinuity of the motion vector, it is possible to reduce the amount of information necessary for transmitting the orthogonal transform coefficient for the block near the object boundary. Therefore, it is possible to improve coding efficiency when performing inter prediction.

<Image decoding device>
Next, the image decoding device according to the present embodiment will be described. However, for the same operations as those of the image encoding device 1 described above, redundant description will be omitted. FIG. 9 is a diagram illustrating a configuration of the image decoding device 2 according to the present embodiment.

  As illustrated in FIG. 9, the image decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse transform unit 210, a combining unit 220, a memory unit 230, and a prediction unit 240.

  The entropy decoding unit 200 decodes the encoded stream generated by the image encoding device 1 and outputs the quantized orthogonal transform coefficients to the inverse quantization / inverse transform unit 210. Further, entropy decoding section 200 acquires control information regarding prediction (intra prediction and inter prediction), and outputs the acquired control information to prediction section 240.

  The inverse quantization / inverse transform unit 210 performs an inverse quantization process and an inverse orthogonal transform process on a CU basis. The inverse quantization / inverse transformation unit 210 includes an inverse quantization unit 211 and an inverse transformation unit 212.

  The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the image encoding device 1. The inverse quantization unit 211 inversely quantizes the quantized orthogonal transform coefficient input from the entropy decoding unit 200 using the quantization parameter (Qp) and the quantization matrix, thereby obtaining the orthogonal transform coefficient of the decoding target CU. The orthogonal transform coefficients thus restored are output to the inverse transform unit 212.

  The inverse transform unit 212 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 restores a prediction residual by performing an inverse orthogonal transform process on the orthogonal transform coefficient input from the inverse quantization unit 211, and combines the restored prediction residual (reconstructed prediction residual) with the combining unit 220. Output to

  The combining unit 220 reconstructs (decodes) the original CU by combining the prediction residual input from the inverse transform unit 212 and the predicted image input from the prediction unit 240 on a pixel-by-pixel basis, and Is output to the memory unit 230.

  The memory unit 230 stores the decoded image input from the combining unit 220. The memory unit 230 stores the decoded image in frame units. Further, the memory unit 230 stores the motion vector calculated by the inter prediction unit 242 for each CU. The memory unit 230 outputs the decoded image in frame units to the outside of the image decoding device 2. Note that the memory unit 230 may be configured by a plurality of memories. A loop filter may be provided between the combining unit 220 and the memory unit 230.

  The prediction unit 240 performs prediction on a CU basis. The prediction unit 240 includes an intra prediction unit 241, an inter prediction unit 242, a predicted image correction unit 243, and a switching unit 244. The predicted image correction unit 243 corresponds to a predicted image correction device.

  The intra prediction unit 241 generates an intra prediction image by referring to the decoded image stored in the memory unit 230 and predicting a decoding target CU by intra prediction according to the control information input from the entropy decoding unit 200. The switched intra prediction image is output to the switching unit 244.

  The inter prediction unit 242 predicts a decoding target CU by inter prediction using the decoded image stored in the memory unit 230 as a reference image. The inter prediction unit 242 generates an inter prediction image by performing inter prediction according to the control information input from the entropy decoding unit 200, and outputs the generated inter prediction image to the prediction image correction unit 243.

  The predicted image correction unit 243 corrects the inter predicted image input from the inter prediction unit 242, and outputs the corrected inter predicted image to the switching unit 244. Specifically, the predicted image correction unit 243 acquires the motion vector applied to the decoding target CU from the inter prediction unit 242, and calculates the motion vector of the decoded block group adjacent to the decoding target CU and the decoded block group. Of the CU to be decoded is acquired from the memory unit 230, and the inter prediction image of the CU to be decoded is corrected based on the acquired information. Details of the predicted image correction unit 243 will be described later.

  The switching unit 244 switches between the intra prediction image input from the intra prediction unit 241 and the inter prediction image input from the prediction image correction unit 243, and outputs one of the prediction images to the synthesis unit 220.

  Next, the predicted image correction unit 243 will be described. FIG. 10 is a diagram illustrating a configuration of the predicted image correction unit 243 of the image decoding device 2.

  As illustrated in FIG. 10, the predicted image correction unit 243 includes a continuity evaluation unit 243a and a correction unit 243b. Since the operations of the continuity evaluation unit 243a and the correction unit 243b are the same as the operations of the continuity evaluation unit 173a and the correction unit 173b of the image encoding device 1, the operations of the continuity evaluation unit 243a and the correction unit 243b will be described below. A brief description will be given.

  The continuity evaluation unit 243a obtains, from the memory unit 230, a motion vector applied to one or a plurality of decoded adjacent CUs adjacent to the CU to be decoded.

  Then, the continuity evaluation unit 243a compares the motion vector applied to the decoding target CU with the motion vector applied to the adjacent decoded CU group, and determines the motion between the adjacent decoded CU group and the decoding target CU. And outputs the evaluation result to the correction unit 243b.

  The correction unit 243b corrects, from the memory unit 230, at least the decoded image of the boundary region with the decoding target CU among the decoded images of the group of decoded CUs adjacent to the decoding target CU in order to correct the inter prediction image of the decoding target CU. get.

  Then, when there is an adjacent CU (predetermined adjacent block) evaluated as having low continuity by the continuity evaluation unit 243a, the correction unit 243b determines at least a boundary region between the decoded image of the adjacent CU and the CU to be decoded. Of the inter prediction image of the decoding target CU, at least the prediction image of the boundary region with the adjacent CU is corrected using the decoding image of (i). Specifically, for the block boundary between the adjacent CU evaluated to have low continuity and the decoding target CU in the continuity evaluating unit 243a, the correction unit 243b converts the prediction image of the decoding target CU into a decoded image of the adjacent CU. Correct using the value.

  When the inter prediction image of the decoding target CU is corrected in this way, the correction unit 243b outputs the corrected inter prediction image to the combining unit 220 via the switching unit 244.

  Note that the operation flow of the prediction image correction unit 243 of the image decoding device 2 is the same as that of FIG. 8, but in the operation flow shown in FIG. 8, “encoding CU” is replaced with “decoding CU”.

  As described above, according to the predicted image correction unit 243, the coding efficiency in the case of performing inter prediction can be improved by correcting the predicted image in consideration of the discontinuity of the motion vector.

<Example of change>
In the above embodiment, in the evaluation of the continuity with the decoded adjacent block, the continuity is calculated by comparing the motion vector applied to the encoding target CU (or the decoding target CU) with the motion vector applied to the decoded adjacent CU. It is evaluated whether the continuity is high and the inter prediction image is corrected in accordance with the low continuity.

  However, in such a method, the correction may not function properly. For example, as shown in FIG. 11, since the CU to be coded and the adjacent CU # 2 above the CU to be coded have different motion vectors, the upper block boundary region in the CU to be coded is corrected. Will be

  However, the object boundaries of the objects # 1 and # 2 are included in the lower block boundary region of the adjacent CU # 2. In such a case, if the inter prediction image of the coding target CU is corrected, the inter prediction image may be deteriorated.

  Thus, in this modified example, in addition to the comparison of the motion vectors as in the above-described embodiment, the presence or absence of correction is determined according to the energy of the prediction residual (residual signal) in the decoded adjacent block.

  In the example shown in FIG. 11, since the object boundaries of objects # 1 and # 2 are included in the lower block boundary region in the adjacent CU # 2, the energy of the prediction residual in the lower block boundary region is large. Likely to be. In such a case, the correction is not performed on the inter prediction image of the encoding target CU. On the other hand, when the energy of the prediction residual in the lower block boundary area is small, the correction is performed on the inter prediction image of the coding target CU.

  An image encoding device 1 according to the present modification will be described. In the present modified example, the memory unit 160 of the image encoding device 1 stores the prediction residual of the decoded CU adjacent to the encoding target CU. By storing the prediction residual obtained in each block process in the memory unit 160, the prediction residual stored in the memory unit 160 can be used in the subsequent block processing.

  The memory unit 160 may store the prediction residual output from the subtraction unit 110, or may store the prediction residual output from the inverse transform unit 142. The prediction image correction unit 173 acquires a prediction residual of a boundary region with the encoding target CU among prediction residuals of the adjacent block stored in the memory unit 160.

  FIG. 12 is a diagram illustrating a configuration of the predicted image correction unit 173 according to the present modification. As illustrated in FIG. 12, the predicted image correction unit 173 according to the present modification includes a residual energy calculation unit 173c in addition to the continuity evaluation unit 173a and the correction unit 173b similar to the above-described embodiment.

  The residual energy calculation unit 173c acquires the prediction residual near the block boundary of the decoded adjacent block adjacent to the encoding target CU from the memory unit 160, and calculates the prediction residual around the block boundary of the acquired decoded adjacent block. The energy is calculated, and the calculated energy value (adjacent residual energy) is output to the correction unit 173b.

  For example, as illustrated in FIG. 13, the residual energy calculation unit 173c calculates the energy of the prediction residual near each block boundary of the decoded adjacent blocks CU # 1 to CU # 7 adjacent to the coding target CU. . It is assumed that the area for calculating the energy is the number of lines defined in advance in the normal direction from the block boundary of the coding target CU. However, the number of lines may be variable according to the block size.

  Further, at the corner of the encoding target CU, an area for a prescribed number of lines in the horizontal and vertical directions is set as an area for residual energy calculation. For example, as for the upper left corner, when the upper left coordinate of the encoding target CU is (0, 0), the region from (−N, −N) to (−1, −1) is used for the residual energy calculation. A region (N is a predetermined number of lines).

  Specifically, the residual energy calculation unit 173c can use the average of the absolute values of the prediction residuals in the calculation area as the residual energy. However, any other index, such as the average of the sum of squares, may be used as long as the index calculates the energy of the prediction residual for each region.

  Note that the residual energy calculating unit 173c does not calculate the energy of the prediction residual of the boundary region for all of the decoded CU # 1 to CU # 7 adjacent to the encoding target CU, and instead calculates the CU # 1 to CU #. 7, the energy of the prediction residual of the boundary region may be calculated for only CU # 1 to CU # 5 (see FIG. 5) evaluated as having low continuity by the continuity evaluation unit 173a.

  The correction unit 173b decodes a boundary region of an adjacent block whose energy calculated by the residual energy calculation unit 173c is smaller than a predetermined threshold among the adjacent blocks evaluated as having low continuity by the continuity evaluation unit 173a. The inter prediction image of the encoding target CU is corrected using the image.

  For example, as shown in FIG. 14, the continuity evaluation unit 173a evaluates that the continuity of CU # 1 to CU # 5 among the CU # 1 to CU # 7 adjacent to the encoding target CU is low, and It is assumed that CUs whose energy calculated by the difference energy calculation unit 173c is smaller than the threshold are CU # 1 to CU # 3. That is, the energy calculated by the residual energy calculation unit 173c for CU # 4 to CU # 7 is larger than the threshold.

  In such a case, the correction unit 173b corrects the block boundary region of the inter prediction image of the coding target CU using the decoded images of the block boundary regions of CU # 1 to CU # 3. On the other hand, the correction unit 173b does not use the decoded image in the block boundary area of CU # 4 to CU # 7 for correcting the inter prediction image of the coding target CU.

  Note that, in the present modification, the image encoding device 1 has been described as an example, but the operation according to the present modification is also applicable to the image decoding device 2.

<Other embodiments>
In the above embodiments and modified examples, the case where the encoding target CU (or the decoding target CU) is divided into a plurality of small regions is not particularly considered. However, the present invention is also applicable when the encoding target CU (or decoding target CU) is divided into a plurality of small areas.

  For example, as illustrated in FIG. 15, when the inside of the encoding target CU is divided into small regions and different motion vectors can be applied to each small region, the continuity evaluation unit 173a determines the position closest to the adjacent decoded CU. The continuity of the motion between the motion vector applied to the small region and the motion vector applied to the adjacent decoded CU is evaluated.

  A program that causes a computer to execute each process performed by the image encoding device 1 and a program that causes a computer to execute each process performed by the image decoding device 2 may be provided. Further, the program may be recorded on a computer-readable medium. With a computer readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

  Further, a circuit for executing each processing performed by the image encoding device 1 may be integrated, and the image encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). Similarly, a circuit for executing each processing performed by the image decoding device 2 may be integrated, and the image decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

  As described above, the embodiments have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist.

1: Image encoding device 2: Image decoding device 100: Block dividing unit 110: Subtraction unit 120: Transform / quantization unit 121: Transform unit 122: Quantization unit 130: Entropy encoding unit 140: Inverse quantization / inverse transformation Unit 141: inverse quantization unit 142: inverse transformation unit 150: combining unit 160: memory unit 170: prediction unit 171: intra prediction unit 172: inter prediction unit 173: predicted image correction unit 173a: continuity evaluation unit 173b: correction unit 173c: residual energy calculation unit 174: switching unit 200: entropy decoding unit 210: inverse quantization / inverse transformation unit 211: inverse quantization unit 212: inverse transformation unit 220: combining unit 230: memory unit 240: prediction unit 241: Intra prediction unit 242: Inter prediction unit 243: Predicted image correction unit 243a: Continuity evaluation unit 243b: Correction unit 244: Switching unit

Claims (5)

A prediction image correction device that corrects a prediction image obtained by performing inter prediction for each block constituting an image,
By comparing a motion vector applied to the target block on which the inter prediction has been performed and a motion vector applied to one or more decoded adjacent blocks adjacent to the target block, the one or more adjacent blocks are compared. Continuity evaluation unit for evaluating the continuity of the motion with the target block,
When there is an adjacent block in which the continuity evaluated by the continuity evaluation unit is lower than a predetermined value, the target block and the decoded image of the adjacent block in which the continuity is lower than a predetermined value. A correction unit that corrects a predicted image of a boundary region with an adjacent block whose continuity is lower than a predetermined value among predicted images of the target block using a decoded image of the boundary region of the target block. Prediction image correction device. The continuity further includes a residual energy calculation unit that calculates energy of a prediction residual of a boundary region with the target block among prediction residuals of adjacent blocks lower than a predetermined value,
The correction unit performs the correction when there is an adjacent block whose continuity is lower than a predetermined value, and when the energy calculated by the residual energy calculation unit is smaller than a threshold value. The predictive image correction device according to claim 1, wherein   An image encoding device comprising: an inter prediction unit that performs inter prediction; and the predicted image correction device according to claim 1 or 2.   An image decoding device comprising: an inter prediction unit that performs inter prediction; and the predicted image correction device according to claim 1.   A program that causes a computer to function as the predicted image correction device according to claim 1.

Images