JP2020005200A - Image decoding device and image encoding device - Google Patents

Abstract

To provide an image decoding device and an image encoding device which solve a problem that OBMC processing increases memory band or the processing becomes complicated and heavy.SOLUTION: A prediction image generation part of an image decoding device includes: an intra prediction image generation part; and an inter prediction image generation part 309 for generating a prediction image for each sub-block. When the difference between a motion vector of a target sub-block and the motion vector of the sub-block adjacent to the target sub-block is equal to or smaller than a predetermined value, the prediction image generation part performs prediction image generation processing using OBMC processing. Moreover, when the target sub-block is a sub-block of a predetermined size or less and the target sub-block is a bidirectional prediction sub-block, the prediction image generation part generates the prediction image by assigning the same motion information as the motion information of the adjacent sub-block adjacent to the target sub-block to the target sub-block.SELECTED DRAWING: Figure 22

Description

  An embodiment of the present invention relates to a video decoding device and a video encoding device.

  In order to efficiently transmit or record a moving image, a moving image encoding device that generates encoded data by encoding a moving image, and a moving image that generates a decoded image by decoding the encoded data An image decoding device is used.

  Specific examples of the moving picture coding method include, for example, methods proposed in H.264 / AVC and HEVC (High-Efficiency Video Coding).

  In such a moving image coding method, an image (picture) constituting a moving image is a slice obtained by dividing the image, and a coding tree unit (CTU: Coding Tree Unit) obtained by dividing the slice. ), A coding unit obtained by dividing the coding tree unit (sometimes called a coding unit (CU)), and a prediction unit which is a block obtained by dividing the coding unit. (PU) and a transform unit (TU), and are managed and encoded / decoded for each CU.

  In addition, in such a moving image encoding method, a predicted image is generally generated based on a locally decoded image obtained by encoding / decoding an input image, and the predicted image is converted from the input image (original image). The prediction residual obtained by subtraction (sometimes called a “difference image” or a “residual image”) is encoded. As a method of generating a predicted image, there are an inter-screen prediction (inter prediction) and an intra-screen prediction (intra prediction).

  In addition, Non-Patent Literature 1 is a recent technique for encoding and decoding moving images.

"Algorithm Description of Joint Exploration Test Model 5", JVET-E1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 12-20 January 2017

  2. Description of the Related Art In recent video coding and decoding technologies, an OBMC (Overlapped block motion compensation) process may be applied to a motion compensation process when a predicted image is generated. The OBMC process uses an interpolated image generated using inter prediction parameters added to a target sub-block (eg, PU) and an interpolated image generated using motion parameters of sub-blocks adjacent to the target sub-block. This is a process for generating an interpolated image of the target sub-block. On the other hand, the OBMC process has a first problem that a memory band for accessing image data is increased.

  In addition, when performing bidirectional prediction for generating a predicted image using two reference pictures for a target sub-block having a small size, a second problem that a process of generating a predicted image is complicated and the process becomes heavy is disadvantageous. Had occurred.

  In order to solve the above problem, an image decoding device according to an aspect of the present invention is an image decoding device that decodes encoded data obtained by encoding a target image, and generates a predicted image for each sub-block. A prediction image generation unit that performs the OBMC when the difference between the motion vector of the target sub-block and the motion vector of the sub-block adjacent to the target sub-block is equal to or smaller than a predetermined value. A predicted image generation process using the process is performed.

  According to another embodiment of the present invention, there is provided an image decoding apparatus for decoding encoded data obtained by encoding a target image, wherein the image decoding apparatus generates a predicted image for each block. A prediction image generation unit, the prediction image generation unit, the target sub-block is a sub-block of a predetermined size or less, when the target sub-block is a bidirectional prediction sub-block, the target sub-block Then, the same motion information as the motion information of the adjacent sub-block adjacent to the target sub-block is assigned to generate the predicted image.

  Further, in order to solve the above problem, an image encoding device according to an aspect of the present invention is an image encoding device that encodes a residual between a predicted image and an encoding target image. A prediction image generation unit that generates a prediction image, wherein the prediction image generation unit determines that a difference between a motion vector of the target sub-block and a motion vector of a sub-block adjacent to the target block is equal to or smaller than a predetermined value. Next, a predicted image generation process using the OBMC process is performed.

  Further, in order to solve the above problem, an image encoding device according to an aspect of the present invention is an image encoding device that encodes a residual between a predicted image and an encoding target image. A prediction image generation unit that generates a prediction image, wherein the prediction image generation unit is configured to, when the target sub-block is a block of a predetermined size or less and the target sub-block is a bidirectional prediction sub-block, The same motion information as the motion information of the adjacent sub-block adjacent to the target sub-block is assigned to the block, and the predicted image is generated.

  According to the above configuration, at least one of the first and second problems can be solved.

FIG. 3 is a diagram illustrating a hierarchical structure of data of an encoded stream according to the embodiment. FIG. 4 is a conceptual diagram illustrating an example of a reference picture and a reference picture list. It is a block diagram showing the composition of the picture coding device concerning this embodiment. It is a schematic diagram showing a configuration of an image decoding device according to the present embodiment. It is a schematic diagram showing the composition of the merge prediction parameter derivation part concerning this embodiment. It is a schematic diagram showing the composition of the AMVP prediction parameter derivation part concerning this embodiment. It is a schematic diagram showing the composition of the inter prediction parameter encoding part of the picture encoding device concerning this embodiment. It is a schematic diagram showing the composition of the inter prediction picture generation part concerning this embodiment. It is a schematic diagram showing the composition of the inter prediction parameter decoding part concerning this embodiment. FIG. 13 is a diagram illustrating an example of deriving a motion vector spMvLX [xi] [yi] of each sub-block configuring a PU (width nPbW) for which a motion vector is predicted. (A) is a figure for demonstrating bilateral matching (Bilateral matching). (B) is a figure for explaining template matching (Template matching). FIG. 14 is a diagram illustrating an example of pseudo code representing a sub-block merging process performed by a sub-block prediction parameter derivation unit according to the present modification. FIG. 15 is a diagram illustrating an example of pseudo code representing a process performed by a sub-block prediction parameter derivation unit according to the present modification to determine whether to perform bidirectional prediction. FIG. 21 is a diagram illustrating another example of the pseudo code indicating the process performed by the sub-block prediction parameter derivation unit according to the present modification to determine whether to perform bidirectional prediction. FIG. 14 is a diagram illustrating an example of pseudo code representing a sub-block merging process performed by a sub-block prediction parameter derivation unit according to the present modification. FIG. 15 is a diagram illustrating another example of the pseudo code indicating the sub-block merging process performed by the sub-block prediction parameter derivation unit according to the present modification. FIG. 7 is a diagram illustrating an example of a region in which a predicted image is generated using a motion parameter of an adjacent PU according to the embodiment. FIG. 3 is a block diagram illustrating a main configuration of a motion compensation unit that performs an OBMC process according to the embodiment. 9 is a flowchart illustrating an example of a flow of a process of a motion compensation unit according to the OBMC process according to the embodiment. FIG. 5 is a diagram illustrating an example of pseudo code representing an OBMC process according to the embodiment. It is a figure explaining whether a motion parameter of an adjacent subblock is unknown. FIG. 14 is a block diagram illustrating a main configuration of a motion compensation unit that performs an OBMC process according to a modification. FIG. 21 is a diagram illustrating an example of a block obtained by merging a target sub-block according to the present modification with sub-blocks in the same PU adjacent to the target sub-block. FIG. 15 is a diagram illustrating an example of a sub-block in the same PU adjacent to a target sub-block to be merged according to the present modified example. FIG. 14 is a diagram illustrating an example of pseudo code representing an OBMC process according to the present modification. FIG. 14 is a diagram illustrating an example of a determination formula for an OBMC processing execution determination unit according to the present modification to derive mergeBlkFlag. FIG. 14 is a diagram illustrating another example of the pseudo code representing the OBMC process according to the modification. FIG. 14 is a diagram illustrating another example of a determination formula for the OBMC processing execution determination unit according to the modification to derive mergeBlkFlag. FIG. 14 is a diagram illustrating an example of a sub-block adjacent to a target sub-block according to the present modification. FIG. 14 is a diagram illustrating an example of pseudo code representing an OBMC process according to the present modification. 15 shows an example of a determination formula for an OBMC process execution determination unit according to the present modification to derive smallMVFlag. 15 shows another example of a determination formula for the OBMC processing execution determination unit according to the present modification to derive smallMVFlag. FIG. 14 is a diagram illustrating another example of the pseudo code representing the OBMC process according to the modification. FIG. 14 is a diagram illustrating an example of a determination formula for an OBMC process execution determination unit according to the present modification to derive smallMVFlag. FIG. 1 is a diagram illustrating a configuration of a transmission device equipped with an image encoding device according to the present embodiment and a reception device equipped with an image decoding device. (A) shows a transmitting device equipped with an image encoding device, and (b) shows a receiving device equipped with an image decoding device. FIG. 1 is a diagram illustrating a configuration of a recording device equipped with an image encoding device according to the present embodiment and a playback device equipped with an image decoding device. (A) shows a recording device equipped with an image encoding device, and (b) shows a reproducing device equipped with an image decoding device. 1 is a schematic diagram illustrating a configuration of an image transmission system according to an embodiment.

(1st Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 37 is a schematic diagram illustrating a configuration of the image transmission system 1 according to the present embodiment.

  The image transmission system 1 is a system that transmits a code obtained by encoding an encoding target image, decodes the transmitted code, and displays the image. The image transmission system 1 includes an image encoding device (moving image encoding device) 11, a network 21, an image decoding device (moving image decoding device) 31, and an image display device 41.

  An image T is input to the image encoding device 11.

  The network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31. The network 21 is the Internet, a wide area network (WAN: Wide Area Network), a small network (LAN: Local Area Network), or a combination thereof. The network 21 is not limited to a two-way communication network, but may be a one-way communication network for transmitting broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Further, the network 21 may be replaced with a storage medium that records the encoded stream Te such as a DVD (Digital Versatile Disc) and a BD (Blue-ray Disc).

  The image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates one or a plurality of decoded images Td respectively decoded.

  The image display device 41 displays all or a part of one or a plurality of decoded images Td generated by the image decoding device 31. The image display device 41 includes a display device such as a liquid crystal display and an organic EL (Electro-luminescence) display. When the image decoding device 31 has a high processing capability, an image with a high image quality is displayed. When the image decoding device 31 has only a low processing capability, an image with a low image quality is displayed.

<Operator>
The operators used in this specification are described below.

  >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, and | = is an OR assignment operator.

  x? y: z is a ternary operator that takes y when x is true (other than 0) and z when x is false (0).

  Clip3 (a, b, c) is a function that clips c to a value greater than or equal to a and less than or equal to b, and returns a if c <a, returns b if c> b, and returns b if c> b Is a function that returns c (where a <= b).

<Structure of encoded stream Te>
Prior to the detailed description of the image encoding device 11 and the image decoding device 31 according to the present embodiment, a data structure of an encoded stream Te generated by the image encoding device 11 and decoded by the image decoding device 31 will be described. .

  FIG. 1 is a diagram showing a hierarchical structure of data in an encoded stream Te. The coded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. 1A to 1F show an encoded video sequence defining a sequence SEQ, an encoded picture defining a picture PICT, an encoded slice defining a slice S, and an encoded slice defining slice data, respectively. FIG. 3 is a diagram illustrating data, a coding tree unit included in the coding slice data, and a coding unit (Coding Unit; CU) included in the coding tree unit.

(Encoded video sequence)
In the encoded video sequence, a set of data referred to by the image decoding device 31 to decode the sequence SEQ to be processed is defined. As shown in FIG. 1A, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an addition. Contains extended information SEI (Supplemental Enhancement Information).

  The video parameter set VPS includes, in a moving image composed of a plurality of layers, a set of encoding parameters common to a plurality of moving images and a plurality of layers included in the moving image and encoding parameters associated with each layer. Sets are defined.

  In the sequence parameter set SPS, a set of encoding parameters referred to by the image decoding device 31 to decode the target sequence is defined. For example, the width and height of a picture are defined. Note that a plurality of SPSs may exist. In that case, one of the plurality of SPSs is selected from the PPS.

  The picture parameter set PPS defines a set of encoding parameters that the image decoding device 31 refers to for decoding each picture in the target sequence. For example, a reference value (pic_init_qp_minus26) of a quantization width used for decoding a picture and a flag (weighted_pred_flag) indicating application of weighted prediction are included. Note that a plurality of PPSs may exist. In that case, any one of the plurality of PPSs is selected from each picture in the target sequence.

(Encoded picture)
In the coded picture, a set of data referred to by the image decoding device 31 for decoding the picture PICT to be processed is defined. As shown in FIG. 1B, the picture PICT includes slices S0 to SNS _-1 (NS is the total number of slices included in the picture PICT).

In the following, when it is not necessary to distinguish each of the slices S0 to SNS _-1 , the description may be omitted with suffixes of the reference numerals. The same applies to other data included in the coded stream Te described below and having a suffix.

(Coding slice)
In the coded slice, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed is defined. The slice S includes a slice header SH and slice data SDATA, as shown in FIG.

  The slice header SH includes an encoding parameter group referred to by the image decoding device 31 to determine a decoding method for the target slice. The slice type designation information (slice_type) that designates a slice type is an example of an encoding parameter included in the slice header SH.

  The slice types that can be designated by the slice type designation information include (1) an I-slice that uses only intra prediction for encoding, (2) a P slice that uses unidirectional prediction or intra prediction for encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding.

  Note that the slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the encoded video sequence.

(Encoded slice data)
In the encoded slice data, a set of data referred to by the image decoding device 31 to decode the slice data SDATA to be processed is defined. The slice data SDATA includes a coding tree unit (CTU: Coding Tree Unit) as shown in FIG. A CTU is a block of a fixed size (for example, 64 × 64) constituting a slice, and may be referred to as a maximum coding unit (LCU).

(Coding tree unit)
As shown in FIG. 1E, a set of data referred to by the image decoding device 31 to decode the coding tree unit to be processed is defined. The coding tree unit is divided by recursive quadtree division. A tree-structured node obtained by recursive quadtree division is referred to as a coding node (CN). The intermediate node of the quadtree is a coding node, and the coding tree unit itself is defined as the highest coding node. The CTU includes a split flag (cu_split_flag). When cu_split_flag is 1, the CTU is split into four coding nodes CN. When cu_split_flag is 0, the coding node CN is not divided and has one coding unit (CU: Coding Unit) as a node. The coding unit CU is a terminal node of the coding node and is not further divided. The encoding unit CU is a basic unit of the encoding process.

  Further, when the size of the coding tree unit CTU is 64 × 64 pixels, the size of the coding unit can take any of 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels.

(Encoding unit)
As shown in FIG. 1F, a set of data referred to by the image decoding device 31 for decoding the encoding unit to be processed is defined. Specifically, the coding unit includes a prediction tree, a transform tree, and a CU header CUH. The CU header specifies a prediction mode, a division method (PU division mode), and the like.

  In the prediction tree, prediction information (a reference picture index, a motion vector, etc.) of each prediction unit (PU) obtained by dividing the coding unit into one or a plurality is defined. In other words, a prediction unit is one or more non-overlapping regions that make up a coding unit. Further, the prediction tree includes one or a plurality of prediction units obtained by the above-described division. Hereinafter, a prediction unit obtained by further dividing the prediction unit is referred to as a “sub-block”. The sub-block is composed of a plurality of pixels. If the size of the prediction unit and the sub-block are equal, there is one sub-block in the prediction unit. If the prediction unit is larger than the size of the sub-block, the prediction unit is divided into sub-blocks. For example, when the prediction unit is 8 × 8 and the sub-block is 4 × 4, the prediction unit is divided into four sub-blocks, which are divided into two horizontally and two vertically.

  The prediction process may be performed for each prediction unit (sub-block).

  Broadly speaking, there are two types of splits in the prediction tree: intra prediction and inter prediction. Intra prediction refers to prediction within the same picture, and inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images).

  In the case of intra prediction, the division method includes 2Nx2N (the same size as the coding unit) and NxN.

Also, in the case of inter prediction, the division method is encoded by the PU division mode (part_mode) of the encoded data, and 2Nx2N (the same size as the encoding unit), 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and NxN and others. Note that 2NxN and Nx2N indicate a 1: 1 symmetric division,
2NxnU, 2NxnD and nLx2N, nRx2N show asymmetric splits of 1: 3, 3: 1. The PUs included in the CU are expressed as PU0, PU1, PU2, and PU3 in order.

  In the transform tree, the coding unit is divided into one or more transform units, and the position and size of each transform unit are defined. In other words, a transform unit is one or more non-overlapping regions that make up a coding unit. Further, the transform tree includes one or a plurality of transform units obtained by the above division.

  The division in the transform tree includes one in which an area having the same size as the encoding unit is assigned as the transform unit, and one in which the recursive quadtree division is performed as in the above-described CU division.

  The conversion process is performed for each conversion unit.

(Forecast parameters)
A prediction image of a prediction unit (PU: Prediction Unit) is derived by a prediction parameter attached to the PU. The prediction parameter includes a prediction parameter for intra prediction or a prediction parameter for inter prediction. Hereinafter, the prediction parameters of the inter prediction (inter prediction parameters) will be described. The inter prediction parameter includes a prediction list use flag predFlagL0, predFlagL1, a reference picture index refIdxL0, refIdxL1, and a motion vector mvL0, mvL1. The prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether reference picture lists called L0 list and L1 list are used, respectively. When the value is 1, the corresponding reference picture list is used. Note that in this specification, when a “flag indicating whether or not XX” is described, a flag other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. Treat 1 as true and 0 as false (the same applies hereinafter). However, other values can be used as a true value and a false value in an actual device or method.

  Syntax elements for deriving the inter prediction parameter included in the encoded data include, for example, PU division mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, There is a difference vector mvdLX.

(Reference picture list)
The reference picture list is a list including reference pictures stored in the reference picture memory 306. FIG. 2 is a conceptual diagram illustrating an example of a reference picture and a reference picture list. In FIG. 2A, a rectangle is a picture, an arrow is a picture reference relationship, a horizontal axis is time, I, P, and B in the rectangle are intra pictures, uni-prediction pictures, bi-prediction pictures, and numbers in the rectangles are decoding. Indicates the order. As shown in the figure, the decoding order of pictures is I0, P1, B2, B3, B4, and the display order is I0, B3, B2, B4, P1. FIG. 2B shows an example of the reference picture list. The reference picture list is a list representing reference picture candidates, and one picture (slice) may have one or more reference picture lists. In the illustrated example, the target picture B3 has two reference picture lists, an L0 list RefPicList0 and an L1 list RefPicList1. When the current picture is B3, reference pictures are I0, P1, and B2, and the reference picture has these pictures as elements. In each prediction unit, which picture in the reference picture list RefPicListX is actually referred to is specified by the reference picture index refIdxLX. The figure shows an example in which reference pictures P1 and B2 are referenced by refIdxL0 and refIdxL1.

(Merge prediction and AMVP prediction)
The prediction parameter decoding (encoding) method includes a merge prediction (merge) mode and an AMVP (Adaptive Motion Vector Prediction) mode. A merge flag merge_flag is a flag for identifying these. The merge prediction mode is a mode in which a prediction list use flag predFlagLX (or an inter prediction identifier inter_pred_idc), a reference picture index refIdxLX, and a motion vector mvLX are not included in encoded data, and are derived from prediction parameters of already processed neighboring PUs. The AMVP mode is a mode in which an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, and a motion vector mvLX are included in encoded data. The motion vector mvLX is encoded as a prediction vector index mvp_LX_idx for identifying the prediction vector mvpLX and a difference vector mvdLX.

  The inter prediction identifier inter_pred_idc is a value indicating the type and number of reference pictures, and takes one of PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate that a reference picture managed by a reference picture list of an L0 list and an L1 list is used, respectively, and indicate that one reference picture is used (uni-prediction). PRED_BI indicates that two reference pictures are used (bi-prediction BiPred), and uses reference pictures managed in an L0 list and an L1 list. The prediction vector index mvp_LX_idx is an index indicating a prediction vector, and the reference picture index refIdxLX is an index indicating a reference picture managed in a reference picture list. Note that LX is a description method used when L0 prediction and L1 prediction are not distinguished, and the parameters for the L0 list and the parameters for the L1 list are distinguished by replacing LX with L0 and L1.

  The merge index merge_idx is an index indicating whether any prediction parameter among prediction parameter candidates (merge candidates) derived from the processed PU is used as the prediction parameter of the decoding target PU.

(Motion vector)
The motion vector mvLX indicates a shift amount between blocks on two different pictures. The prediction vector and the difference vector related to the motion vector mvLX are called a prediction vector mvpLX and a difference vector mvdLX, respectively.

(Inter prediction identifier inter_pred_idc and prediction list use flag predFlagLX)
The relationship between the inter prediction identifier inter_pred_idc and the prediction list use flags predFlagL0 and predFlagL1 is as follows, and can be mutually converted.

inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
As the inter prediction parameter, a prediction list use flag or an inter prediction identifier may be used. Further, the determination using the prediction list use flag may be replaced with a determination using an inter prediction identifier. Conversely, the determination using the inter prediction identifier may be replaced with a determination using a prediction list use flag.

(Judgment of bi-prediction biPred)
The flag biPred as to whether it is bi-prediction BiPred can be derived based on whether both of the two prediction list use flags are “1”. For example, it can be derived by the following equation.

biPred = (predFlagL0 == 1 && predFlagL1 == 1)
The flag biPred can also be derived based on whether or not the inter prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived by the following equation.

biPred = (inter_pred_idc == PRED_BI)? 1: 0
The above equation can also be expressed by the following equation.

biPred = (inter_pred_idc == PRED_BI)
Note that a value of, for example, 3 can be used for PRED_BI.

(Configuration of image decoding device)
Next, the configuration of the image decoding device 31 according to the present embodiment will be described. FIG. 4 is a schematic diagram illustrating a configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, and a reverse. It is configured to include a quantization / inverse transforming unit 311 and an adding unit 312.

  Further, the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The prediction image generation unit 308 includes an inter prediction image generation unit 309 and an intra prediction image generation unit 310.

  The entropy decoding unit 301 performs entropy decoding on the encoded stream Te input from the outside, and separates and decodes individual codes (syntax elements). The separated codes include prediction information for generating a prediction image, residual information for generating a difference image, and the like.

  Entropy decoding section 301 outputs a part of the separated code to prediction parameter decoding section 302. The part of the separated code is, for example, prediction mode predMode, PU division mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX. Control of which code is to be decoded is performed based on an instruction from the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantized coefficients to the inverse quantization / inverse transform unit 311. In the encoding process, the quantized coefficients are used for DCT (Discrete Cosine Transform, Discrete Cosine Transform), DST (Discrete Sine Transform, Discrete Sine Transform), KLT (Karyhnen Loeve Transform, Karhunen-Loeve transform) Is a coefficient obtained by performing frequency conversion and quantization.

  The inter prediction parameter decoding unit 303 decodes the inter prediction parameters based on the code input from the entropy decoding unit 301 with reference to the prediction parameters stored in the prediction parameter memory 307.

  The inter prediction parameter decoding unit 303 outputs the decoded inter prediction parameter to the prediction image generation unit 308, and stores the decoded inter prediction parameter in the prediction parameter memory 307. Details of the inter prediction parameter decoding unit 303 will be described later.

  Intra prediction parameter decoding section 304 decodes the intra prediction parameters with reference to the prediction parameters stored in prediction parameter memory 307 based on the code input from entropy decoding section 301. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameters to the prediction image generation unit 308, and stores the decoded intra prediction parameters in the prediction parameter memory 307.

  The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adding unit 312.

  The reference picture memory 306 stores the decoded image of the CU generated by the adding unit 312 at a position predetermined for each decoding target picture and CU.

  The prediction parameter memory 307 stores the prediction parameters at predetermined positions for each picture to be decoded and each prediction unit (or sub-block, fixed-size block, pixel). Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode predMode separated by the entropy decoding unit 301. . The stored inter prediction parameters include, for example, a prediction list use flag predFlagLX (inter prediction identifier inter_pred_idc), a reference picture index refIdxLX, and a motion vector mvLX.

  The prediction mode generation unit 308 receives the prediction mode predMode input from the entropy decoding unit 301 and receives the prediction parameters from the prediction parameter decoding unit 302. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a PU or a sub-block in the prediction mode indicated by the prediction mode predMode, using the input prediction parameters and the read reference picture (reference picture block).

  Here, when the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture (reference picture block) to perform the inter prediction. Generates a predicted image of a PU or a sub-block.

  The inter-prediction image generation unit 309 generates, for a reference picture list (L0 list or L1 list) whose prediction list use flag predFlagLX is 1, a motion vector based on a decoding target PU from a reference picture indicated by a reference picture index refIdxLX. The reference picture block at the position indicated by mvLX is read from the reference picture memory 306. The inter-prediction image generation unit 309 performs prediction based on the read reference picture block to generate a PU prediction image. The inter prediction image generation unit 309 outputs the generated prediction image of the PU to the addition unit 312. Here, a reference picture block is a set of pixels on a reference picture (usually called a block because it is rectangular), and is a region to be referred to in order to generate a predicted image of a PU or a sub-block.

  When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs the intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the read reference pictures.

  The inverse quantization / inverse transform unit 311 inversely quantizes the quantized coefficient input from the entropy decoding unit 301 to obtain a transform coefficient. The inverse quantization / inverse transform unit 311 performs an inverse frequency transform such as an inverse DCT, an inverse DST, an inverse KLT, etc. on the obtained transform coefficient, and calculates a residual signal. The inverse quantization / inverse transforming unit 311 outputs the calculated residual signal to the adding unit 312.

  The addition unit 312 adds, for each pixel, the PU prediction image input from the inter prediction image generation unit 309 or the intra prediction image generation unit 310 and the residual signal input from the inverse quantization / inverse conversion unit 311. Generate a PU decoded image. The addition unit 312 stores the generated PU decoded image in the reference picture memory 306, and outputs a decoded image Td obtained by integrating the generated PU decoded image for each picture to the outside.

(Configuration of inter prediction parameter decoding unit)
Next, the configuration of inter prediction parameter decoding section 303 will be described.

  FIG. 9 is a schematic diagram illustrating a configuration of the inter prediction parameter decoding unit 303 according to the present embodiment. The inter prediction parameter decoding unit 303 includes an inter prediction parameter decoding control unit 3031, an AMVP prediction parameter deriving unit 3032, an adding unit 3038, a merge prediction parameter deriving unit 3036, and a sub-block prediction parameter deriving unit 3037.

  The inter prediction parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode a code (syntax element) related to inter prediction, and codes (syntax elements) included in the encoded data, for example, the PU division mode part_mode , Merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, and difference vector mvdLX.

  First, the inter prediction parameter decoding control unit 3031 extracts a merge flag merge_flag. When the inter prediction parameter decoding control unit 3031 expresses that a certain syntax element is to be extracted, it means that the decoding of a certain syntax element is instructed to the entropy decoding unit 301 and the corresponding syntax element is read from the encoded data. I do.

  When the merge flag merge_flag is 0, that is, indicating the AMVP prediction mode, the inter prediction parameter decoding control unit 3031 extracts the AMVP prediction parameter from the encoded data using the entropy decoding unit 301. The AMVP prediction parameters include, for example, an inter prediction identifier inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference vector mvdLX. The AMVP prediction parameter deriving unit 3032 derives a prediction vector mvpLX from the prediction vector index mvp_LX_idx. Details will be described later. Inter prediction parameter decoding control section 3031 outputs difference vector mvdLX to addition section 3038. The adding unit 3038 adds the prediction vector mvpLX and the difference vector mvdLX to derive a motion vector.

  When the merge flag merge_flag is 1, that is, indicating the merge prediction mode, the inter prediction parameter decoding control unit 3031 extracts a merge index merge_idx as a prediction parameter related to merge prediction. The inter prediction parameter decoding control unit 3031 outputs the extracted merge index merge_idx to the merge prediction parameter derivation unit 3036 (details will be described later), and outputs the sub-block prediction mode flag subPbMotionFlag to the sub-block prediction parameter derivation unit 3037. The sub-block prediction parameter deriving unit 3037 divides the PU into a plurality of sub-blocks according to the value of the sub-block prediction mode flag subPbMotionFlag, and derives a motion vector for each sub-block. That is, in the sub-block prediction mode, a prediction block is predicted in a small block unit of 4 × 4 or 8 × 8. In an image encoding device 11 described below, a CU is divided into a plurality of partitions (PUs such as 2NxN, Nx2N, and NxN), and a sub-block prediction mode is used for encoding a prediction parameter syntax in partition units. Since a plurality of sub-blocks are grouped into a set and the syntax of the prediction parameter is encoded for each set, motion information of many sub-blocks can be encoded with a small code amount.

  More specifically, the sub-block prediction parameter deriving unit 3037 performs at least one of the spatio-temporal sub-block prediction unit 30371, the affine prediction unit 30372, and the matching motion derivation unit 30373 that performs sub-block prediction in the sub-block prediction mode. Prepare.

(Sub-block prediction mode flag)
Here, a method of deriving a sub-block prediction mode flag subPbMotionFlag indicating whether the prediction mode of a certain PU is the sub-block prediction mode in the image decoding device 31 and the image encoding device 11 (details will be described later) will be described. . The image decoding device 31 and the image encoding device 11 set a sub-block prediction mode flag subPbMotionFlag based on which of a spatial sub-block prediction SSUB, a temporal sub-block prediction TSUB, an affine prediction AFFINE, and a matching motion derivation MAT described later is used. Derive. For example, when the prediction mode selected by a certain PU is N (for example, N is a label indicating the selected merge candidate), the sub-block prediction mode flag subPbMotionFlag may be derived by the following equation.

subPbMotionFlag = (N == TSUB) || (N == SSUB) || (N == AFFINE) || (N == MAT)
Here, || indicates a logical sum (the same applies hereinafter).

(Sub-block predictor)
Next, the sub-block prediction unit will be described.

(Spatiotemporal sub-block prediction unit 30371)
The spatio-temporal sub-block prediction unit 30371 calculates the target PU from the motion vector of the PU on the reference image temporally adjacent to the target PU (for example, the immediately preceding picture) or the motion vector of the PU spatially adjacent to the target PU. Is derived to obtain a motion vector of a sub-block obtained by dividing. Specifically, by scaling the motion vector of the PU on the reference image according to the reference picture referenced by the target PU, the motion vector spMvLX [xi] [yi] of each sub-block in the target PU (xi = xPb + nSbW * i, yj = yPb + nSbH * j, i = 0, 1, 2, ..., nPbW / nSbW-1, j = 0, 1, 2, ..., nPbH / nSbH-1) Deriving (temporal sub-block prediction). Here, (xPb, yPb) is the upper left coordinate of the target PU, nPbW, nPbH is the size of the target PU, and nSbW, nSbH are the sizes of the sub-blocks.

  Also, by calculating a weighted average according to the distance between the motion vector of the PU adjacent to the target PU and the sub-block obtained by dividing the target PU, the motion vector spMvLX [ xi] [yi] (xi = xPb + nSbW * i, yj = yPb + nSbH * j, i = 0, 1, 2, ..., nPbW / nSbW-1, j = 0, 1, 2, ... -NPbH / nSbH-1) may be derived (spatial sub-block prediction).

  The candidate TSUB for temporal sub-block prediction and the candidate SSUB for spatial sub-block prediction are selected as one of the merge modes (merge candidate).

(Affine prediction unit)
The affine prediction unit 30372 derives affine prediction parameters of the target PU. In the present embodiment, motion vectors (mv0_x, mv0_y) (mv1_x, mv1_y) of two control points (V0, V1) of the target PU are derived as affine prediction parameters. Specifically, the motion vector of each control point may be derived by predicting from the motion vector of the PU adjacent to the target PU, or the prediction vector and the coded data The motion vector of each control point may be derived from the sum of the difference vectors derived from.

  FIG. 10 shows an example in which the motion vector spMvLX of each sub-block constituting the target PU (nPbW × nPbH) is derived from the motion vector (mv0_x, mv0_y) of the control point V0 and the motion vector (mv1_x, mv1_y) of V1. FIG. As shown in FIG. 10, the motion vector spMvLX of each sub-block is derived as a motion vector for each point located at the center of each sub-block.

(Matching motion deriving unit 30373)
The matching motion deriving unit 30373 derives a motion vector spMvLX of a block or a sub-block constituting the PU by performing either a matching process of bilateral matching or template matching. FIG. 11 is a diagram for explaining (a) Bilateral matching and (b) Template matching. The matching motion derivation mode is selected as one merge candidate (matching candidate) in the merge mode.

  The matching motion deriving unit divides the target PU into a plurality of sub-blocks and performs bilateral matching or template matching described below in units of the divided sub-blocks, thereby obtaining motion vectors spMvLX [xi] [yi] (xi = xPb + nSbW * i, yj = yPb + nSbH * j, i = 0, 1, 2, ..., nPbW / nSbW-1, j = 0, 1, 2, ..., nPbH / nSbH-1 ) Is derived.

(Modification of Sub-block Prediction Parameter Deriving Unit 3037)
Next, a modification of the sub-block prediction parameter deriving unit 3037 will be described with reference to FIGS.

  The sub-block prediction parameter deriving unit 3037 according to the present modification performs the following processing when the target sub-block is a sub-block having a predetermined size or less and the target sub-block is a bidirectional prediction sub-block. The sub-block prediction parameter deriving unit 3037 assigns the same motion information as the motion information of the adjacent sub-block adjacent to the current sub-block to the current sub-block.

  According to the above configuration, the sub-block prediction parameter deriving unit 3037 allocates the same motion information as the adjacent sub-block to the target sub-block to the target sub-block having a predetermined size or less. Therefore, the inter-prediction image generation unit 309 can perform bidirectional prediction with the target sub-block and the adjacent sub-block having a predetermined size or less as the same block. In other words, the sub-block prediction parameter deriving unit 3037 can be expressed as merging (combining) a target sub-block having a predetermined size or less with an adjacent sub-block. For example, in the present modification, when a target sub-block is bidirectionally predicted in a configuration in which a motion vector is derived for each 4 × 4 unit sub-block such as ATMVP or the like, a 4 × 4 unit target sub-block Merge with × 4 adjacent sub-blocks.

  Therefore, there is an effect that it is possible to reduce processing in generating a predicted image using bidirectional prediction.

(Processing example 1 of sub-block prediction parameter deriving unit 3037)
Next, an example of processing performed by the sub-block prediction parameter deriving unit 3037 will be described with reference to FIGS. FIG. 12 is a diagram illustrating an example of pseudo code representing a sub-block merging process performed by the sub-block prediction parameter deriving unit 3037 according to the present example. In FIG. 12, (nPbW, nPbH) indicates the block size of the block including the target sub-block (the size of the target PU). Also, in FIG. 12, (subX, subY) indicates the position (subblock unit) of the target subblock in the target block (target PU). . As illustrated in FIG. 12, when the target sub-block is a sub-block smaller than 4 × 4 units and bidirectional prediction (bi-prediction) is specified for the target sub-block, the sub-block prediction parameter deriving unit 3037 , The target sub-block and the adjacent sub-block are merged. For example, the sub-block prediction parameter deriving unit 3037 may derive a bi-prediction flag BiPred indicating that bi-prediction is applied to the target sub-block by the following equation (determining whether or not to perform bi-prediction). May be). Note that (xSb, ySb) indicates the upper left coordinates of the target sub-block (for example, the target PU) in pixel units.

bipred = (predFlagL0 [xSb] [ySb] == 1 && predFlagL1 [xSb] [ySb] == 1)? 1: 0
Here, (xSb, ySb) indicates the position of the target sub-block in pixel units.

  Further, the sub-block prediction parameter deriving unit 3037 may determine whether or not to perform bi-directional prediction on every other sub-block according to the scanning order of the sub-blocks. In this case, the sub-block prediction parameter deriving unit 3037 may derive a bi-prediction flag BiPred indicating that bi-prediction is applied to the target sub-block by the following equation. Note that nSbW and nSbH indicate the size of the sub-block.

((xSb% (2 * nSbW)) && predFlagL0 [xSb] [ySb] == 1 && predFlagL1 [xSb] [ySb] == 1)
FIG. 13 is a diagram illustrating an example of pseudo code representing a process performed by the sub-block prediction parameter deriving unit 3037 to determine whether to perform bidirectional prediction for every other sub-block.

  FIG. 14 is a diagram illustrating another example of pseudo code indicating a process performed by the sub-block prediction parameter deriving unit 3037 to determine whether to perform bidirectional prediction for every other sub-block. In the example illustrated in FIG. 14, the sub-block prediction parameter deriving unit 3037 performs an example of a process performed when bidirectional prediction is applied to the current sub-block and the motion vector of the current sub-block is different from the motion vector of the adjacent sub-block. It is shown. In this example, the sub-block prediction parameter deriving unit 3037 merges the current sub-block and the adjacent sub-block by allocating the same motion vector to the current sub-block and the adjacent sub-block. Further, as shown in FIG. 14, the sub-block prediction parameter deriving unit 3037 refers to the right adjacent sub-block for every other sub-block (8 × 4 unit) and determines whether or not to perform merging.

(Example 1 of motion vector assigned to merged sub-block)
Next, an example of a motion vector assigned to the merged sub-block will be described with reference to FIGS. FIGS. 15 and 16 are diagrams showing an example of pseudo code representing a sub-block merging process performed by the sub-block prediction parameter deriving unit 3037 according to the present example.

  In the example illustrated in FIG. 15, the sub-block prediction parameter deriving unit 3037 allocates a motion vector added to the current sub-block to an adjacent sub-block. Note that the sub-block prediction parameter deriving unit 3037 may assign the motion vector added to the adjacent sub-block to the target sub-block. In this example, the common motion vectors cMvL0 and cMvL1 allocated to the target sub-block and the adjacent sub-block can be represented by the following equations.

mvL0 [xSb] [ySb] = mvL0 [xSb + nSbW] [ySb] = cMvL0
mvL1 [xSb] [ySb] = mvL1 [xSb + nSbW] [ySb] = cMvL1
predFlagL0 [xSb + 1] [ySb] = 1
predFlagL1 [xSb + 1] [ySb] = 1
In the example illustrated in FIG. 16, the sub-block prediction parameter deriving unit 3037 allocates the same motion vector to the target sub-block and the adjacent sub-block. For example, the average vector of the motion vector of the target sub-block and the motion vector of the adjacent sub-block is assigned to the target sub-block and the adjacent sub-block.

  According to the above configuration, the average of the motion vector of the target sub-block and the motion vector of the adjacent sub-block is calculated for the target sub-block having the predetermined size or less and the adjacent sub-block. Assign a vector. That is, the target sub-block and the adjacent sub-block having a predetermined size or less can be bidirectionally predicted as the same sub-block. Therefore, there is an effect that the processing of generating a predicted image using bidirectional prediction can be reduced.

  In this example, the common motion vectors cMvL0 and cMvL1 allocated to the target sub-block and the adjacent sub-block can be represented by the following equations.

cMvL0 = (mvL0 [xSb] [ySb] + mvL0 [xSb + nSbW] [ySb] +1) >> 1
cMvL1 = (mvL1 [xSb] [ySb] + mvL1 [xSb + nSbW] [ySb] +1) >> 1
FIG. 5 is a schematic diagram illustrating a configuration of the merge prediction parameter deriving unit 3036 according to the present embodiment. The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361, a merge candidate selection unit 30362, and a merge candidate storage unit 30363. The merge candidate storage unit 30363 stores the merge candidate input from the merge candidate derivation unit 30361. The merging candidate includes a prediction list use flag predFlagLX, a motion vector mvLX, and a reference picture index refIdxLX. In the merge candidate storage unit 30363, an index is assigned to the stored merge candidate according to a predetermined rule.

  The merge candidate deriving unit 30361 derives a merge candidate using the motion vector of the adjacent PU already decoded and the reference picture index refIdxLX as they are.

(Spatial merge candidate derivation processing)
As a spatial merge candidate deriving process, the merge candidate deriving unit 30361 reads and reads prediction parameters (prediction flag use flag predFlagLX, motion vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule. The predicted parameters are derived as merge candidates. The prediction parameter to be read is a prediction parameter relating to each of the PUs (for example, all or a part of the PUs respectively in contact with the lower left end, the upper left end, and the upper right end of the decoding target PU) within a predetermined range from the decoding target PU. is there.

(Time merge candidate derivation process)
As a temporal merge derivation process, the merge candidate derivation unit 30361 reads out the prediction parameters of the PU in the reference image including the coordinates of the lower right of the decoding target PU from the prediction parameter memory 307 and sets them as merge candidates. The reference image may be specified by, for example, the reference picture index refIdxLX specified in the slice header, or may be specified using the smallest reference picture index refIdxLX of the PU adjacent to the decoding target PU.

  The merge candidate derived by the merge candidate deriving unit 30361 is stored in the merge candidate storage unit 30363.

  The merge candidate selection unit 30362 determines, from among the merge candidates stored in the merge candidate storage unit 30363, a merge candidate to which an index corresponding to the merge index merge_idx input from the inter prediction parameter decoding control unit 3031 is assigned, as a target PU. Is selected as an inter prediction parameter. The merge candidate selection unit 30362 stores the selected merge candidate in the prediction parameter memory 307 and outputs the selected merge candidate to the prediction image generation unit 308.

  FIG. 6 is a schematic diagram illustrating a configuration of the AMVP prediction parameter deriving unit 3032 according to the present embodiment. The AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit 3033, a vector candidate selection unit 3034, and a vector candidate storage unit 3035. The vector candidate derivation unit 3033 derives a prediction vector candidate from the already processed PU motion vector mvLX stored in the prediction parameter memory 307 based on the reference picture index refIdx, and a vector candidate storage unit 3035 prediction vector candidate list mvpListLX [] To be stored.

  The vector candidate selection unit 3034 selects the motion vector mvpListLX [mvp_LX_idx] indicated by the prediction vector index mvp_LX_idx among the prediction vector candidates of the prediction vector candidate list mvpListLX [] as the prediction vector mvpLX. The vector candidate selection unit 3034 outputs the selected prediction vector mvpLX to the addition unit 3038.

  Note that the prediction vector candidate is a PU for which decoding processing has been completed, and is derived by scaling a motion vector of a PU (for example, an adjacent PU) in a predetermined range from the decoding target PU. Note that the adjacent PU includes, in addition to the PU spatially adjacent to the decoding target PU, for example, the left PU and the upper PU, a region temporally adjacent to the decoding target PU, for example, the same position as the decoding target PU. Includes regions obtained from prediction parameters of PUs with different times.

  The adding unit 3038 calculates the motion vector mvLX by adding the prediction vector mvpLX input from the AMVP prediction parameter deriving unit 3032 and the difference vector mvdLX input from the inter prediction parameter decoding control unit 3031. The addition unit 3038 outputs the calculated motion vector mvLX to the prediction image generation unit 308 and the prediction parameter memory 307.

(Inter prediction image generation unit 309)
FIG. 8 is a schematic diagram illustrating a configuration of the inter prediction image generation unit 309 included in the prediction image generation unit 308 according to the present embodiment. The inter prediction image generation unit 309 includes a motion compensation unit (prediction image generation device) 3091 and a weight prediction unit 3094.

(Motion compensation)
The motion compensation unit 3091 reads the reference picture index refIdxLX from the reference picture memory 306 based on the inter prediction parameters (predFlagLX, reference picture index refIdxLX, motion vector mvLX) input from the inter prediction parameter decoding unit 303. In the reference picture RefX specified by, starting from the position of the decoding target PU, a block located at a position shifted by the motion vector mvLX is read to generate an interpolation image (motion-compensated images predSamplesLX). Here, when the precision of the motion vector mvLX is not integer precision, a filter called a motion compensation filter for generating a pixel at a decimal position is applied to generate a motion compensated image.

(OBMC processing)
A general OBMC process will be described with reference to FIGS.

(Overview of OBMC processing)
The motion compensation unit 3091 according to the present embodiment may generate a predicted image using OBMC (Overlapped block motion compensation) processing. Here, the OBMC process will be described. The OBMC processing includes an interpolation image (PU interpolation image) generated using an inter prediction parameter (hereinafter, motion parameter) added to a target sub-block (target PU), and a motion parameter of a sub-block adjacent to the target sub-block. This is a process of generating an interpolated image (motion-compensated image) of the target sub-block by using an interpolated image (OBMC interpolated image) generated using. The OBMC corresponds to both the case where the sub-block prediction mode flag subPbMotionFlag is 0 (when performing prediction in units of blocks) and the case where the sub-block prediction mode flag subPbMotionFlag is 1 (when performing prediction in units of sub-blocks). In the following description, when prediction is performed in units of sub-blocks, the PU is read as a sub-block.

  In particular, for a pixel (boundary pixel) in the target PU that is short in distance from the boundary between PUs, a process of correcting the interpolated image of the target PU may be performed using an OBMC interpolated image based on the motion parameters of the adjacent PU.

  FIG. 17 is a diagram illustrating an example of a region in which a predicted image is generated by using a motion parameter of an adjacent PU according to the present embodiment. As shown in FIG. 17, when the OBMC process is applied to the generation of a predicted image, pixels within a predetermined distance from a PU boundary filled with black are applied to the OBMC process.

  Since the shape of the target PU and the shape of the adjacent PU are not necessarily the same, it is preferable that the OBMC process is performed in units of sub-blocks obtained by dividing the PU. The size of the sub-block can take various values from 4 × 4, 8 × 8 to PU size. At this time, the OBMC interpolation image generation and correction processing using the motion parameters of the adjacent sub-block at the boundary pixel of the sub-block is called OBMC processing.

(Interpolated image generation)
FIG. 18 is a block diagram illustrating a main configuration of a motion compensation unit 3091 included in the inter prediction image generation unit 309 that performs the OBMC process according to the present embodiment. As shown in FIG. 18, the motion compensation unit 3091 includes an interpolation image generation unit 3092 (a PU interpolation image generation unit 30911 and an OBMC interpolation image generation unit 30912) and an OBMC correction unit 3093.

  The interpolated image generation unit 3092 derives an interpolated image based on inter prediction parameters (prediction flag use flag predFlagLX, reference picture index refIdxLX, motion vector mvLX, OBMC flag OBMC_flag).

  The PU interpolation image generation unit 30911 includes a prediction list use flag predFlagLX [xPb] [yPb] input from the inter prediction parameter decoding unit 303, a reference picture index refIdxLX [xPb] [yPb], and a motion vector mvLX [xPb] [yPb]. , A PU interpolation image Pred_C [x] [y] (x = 0..nPbW-1, y = 0..nPbH-1) is generated. The PU interpolation image generation unit 30911 transmits the generated PU interpolation image Pred_C [x] [y] to the OBMC correction unit 3093. (XPb, yPb) is the upper left coordinate of the PU, and nPbW, nPbH are the width and height of the PU. Here, the suffix C means current.

  In other words, an interpolation image is generated by applying the motion information of the target prediction unit (PU) to a sub-block on the reference image corresponding to the target sub-block.

  Also, the OBMC interpolation image generation unit 30912 receives the OBMC_flag, the prediction list use flag predFlagLXN [xNb] [yNb], the reference picture index refIdxLXN [xNb] [yNb] of the adjacent reference block input from the inter prediction parameter decoding unit 303. An OBMC interpolation image Pred_N [x] [y] (x = 0..nPbW-1, y = 0..nPbH-1) is generated based on the motion vector mvLXN [xNb] [yNb] of the adjacent reference block.

  In other words, the additional interpolation image is generated by applying the motion information of the adjacent sub-block adjacent to the target sub-block to the sub-block on the reference image corresponding to the target sub-block. The OBMC interpolation image generation unit 30912 transmits the generated OBMC interpolation image Pred_N [x] [y] to the OBMC correction unit 3093. Note that (xNb, yNb) is the position of a sub-block adjacent to the PU. Here, the suffix N means neighbor.

  In the above description, the PU interpolated image generation unit 30911 and the OBMC interpolated image generation unit 30912 are distinguished. However, since both perform the process of generating the interpolated image from the motion parameters, there is one means for performing the two processes. May be prepared and executed by this means.

  When the motion vector mvLX or the motion vector mvLXN input to the interpolation image generation unit 3092 is not 1-M pixel accuracy (M is a natural number of 2 or more) instead of integer accuracy, the interpolation image generation unit 3092 (PU interpolation image generation unit 30911) , An OBMC interpolated image generation unit 30912) generates an interpolated image from the pixel value of the reference image at the integer pixel position by using an interpolation filter.

  If the motion vector mvLX is not of integer precision, the interpolated image generation unit 3092 determines the filter coefficient mcFilter [nFrac] [k] (k = 0..NTAP-1) of the NTAP tap corresponding to the phase nFrac and the pixel of the reference image. From the product-sum operation, the above-described PU interpolation image Pred_C [x] [y] or the OBMC interpolation image Pred_N [x] [y] is generated.

  Specifically, the interpolation image generation unit 3092 specifies the upper left coordinates (xb, yb), size (nW, nH), motion vector mvLX, reference image refImg (pointed by the reference picture index refIdxLX) of the prediction block (PU or sub-block). A reference image), an interpolation filter coefficient mcFilter [], and an interpolation filter tap number NTAP are used as input parameters to generate an interpolation image.

  In deriving the PU interpolation image Pred_C [x] [y], the interpolation image generation unit 3092 calculates (xb, yb) = (xPb, yPb), (nW, nH) = (nPbW, nPbH), mvLX = target PU ( The motion vector mvLX [xb] [yb] of the target sub-block), the reference picture indicated by the reference image refImg = refIdxLX [xb] [yb], the motion vector accuracy M, and the number of taps NTAP are used as input parameters. When the PU is divided into sub-blocks and processed, (xb, yb) = (xPb + nSbW * i, yPb + nSbH * j) (where i = 0, 1, 2,... NPbW / nSbW− 1, j = 0, 1, 2,... NPbH / nSbH-1), (nW, nH) = (nSbW, nSbH) are set. In deriving the OBMC interpolation image Pred_N [x] [y], the interpolation image generation unit 3092 calculates (xb, yb) = (xPb + nSbW * i, yPb + nSbH * j) (where i = 0, 1, 2, ... nPbW / nSbW-1, j = 0,1,2, ... nPbH / nSbH-1), (nW, nH) = (nSbW, nSbH), mvLX = mvLXN [xNb] [yNb], reference image refImg = refIdxLXN [xb] [yb], the reference picture, the motion vector accuracy M, and the number of taps NTAP are used as input parameters.

  First, the interpolation image generation unit 3092 derives an integer position (xInt, yInt) and a phase (xFrac, yFrac) corresponding to the coordinates (x, y) in the prediction block by the following formula.

xInt = xb + (mvLX [0] >> (log2 (M))) + x
xFrac = mvLX [0] & (M-1)
yInt = yb + (mvLX [1] >> (log2 (M))) + y
yFrac = mvLX [1] & (M-1)
Here, x = 0..nW-1, y = 0..nH-1, and M indicate the accuracy (1 / M pixel accuracy) of the motion vector mvLX.

  That is, as shown in the above equation, the interpolation image generation unit 3092 derives a sub-block position on the reference image to which the motion information of the target sub-block or the adjacent sub-block is applied.

  The interpolation image generation unit 3092 derives a temporary image temp [] [] by performing vertical interpolation processing on the reference image refImg using an interpolation filter (the following Σ is k = 0..k of NTAP-1). Sum, shift1 is a normalization parameter that adjusts the range of values).

temp [x] [y] = (ΣmcFilter [yFrac] [k] * refImg [xInt] [yInt + k-NTAP / 2 + 1] + offset1) >> shift1
Subsequently, the interpolated image generation unit 3092 derives an interpolated image Pred [] [] by performing horizontal interpolation processing on the temporary image temp [] [] (the following Σ relates to k of k = 0..NTAP-1). Sum, shift2 is a normalization parameter that adjusts the range of values).

Pred [x] [y] = (ΣmcFilter [xFrac] [k] * temp [xInt + k-NTAP / 2 + 1] [yInt] + offset2) >> shift2
(Example of filter coefficient)
Next, examples of the interpolation filters mcFilterN2, mcFilterN4, mcFilterN6, and mcFilterN8 corresponding to each tap number of NTAP = 2, 4, 6, 8 when the motion vector accuracy is M = 4. Note that the filter coefficient is not limited to this.
mcFilterN8 [] = {
{0, 0, 0, 64, 0, 0, 0, 0},
{-1, 4, -10, 58, 17, -5, 1, 0},
{-1, 4, -11, 40, 40, -11, 4, -1},
{0, 1, -5, 17, 58, -10, 4, -1}
}
mcFilterN6 [] = {
{0, 0, 64, 0, 0, 0},
{2, -8, 56, 18, -6, 2},
{2, -10, 40, 40, -10, 2},
{2, -6, 18, 56, -8, 2},
}
mcFilterN4 [] = {
{0, 64, 0, 0},
{-4, 54, 16, -2},
{-4, 36, 36, -4},
{-2, 16, 54, -4}
}
mcFilterN2 [] = {
{64, 0},
{48, 16},
{32, 32},
{16, 48}
}
When the OBMC process is not performed, only the PU interpolation image generation unit 30911 may perform the process.

(weighted average)
In the configuration for performing the OBMC process, the above-described OBMC correction unit 3093 performs the weighted averaging process on the received OBMC interpolated image Pred_N [x] [y] and the PU interpolated image Pred_C [x] [y], thereby performing prediction. Generate or update image Pred_ [x] [y]. More specifically, when the OBMC flag OBMC_flag input from the inter prediction parameter decoding unit 303 is 1 (OBMC processing is valid), the OBMC correction unit 3093 performs a weighted average process represented by the following equation.

Prediction image Pred_ [x] [y] = ((w1 * PU interpolation image Pred_C [x] [y] + w2 * OBMC interpolation image Pred_N [x] [y]) + o) >> shift
Here, the weights w1 and w2 in the weighted average processing will be described. The weights w1 and w2 in the weighted averaging process are determined according to the distance (the number of pixels) of the target pixel from the PU boundary. The shift value shift may be changed according to the distance, or may be fixed.

  Hereinafter, generation of a predicted image Pred_ [x] [y] in a case where the OBMC processing sizes nOBMCW and nOBMCH are four pixels will be described.

  To change the shift value according to the distance, for example, {w1, w2, o, shift} = {3, 1, 2, 2}, {7, 1, 4, 3}, {15, 1, 8, 4}, {31, 1, 16, 5}. In this case, the predicted image Pred_ [x] [y] is generated by the following equation.

Pred_ [x] [y] = (3 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +2) >> 2 Distance = 0 pixel
Pred_ [x] [y] = (7 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +4) >> 3 Distance = 1 pixel
Pred_ [x] [y] = (15 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +8) >> 4 Distance = 2 pixels
Pred_ [x] [y] = (31 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +16) >> 5 Distance = 3 pixels Also, when the shift value is fixed regardless of the distance For example, {w1, w2} = {24, 8}, {28, 4}, {30, 2}, {31, 1}, o = 16, shift = 5. In this case, the predicted image Pred_ [x] [y] is generated by the following equation.

Pred_ [x] [y] = (24 * Pred_C [x] [y] + 8 * Pred_N [x] [y] +16) >> 5 Distance = 0 pixel
Pred_ [x] [y] = (28 * Pred_C [x] [y] + 4 * Pred_N [x] [y] +16) >> 5 Distance = 1 pixel
Pred_ [x] [y] = (30 * Pred_C [x] [y] + 2 * Pred_N [x] [y] +16) >> 5 Distance = 2 pixels
Pred_ [x] [y] = (31 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +16) >> 5 Distance = 3 pixels When changing the shift value according to the distance Pred_ [x] [y] generated from the above equation is equivalent to Pred_ [x] [y] generated from the above equation when the shift value is fixed regardless of the distance.

  The generation of the predicted image Pred_ [x] [y] when the OBMC processing sizes nOBMCW and nOBMCH are two pixels is as follows.

  When the shift value is changed according to the distance, for example, {w1, w2, o, shift} = {3, 1, 2, 2}, {7, 1, 4, 3}. In this case, the predicted image Pred_ [x] [y] is generated by the following equation.

Pred_ [x] [y] = (3 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +2) >> 2 Distance = 0 pixel
Pred_ [x] [y] = (7 * Pred_C [x] [y] + 1 * Pred_N [x] [y] +4) >> 3 Distance = 1 pixel Also, when the shift value is fixed regardless of the distance For example, {w1, w2} = {24, 8}, {28, 4}, o = 16, shift = 5. In this case, the predicted image Pred_ [X] [Y] is generated by the following equation.

Pred_ [x] [y] = (24 * Pred_C [x] [y] + 8 * Pred_N [x] [y] +16) >> 5 Distance = 0 pixel
Pred_ [x] [y] = (28 * Pred_C [x] [y] + 4 * Pred_N [x] [y] +16) >> 5 Distance = 1 pixel
In the OBMC process, a predicted image is generated using the motion parameters of a plurality of adjacent PUs (PUs adjacent to the upper left, lower right, and lower sides of the target PU). Here, an outline of a method of generating Pred_ [x] [y] from the motion parameters of a plurality of adjacent PUs will be described.

  First, the OBMC correction unit 3093 converts the PU interpolation image Pred_C [x] [y] and the OBMC interpolation image Pred_N [x] [y] created using the motion parameters of the PU adjacent above the target PU into the above equation. To generate a predicted image Pred_ [x] [y].

Predicted image Pred_ [x] [y] = ((w1 * predicted image Pred_C [x] [y] + w2 * OBMC interpolated image Pred_N [x] [y]) + o) >> shift
Next, the OBMC corrector 3093 corrects the OBMC interpolation image Pred_N [x] [y] created using the motion parameters of the PU adjacent to the left side of the target PU, and the predicted image Pred [x] [y generated earlier. ] To update the predicted image Pred_ [x] [y]. That is, it is updated by the following equation.

Predicted image Pred_ [x] [y] = ((w1 * predicted image Pred_ [x] [y] + w2 * OBMC interpolated image Pred_N [x] [y]) + o) >> shift
The OBMC correction unit 3093 sets Pred_N [x] [y] as the OBMC interpolation image created using the motion parameters of the PU adjacent to the lower side of the target PU, and calculates the predicted image Pred_ [x] [y] by the following equation. Update.

Predicted image Pred_ [x] [y] = ((w1 * predicted image Pred_ [x] [y] + w2 * OBMC interpolated image Pred_N [x] [y]) + o) >> shift
The OBMC correction unit 3093 sets Pred_N [x] [y] as an OBMC interpolated image created using the motion parameters of the PU adjacent to the right side of the target PU, and updates the predicted image Pred_ [x] [y] according to the following equation. I do.

Predicted image Pred_ [x] [y] = ((w1 * predicted image Pred_ [x] [y] + w2 * OBMC interpolated image Pred_N [x] [y]) + o) >> shift
According to the above configuration, the motion compensation unit 3091 generates an additional interpolated image using the motion parameters of the PU adjacent to the target PU. Then, the motion compensation unit 3091 can generate a prediction image using the generated interpolation image. Therefore, a predicted image with high prediction accuracy can be generated.

  The size of the sub-block to be subjected to the OBMC process may be any size (4 × 4 to PU size). Further, the partitioning method of the PU including the sub-block to be subjected to the OBMC process may be any partitioning method such as 2N × N, N × 2N, and N × N.

(Flow of OBMC processing)
Next, the flow of the OBMC process according to the present embodiment will be described.

  FIG. 19 is a flowchart illustrating the flow of the process of the motion compensation unit 3091 according to the present embodiment. FIG. 20 is a diagram showing an example of a pseudo code representing the OBMC process. In FIG. 20, (xSb, ySb) indicates the upper left coordinates of the target sub-block (for example, the target PU) in pixel units. In FIG. 20, (nPbW, nPbH) indicates the block size (the size of the target PU). In FIG. 20, (subX, subY) indicates the position of the target sub-block (sub-block unit) within the target block (target PU). In FIG. 20, dir indicates the direction of an adjacent reference block in the OBMC process. Also, nSbW and nSbH are the width and height of the sub-block.

  The interpolation image generation unit 3092 derives a PU interpolation image Pred_C [x] [y] (S1).

  The motion compensation unit 3091 receives the OBMC flag OBMC_flag, and determines whether or not the OBMC flag OBMC_flag is 1 (S11). When OBMC_flag is 1, that is, when OBMC is on (YES in S11), the interpolation image generation unit 3092 and the OBMC correction unit 3093 perform the following sub-block loop processing and direction loop processing (S2).

  More specifically, the interpolated image generation unit 3092 and the OBMC correction unit 3093 perform a loop process for each sub-block constituting the PU. That is, a sub-block for performing the OBMC process is determined. The loop variables of the sub-block loop are coordinates (subX, subY), and the interpolated image generation unit 3092 and the OBMC correction unit 3093 perform a loop process by sequentially setting the coordinates of the sub-blocks in the PU (the end of the loop is S7). .

  Further, the interpolated image generation unit 3092 and the OBMC correction unit 3093 perform a loop process for each direction dir of upper above, left left, lower bottom, and right right. The loop process is performed by sequentially setting the values of the directions included in the direction set dirSet (dirSet = {above, left, bottom, right}) for the loop variable dir of the direction loop (the end of the loop is S7). Note that values of 0, 1, 2, and 3 may be assigned to above, left, bottom, and right, and processing may be performed on dirSet = {0, 1, 2, 3}.

  When the target CU is a 2N × 2N partition and the prediction mode is the merge mode (including the skip mode), the entropy decoding unit 301 does not decode the OBMC flag OBMC_flag from the encoded data, and the OBMC is valid. Then, a value (1) indicating that is obtained is derived. That is, when the target PU is 2N × 2N and in the merge mode, the OBMC process is always turned on.

  Hereinafter, in the loop, when the conditions of S3 and S4 are satisfied for the direction dir of the adjacent sub-block determined in S2 and the sub-block of the coordinates specified in S2, the interpolation image generation unit 3092 sets the OBMC interpolation image Pred_N [x] [y] is generated (S5), and the predicted image is corrected by the OBMC correction unit 3093 (S6).

  The details of the sub-block loop processing will be described. The OBMC interpolation image generation unit 30912 determines whether or not the prediction parameter of the adjacent sub-block located in the direction dir of the target sub-block is valid (S3). Note that the positions (xNb, yNb) of adjacent sub-blocks that refer to the motion parameters when the direction dir is upper, left, lower, and right are (xSb, ySb-1), (xSb-1, ySb), and ( xSb, ySb + nSbH) and (xSb + nSbW, ySb).

  Describing S3 in detail, the OBMC interpolation image generation unit 30912 determines whether or not the motion parameter of the adjacent sub-block in the direction is valid. For example, when the prediction mode of the adjacent sub-block is intra prediction, when the adjacent sub-block is out of the screen, or when the motion parameter of the adjacent sub-block is unknown, the OBMC interpolation image generation unit 30912 determines the motion of the adjacent sub-block. It is determined that the parameter cannot be used. In other words, the prediction mode of the adjacent sub-block (xNb, yNb) is a mode other than the intra prediction, and the position (xNb, yNb) of the adjacent sub-block is in the screen, and the adjacent sub-block (xNb, yNb) ) Is a parameter indicating a known motion parameter (for example, the reference picture index is other than −1), the OBMC interpolation image generation unit 30912 determines that the motion parameter of the adjacent sub-block can be used. When the adjacent sub-block is a PU of an intra block copy, the adjacent sub-block holds the motion parameter, but the OBMC interpolation image generation unit 30912 may determine that the motion parameter cannot be used.

  The case where the motion parameter of the adjacent sub-block is unknown will be described in detail with reference to FIG. FIG. 21 is a diagram illustrating whether or not the motion parameter of the adjacent sub-block is unknown. For example, when scanned in the Z scan order, the motion parameters of the PUs included in the already processed neighboring CUs (upper and left neighboring CUs) are known. However, the motion parameters of the PUs included in the pre-processing CU (the lower and right neighboring CUs) are unknown. Since the motion parameters of the PUs in the same CU are derived at the same time, the motion parameters of the left, upper, right, and lower adjacent PUs are known. In other words, at the CU boundary (indicated by the thick line in FIG. 21), the OBMC process is executed using only the motion parameters of the PUs adjacent to the left and upper. On the other hand, at the PU boundary (indicated by the broken line in FIG. 21), the OBMC process is performed using the motion parameters of the PUs adjacent to the left, upper, right, and lower.

  Hereinafter, in the present invention, boundaries between sub-blocks (PUs) are distinguished into CU boundaries and PU boundaries as follows.

  CU boundary: Among the boundaries between the target sub-block and the adjacent sub-block, when the CU including the target sub-block and the CU including the adjacent sub-block belong to different CUs, this boundary is called a CU boundary. For example, in FIG. 21, the boundary between the target PU and the upper adjacent CU, the boundary between the target PU and the left adjacent CU, and the boundary between the target PU and the lower adjacent CU are CU boundaries.

  PU boundary: Among the boundaries between the target sub-block and the adjacent sub-block, a boundary other than the CU boundary (the CU including the target sub-block and the CU including the adjacent sub-block belong to the same CU) is called a PU boundary. For example, in FIG. 21, the boundary between the target PU and the PU on the right side of the target PU is a PU boundary.

If the adjacent sub-block is valid (YES in S3), the OBMC interpolation image generation unit 30912 determines whether the motion parameter of the adjacent sub-block (xNb, yNb) is equal to the motion parameter of the target sub-block (S4). . Here, DiffMotionAvail is set to 1 if the motion parameters are different, and 0 if they are equal. Conversely, when the adjacent sub-block is not valid (NO in S3), the identity determination processing (S4), the generation of the OBMC interpolation image (S5), and the correction of the predicted image (S6) are omitted, and the following Transition to sub-block processing (S7)
Describing S4 in detail, for example, the OBMC interpolated image generation unit 30912 may use a motion vector as a motion parameter used for identity determination. In this case, it is determined whether the motion vector mvLXN of the adjacent sub-block (xNb, yNb) is equal to the motion vector mvLX of the target sub-block.

  The OBMC interpolation image generation unit 30912 may make a determination based on a reference picture index in addition to the motion vector as a motion parameter used for the identity determination.

  The motion vector of the target sub-block is (mvLX [0], mvLX [1]), the reference picture index is (refIdxLX), the motion vector of the adjacent sub-block is (mvLXN [0], mvLXN [1]), and the reference picture index is When (refIdxLXN), if the motion vector or the reference picture index differs between the target sub-block and the adjacent sub-block, the OBMC interpolation image generation unit 30912 sets DiffMotionAvail = 1, and the OBMC interpolation image generation unit 30912 sets the motion parameter to It is determined that they are different (NO in S4).

When the determination of the identity of the motion parameters is expressed by an equation,
DiffMotionAvail = (mvLX [0]! = MvLXN [0]) || (mvLX [1]! = MvLXN [1]) || (refIdxLX! = RefIdxLXN)
When the motion parameters are different, DiffMotionAvail = 1.

  When the motion vector and the reference picture index are equal between the current sub-block and the adjacent sub-block, DiffMotionAvail = 0. In this case, the OBMC interpolation image generation unit 30912 determines that the motion parameters are equal (YES in S4).

As a motion parameter used for the identity determination, a POC (Picture Order Count) may be used instead of the reference picture index. In this case, if the motion vector or the POC differs between the target sub-block and the adjacent sub-block, the OBMC interpolation image The generation unit 30912 sets DiffMotionAvail = 1, and the OBMC interpolation image generation unit 30912 determines that the motion parameters are different (NO in S4).
When the determination of the identity of the motion parameters is expressed by an equation,
DiffMotionAvail = (mvLX [0]! = MvLXN [0]) || (mvLX [1]! = MvLXN [1]) || (refPOC! = RefPOCN)
Here, refPOC and refPOCN are POCs of reference images of the target sub-block and the adjacent sub-block.

  If the motion parameter of the adjacent sub-block is not equal to the motion parameter of the target sub-block (NO in S4), the OBMC interpolation image generation unit 30912 generates (derives) the OBMC interpolation image Pred_N [x] [y] of the target sub-block. Then (S5), the OBMC correction unit 3093 generates a prediction image Pred_ [x] [y] using the OBMC interpolation image Pred_N [x] [y] and the PU interpolation image Pred_C [x] [y] of the target sub-block. Or, it is updated (S6).

  Explaining S5 in detail, the OBMC interpolation image generation unit 30912 generates (derives) an OBMC interpolation image Pred_N [x] [y] using the motion parameters of the adjacent sub-blocks. When the prediction mode is not the sub-block prediction (processing is performed without dividing the PU), the OBMC interpolation image generation unit 30912 sets nOBMCW indicating the width of the OBMC processing size and nOBMCH indicating the height to four pixels. When the prediction mode is sub-block prediction, the OBMC interpolation image generation unit 30912 sets nOBMCW and nOBMCH, which are the OBMC processing sizes, to two pixels.

  When dir == above, the OBMC interpolation image generation unit 30912 refers to the motion parameter of the sub-block (xNb, yNb) = (xSb, ySb-1) adjacent to the target sub-block, and (xSb, ySb) The interpolation image of the area of the size (nSbW, nOBMCH) where is set to the upper left coordinate is derived.

  Also, when dir == left, the OBMC interpolation image generation unit 30912 refers to the motion parameter of the subblock (xNb, yNb) = (xSb-1, ySb) adjacent to the left of the target subblock, and (xSb, An interpolated image of an area of size (nOBMCW, nSbH) with ySb) as the upper left coordinate is derived.

  Also, when dir == bottom, the OBMC interpolation image generation unit 30912 refers to the motion parameter of the sub-block (xNb, yNb) = (xSb, ySb + nSbH) adjacent to the target sub-block, and (xSb, An interpolation image of a region of size (nSbW, nOBMCH) with ySb + nSbH-nOBMCH) as the upper left coordinate is derived.

  Further, when dir == right, the OBMC interpolation image generation unit 30912 refers to the motion parameter of the sub-block (xNb, yNb) = (xSb + nSbW, ySb) adjacent to the right of the target sub-block, and (xSb + An interpolated image of a region of size (nOBMCW, nSbH) using nSbW-nOBMCW, ySb) as the upper left coordinate is derived.

The above process can be shown in pseudo code as follows: In the pseudo code, an interpolation image generation process of the reference image refPic, the upper left coordinates (xb, yb), the size nW, nH, and the motion vector mvRef is represented by Interpolation (refPic, xb, yb, nW, nH, mvRef). Here, the motion parameters of the coordinates (x, y) are represented by mvLX [x] [y] and refIdxLX [x] [y].
nOBMCW = nOBMCH = (PU is not a sub-block)? 4: 2
if (dir == above)
mvRef = mvLXN [xSb] [ySb-1], refPic = refIdxLXN [xSb] [ySb-1]
predN = Interpolation (refPic, xSb, ySb, nSbW, nOBMCH, mvRef)
else if (dir == left)
mvRef = mvLXN [xSb-1] [ySb], refPic = refIdxLXN [xSb-1] [ySb]
predN = Interpolation (refPic, xSb, ySb, nOBMCW, nSbH, mvRef)
else if (dir == bottom)
mvRef = mvLXN [xSb] [ySb + nSbH], refPic = refIdxLXN [xSb] [ySb + nSbH]
predN = Interpolation (refPic, xSb, ySb + nSbH-nOBMCH, nSbW, nOBMCH, mvRef)
else if (dir == right)
mvRef = mvLXN [xSb + nSbW] [ySb], refPic = refIdxLXN [xSb + nSbW] [ySb]
predN = Interpolation (refPic, xSb + nSbW-nOBMCW, ySb, nOBMCW, nSbH, mvRef)
Next, the OBMC correction unit 3093 performs weighted averaging processing with reference to the target sub-block OBMC interpolation image Pred_N [x] [y] and the PU interpolation image Pred_C [x] [y], and performs a prediction image Pred_ [x] [ y] is generated or updated (S6).

  Describing S6 in detail, the OBMC correction unit 3093 refers to the OBMC interpolation image Pred_N [x] [y] and the PU interpolation image Pred_C [x] [y] according to the distance from the boundary with the adjacent sub-block. Weighted averaging. The weighted average processing is as follows.

Pred_ [x] [y] = Pred_C [x] [y]
When dir == above, the OBMC correction unit 3093 derives a predicted image Pred_ [x] [y] from the following equation, where i = 0..nSbW-1 and j = 0..nOBMCH-1.

x = xSb + i, y = ySb + j
Pred_ [x] [y] = (w1 * Pred_ [x] [y] + w2 * Pred_N [i] [j] + o) >> shift
w1 = weightOBMC [j], w2 = (1 << shift)-w1, o = (1 << (shift-1))
When dir == left, the OBMC correction unit 3093 derives a predicted image Pred_ [x] [y] from the following equation, assuming i = 0..nOBMCW-1, j = 0..nSbH-1. .

x = xSb + i, y = ySb + j
Pred_ [x] [y] = (w1 * Pred_ [x] [y] + w2 * Pred_N [i] [j] + o) >> shift
w1 = weightOBMC [i], w2 = (1 << shift)-w1, o = (1 << (shift-1))
Also, when dir == bottom, the OBMC correction unit 3093 derives a predicted image Pred_ [x] [y] from the following equation, with i = 0..nSbW-1, j = 0..nOBMCH-1. .

x = xSb + i, y = ySb + nSbH-nOBMCH + j
Pred_ [x] [y] = (w1 * Pred_ [x] [y] + w2 * Pred_N [i] [j] + o) >> shift
w1 = weightOBMC [nOBMCH-1-j], w2 = (1 << shift)-w1, o = (1 << (shift-1))
Also, when dir == right, the OBMC correction unit 3093 derives a predicted image Pred_ [x] [y] from the following equation, with i = 0..nOBMCW-1, j = 0..nSbH-1. .

x = xSb + nSbW-nOBMCW + i, y = ySbH + j
Pred_ [x] [y] = (w1 * Pred_ [x] [y] + w2 * Pred_N [i] [j] + o) >> shift
w1 = weightOBMC [nOBMCW-1-i], w2 = (1 << shift)-w1, o = (1 << (shift-1))
As described above, the weight is set according to the distance (the number of pixels) from the boundary, and the weight table weightOBMC may be as follows.

weightOBMC [] = {24, 28, 30, 31}, shift = 5
Or
weightOBMC [] = {8, 4, 2, 1}, shift = 5
Next, the motion compensating unit 3091 performs the OBMC processing of S3 to S6 by using whether or not there is an unprocessed subblock among the subblocks to be subjected to the OBMC processing and the motion parameters of the adjacent subblocks in all directions. It is determined whether or not it has been touched (S7).

  If there is no unprocessed sub-block and the OBMC process has been performed using the motion parameters of the adjacent sub-blocks in all directions (NO in S7), the process ends.

  If there is an unprocessed sub-block or if the OBMC process has not been performed using the motion parameters of the adjacent sub-blocks in all directions (YES in S7), the process proceeds to S2 and the process in the next direction is performed. .

  When OBMC_flag = 0, PRED _ [] [] = PRED_C [] [].

(Modification of motion compensation)
(Motion compensation unit 3091)
Next, a modified example of the motion compensation performed by the motion compensation unit 3091 will be described. FIG. 22 is a block diagram illustrating a main configuration of a motion compensation unit 3091 included in the inter prediction image generation unit 309 that performs the OBMC process according to the present modification. As shown in FIG. 22, the motion compensation unit 3091 includes an interpolation image generation unit 3092 (PU interpolation image generation unit 30911 and OBMC interpolation image generation unit 30912), an OBMC correction unit 3093, and an OBMC processing execution determination unit 3095. It should be noted that the processing performed by the interpolation image generation unit 3092 and the OBMC correction unit 3093 according to the present modified example will be described only with respect to differences from the processing performed by the interpolation image generation unit 3092 and the OBMC correction unit 3093 described above.

(OBMC processing execution determination unit 3095)
The OBMC processing execution determination unit 3095 according to the present modification generates an interpolated image when the difference between the motion vector of the target sub-block and the motion vector of the sub-block adjacent to the target sub-block is equal to or smaller than a predetermined value. It is determined that the OBMC processing in the unit 3092 and the OBMC correction unit 3093 is to be performed.

(Judgment example 1 of OBMC processing execution judgment unit 3095)
Next, an example of the determination performed by the OBMC processing execution determination unit 3095 will be described. In this example, the OBMC processing execution determination unit 3095 determines the motion vector of the target sub-block and the motion vector of the sub-block (adjacent sub-block) in the same PU (block including the target sub-block) adjacent to the target sub-block Are determined to be the same. That is, in this example, when the difference between the motion vector of the target block and the motion vector of the adjacent sub-block is 0, the OBMC process is performed. When the motion vector of the target sub-block is the same as the motion vector of the adjacent sub-block, the motion compensator 3091 performs a predicted image generation process using the OBMC process. The OBMC process is performed on a sub-block obtained by merging (combining) the target sub-block and the adjacent sub-block. In other words, the OBMC processing execution determination unit 3095 determines whether the motion vector of the target sub-block is the same as the motion vector of the adjacent sub-block, thereby merging the target sub-block and the adjacent sub-block. It is determined whether or not it is possible. If the target sub-block and the adjacent sub-block can be merged, the interpolation image generation unit 3092 and the OBMC correction unit 3093 perform the OBMC process on the merged sub-block.

  FIG. 23 is a diagram illustrating an example of a sub-block obtained by merging a target sub-block and a sub-block in the same PU adjacent to the target sub-block. The thin frame lines shown in FIG. 23 indicate each sub-block. Also, the arrows described in FIG. 23 indicate the motion vectors of each sub-block. Further, a thick frame line shown in FIG. 23 indicates a merged sub-block when the sub-block can be merged. As shown in FIG. 23, when the motion vector of the target sub-block is the same as the motion vector of the adjacent sub-block, the OBMC interpolation image generation unit 30912 merges these sub-blocks.

  The above description means that when adjacent sub-blocks Si and Sj have the same motion vector (when merging is possible), OBMC processing is performed between sub-block Si and sub-block Sj. Not something.

  Specifically, the OBMC interpolation image generation unit 30912 determines that the sub-block Sa (adjacent sub-block) in the direction dirA (for example, right) of the target sub-block St has the same motion vector as the target sub-block St, and When it is determined that the block St and the sub-block Sa can be merged, the following OBMC processing is performed. The OBMC interpolation image generation unit 30912 performs OBMC processing (correction processing) such as weighted averaging between the target sub-block St and the sub-block Sb (adjacent reference sub-block) adjacent to dirB (for example, above) in a direction different from the direction dirA. Do.

  In this case, the OBMC interpolation image generation unit 30912 corrects the predicted image Pred of the target sub-block St using the predicted image PredB of the target sub-block St derived using the motion vector MVb of the adjacent reference sub-block Sb. It is not necessary to perform the OBMC correction process between the target sub-block St and the sub-block Sa (adjacent sub-block) adjacent to the above dirA. Since the motion vector MVA of the sub-block Sa is equal to the motion vector MVt of the target sub-block St, the predicted image PredA of the target sub-block St derived using the motion vector MVA of the sub-block Sa is the prediction of the target sub-block St. It is the same as the image Pred. Therefore, it is not necessary to perform the OBMC correction process. Even if the correction processing is performed between the same predicted images, there is no change in the predicted images, so that the OBMC correction processing may be performed.

  Here, the sub-block in the same PU adjacent to the target sub-block to be merged in this determination example will be described with reference to FIG. FIG. 24 is a diagram illustrating an example of a subblock in the same PU adjacent to the target subblock. As shown in FIG. 24, the sub-block T is a sub-block included in the target PU. Further, as shown in FIG. 24, the sub-block TN to be merged with the sub-block T is a sub-block included in the target PU, and is a sub-block processed after the sub-block T. In this example, in the OBMC process, when the motion vector of the sub-block TN is the same as the motion vector of the sub-block T, these sub-blocks are merged into one sub-block.

  FIG. 25 is a diagram showing an example of pseudo code representing the OBMC process according to the present example. As illustrated in FIG. 25, the OBMC process execution determination unit 3095 may determine whether the target sub-block can be merged with an adjacent sub-block in the target block (eg, the target PU). When the OBMC processing execution determination unit 3095 determines that merging is possible, the interpolation image generation unit 3092 and the OBMC correction unit 3093 perform OBMC processing on the merged sub-block. If the OBMC processing execution determination unit 3095 determines that merging is not possible, the interpolation image generation unit 3092 and the OBMC correction unit 3093 do not perform OBMC processing on the target sub-block, and shift to the next sub-block.

  More specifically, the OBMC process execution determination unit 3095 derives mergeBlkFlag indicating whether or not the target sub-block can be merged with an adjacent sub-block in the target block. FIG. 26 illustrates an example of a determination formula for the OBMC process execution determination unit 3095 to derive mergeBlkFlag. In FIG. 26, the position of a subblock in the same PU adjacent to the target subblock is (xNbC, yNbC).

  As shown in FIG. 26, the OBMC processing execution determination unit 3095 determines whether the motion vector of the target sub-block is the same as the motion vector of the sub-block in the same PU adjacent to the target sub-block, and determines whether the mergeBlkFlag Is derived. The OBMC processing execution determination unit 3095 outputs the derived mergeBlkFlag to the interpolation image generation unit 3092 and the OBMC correction unit 3093. The interpolation image generation unit 3092 and the OBMC correction unit 3093 refer to mergeBlkFlag and perform OBMC processing when the target sub-block can be merged.

  According to the above configuration, when the motion vector of the target sub-block is the same as the motion vector of the sub-block in the same PU adjacent to the target sub-block, that is, the target sub-block and the adjacent sub-block can be merged. If, the OBMC process is performed.

  For example, OBMC processing is performed on an 8 × 4 unit block obtained by merging a 4 × 4 unit target sub-block and a 4 × 4 unit adjacent sub-block.

  Therefore, it is possible to avoid the OBMC processing for the target sub-block having a small size. Therefore, it is possible to reduce the amount of processing in the OBMC processing, and to suppress an increase in a memory band for accessing image data.

  FIG. 27 is a diagram illustrating another example of the pseudo code representing the OBMC process according to the present example. As shown in FIG. 27, the OBMC processing execution determination unit 3095 determines whether the target sub-block can be merged with a horizontally adjacent sub-block in the same PU (block including the target sub-block) in the target block. You may. When the OBMC processing execution determination unit 3095 determines that merging is possible, the interpolation image generation unit 3092 and the OBMC correction unit 3093 merge the target sub-block and a sub-block in the same PU horizontally adjacent to the target block. OBMC processing is performed on the set sub-block.

  More specifically, the OBMC processing execution determination unit 3095 derives mergeBlkFlag indicating whether or not the target sub-block can be merged with a horizontally adjacent sub-block in the same PU in the target block. FIG. 28 illustrates an example of a determination formula for the OBMC process execution determination unit 3095 to derive mergeBlkFlag. When the adjacent sub-block is adjacent to the target sub-block in the horizontal direction, the position of the target sub-block is (xSb = subX * nSbW, ySb = subY * nSbH), and the position of the sub-block adjacent to the target sub-block Is (xNbC = (subX + 1) * nSbW, yNb = subY * nSbH). As shown in FIG. 28, the OBMC processing execution determination unit 3095 derives mergeBlkFlag depending on whether the motion vector of the target sub-block is the same as the motion vector of the adjacent sub-block.

  As described above, it is assumed that the sub-block Sa in the direction dirA (for example, right) of the target sub-block St has the same motion vector as the target sub-block St, and the sub-block St and the sub-block Sa can be merged. When it is determined, the following OBMC processing is performed. That is, the OBMC interpolation image generation unit 30912 performs the OBMC process between the sub-block St and the sub-block Sb (adjacent reference sub-block) that is in contact with dirB (for example, above) in a direction different from the direction dirA. In this case, the OBMC interpolation image generating unit 30912 corrects the predicted image Pred of the target sub-block St using the predicted image PredB of the target sub-block St derived using the motion vector of the sub-block Sb.

  Note that “adjacent in the horizontal direction” means that the target sub-block is located on the right side in the same block. Alternatively, “being horizontally adjacent” means being located on the left or right side within the same block with respect to the target sub-block.

  According to the above configuration, when a target sub-block and a sub-block in the same PU horizontally adjacent to the target sub-block can be merged, the OBMC process is performed on the merged sub-block. Therefore, it is possible to reduce the amount of processing in the OBMC processing, and to suppress an increase in a memory band for accessing image data.

  This example is suitable for a configuration in which the processing of the sub-block is performed in the raster scan order in which the priority in the horizontal direction is given.

(Judgment example 2 of OBMC processing execution judgment unit 3095)
Next, another example of the determination performed by the OBMC processing execution determination unit 3095 will be described. In this example, when the difference between the motion vector of the target sub-block and the motion vector of the adjacent sub-block adjacent to the target sub-block is equal to or smaller than a predetermined value, the It is determined that the OBMC processing in the generation unit 3092 and the OBMC correction unit 3093 is to be performed. The adjacent sub-block adjacent to the target sub-block in this example is a sub-block having a motion vector that can be used for the OBMC processing of the target sub-block. Here, a specific example of the sub-block adjacent to the target sub-block (adjacent reference sub-block) in this determination example will be described with reference to FIG. FIG. 29 is a diagram illustrating an example of the adjacent reference sub-block.

  As shown in FIG. 29, the target sub-block T is a target sub-block included in the target CU. For example, the OBMC processing execution determination unit 3095 determines whether a subblock adjacent above, to the left, to the right of, or below the target subblock T has a motion vector that can be used for the OBMC processing of the target subblock T. In the example illustrated in FIG. 29, the adjacent sub-block TA, adjacent sub-block TL, adjacent sub-block TR, and adjacent sub-block TB of the target sub-block T are blocks having motion vectors that can be used for the OBMC process of the target sub-block T. is there. That is, in the example illustrated in FIG. 29, the OBMC processing execution determination unit 3095 determines that the difference between the motion vector of the sub-block adjacent to the upper, left, right, and lower of the target sub-block T from the motion vector of the target sub-block is a predetermined A determination is made whether it is less than or equal to the value.

  In other words, the adjacent sub-block TA, adjacent sub-block TL, adjacent sub-block TR, and adjacent sub-block TB of the target sub-block T are sub-blocks having motion vectors that can be used for OBMC processing of the target sub-block T. That is, the adjacent sub-block for which the motion vector is determined is a sub-block (adjacent reference sub-block) that can be a reference sub-block in the OBMC process. The adjacent reference sub-block is, for example, a sub-block specified by dir (see FIG. 20, FIG. 30, etc.).

  FIG. 30 is a diagram illustrating an example of a pseudo code representing the OBMC process according to the present example. As shown in FIG. 30, the OBMC processing execution determination unit 3095 determines whether the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is smaller than a predetermined value. When the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is smaller than a predetermined value, the interpolation image generation unit generation unit 3092 and the OBMC correction unit 3093 use the motion vector of the adjacent reference sub-block. Perform OBMC processing.

  More specifically, the OBMC process execution determination unit 3095 derives a smallMVFlag indicating whether a difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is smaller than a predetermined value. FIG. 31 illustrates an example of a determination formula for the OBMC process execution determination unit 3095 to derive a smallMVFlag. As shown in FIG. 31, the OBMC processing execution determination unit 3095 indicates a value according to whether or not the difference mvdiff between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is smaller than a predetermined value TH. Derive smallMVFlag. In FIG. 31, (xSb, ySb) indicates the upper left coordinates of the target sub-block (for example, the target PU) in pixel units. The upper left coordinate of the adjacent reference sub-block is set to (xNb, yNb).

  FIG. 32 illustrates another example of a determination formula for the OBMC process execution determination unit 3095 to derive a smallMVFlag. As shown in FIG. 32, the OBMC processing execution determination unit 3095 determines that either the difference in the x direction or the difference in the y direction between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is smaller than a predetermined value TH. A smallMVFlag indicating a value according to whether or not may be derived. In FIG. 32, mvL0diffH and mvL1diffH indicate the difference in the x direction between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block. In addition, mvL0diffV and mvL1diffV indicate a difference in the y direction between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block.

  (XSb, ySb) indicates the upper left coordinates of the target sub-block (for example, the target PU) in pixel units. Also, the upper left coordinate of the sub-block adjacent to the target sub-block is set to (xNb, yNb). For example, the upper left coordinates (xSb, ySb) of the target sub-block and the upper left coordinates (xNb, yNb) of the adjacent reference sub-block may be derived as follows.

(xSb, ySb) = (subX * 4, subY * 4) // T
(xNb, yNb) = (xSb, ySb-1) = (subX * 4, subY * 4-1) // TA
(xNb, yNb) = (xSb-1, ySb) = (subX * 4-1, subY * 4) // TL
(xNb, yNb) = (xSb + nSbW, ySb) = (subX * 4 + nSbW, subY * 4) // TB
(xNb, yNb) = (xSb, ySb + nSbH) = (subX * 4, subY * 4 + nSbH) // TR
The OBMC processing execution determination unit 3095 outputs the derived smallMVFlag to the interpolation image generation unit 3092 and the OBMC correction unit 3093. The interpolation image generation unit 3092 and the OBMC correction unit 3093 perform OBMC processing on the target sub-block with reference to the smallMVFlag.

  According to the above configuration, when the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is equal to or smaller than a predetermined value, the interpolation image generation unit 3092 and the OBMC correction unit 3093 perform the OBMC process. Do.

  For example, when the difference between the motion vector of the target sub-block and the motion vector used for the OBMC process of the adjacent reference sub-block is small, the reference image used for the OBMC process is as follows. That is, the overlapping area of the reference image referred to using the motion vector added to the target sub-block and the reference image referred to using the motion vector of the adjacent reference sub-block increases.

  Therefore, when the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block used for the OBMC process is small, it is possible to suppress an increase in the memory band even when the OBMC process is performed. The target sub-block may be a block such as a PU.

  FIG. 33 is a diagram illustrating another example of the pseudo code representing the OBMC process according to the present example. As shown in FIG. 33, the OBMC processing execution determination unit 3095 determines that the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block adjacent to the target sub-block is not 0, and is larger than a predetermined value. It is determined whether it is small. When the difference is not 0 and smaller than a predetermined value, the interpolation image generation unit generation unit 3092 and the OBMC correction unit 3093 perform the OBMC process using the motion vector of the adjacent reference sub-block.

  FIG. 34 illustrates an example of a determination formula for the OBMC process execution determination unit 3095 to derive a smallMVFlag. As shown in FIG. 34, the OBMC process execution determination unit 3095 determines whether the difference mvdiff between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block is not 0 and is smaller than a predetermined value TH. Derive a smallMVFlag indicating a value corresponding to.

  When the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block used for the OBMC process is 0, the interpolated image is as follows. That is, the interpolated image generated using the motion vector added to the target sub-block is the same as the interpolated image generated using the motion vector of the adjacent reference sub-block.

  Therefore, when the difference between the motion vector of the target sub-block and the motion vector of the adjacent reference sub-block used in the OBMC process is 0, the OBMC process is not required. According to the above configuration, there is an effect that unnecessary OBMC processing can be suppressed.

(Weight prediction)
The weight prediction unit 3094 generates a PU predicted image by multiplying the input motion compensated image predSamplesLX by a weight coefficient. When one of the prediction list use flags (predFlagL0 or predFlagL1) is 1 (in the case of simple prediction), and when weight prediction is not used, the input motion-compensated image predSamplesLX (LX is L0 or L1) is the number of pixel bits bitDepth. Perform the processing of the following equation to match.

predSamples [x] [y] = Clip3 (0, (1 << bitDepth)-1, (predSamplesLX [x] [y] + offset1) >> shift1)
Here, shift1 = 14−bitDepth, offset1 = 1 << (shift1-1).
When both the reference list use flags (predFlagL0 and predFlagL1) are 1 (in the case of bi-prediction BiPred) and weight prediction is not used, the input motion compensated images predSamplesL0 and predSamplesL1 are averaged and the pixel bit number is calculated. Perform the processing of the following equation to match.

predSamples [x] [y] = Clip3 (0, (1 << bitDepth)-1, (predSamplesL0 [x] [y] + predSamplesL1 [x] [y] + offset2) >> shift2)
Here, shift2 = 15-bitDepth and offset2 = 1 << (shift2-1).

  Furthermore, in the case of simple prediction, when weight prediction is performed, the weight prediction unit 3094 derives a weight prediction coefficient w0 and an offset o0 from the encoded data, and performs the processing of the following equation.

predSamples [x] [y] = Clip3 (0, (1 << bitDepth)-1, ((predSamplesLX [x] [y] * w0 + 2 ^ (log2WD-1)) >> log2WD) + o0)
Here, log2WD is a variable indicating a predetermined shift amount.

  Furthermore, in the case of bi-prediction BiPred, when performing weight prediction, the weight prediction unit 3094 derives weight prediction coefficients w0, w1, o0, and o1 from the encoded data, and performs the processing of the following equation.

predSamples [x] [y] = Clip3 (0, (1 << bitDepth)-1, (predSamplesL0 [x] [y] * w0 + predSamplesL1 [x] [y] * w1 + ((o0 + o1 + 1) << log2WD)) >> (log2WD + 1))
(Configuration of image encoding device)
Next, the configuration of the image encoding device 11 according to the present embodiment will be described. FIG. 3 is a block diagram illustrating a configuration of the image encoding device 11 according to the present embodiment. The image encoding device 11 includes a predicted image generation unit 101, a subtraction unit 102, a transformation / quantization unit 103, an entropy encoding unit 104, an inverse quantization / inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory. (Prediction parameter storage unit, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, and prediction parameter coding unit 111. The prediction parameter coding unit 111 includes an inter prediction parameter coding unit 112 and an intra prediction parameter coding unit 113.

  The prediction image generation unit 101 generates a prediction image P of the prediction unit PU for each picture of the image T for each coding unit CU that is a region obtained by dividing the picture. Here, the predicted image generation unit 101 reads a decoded block from the reference picture memory 109 based on the prediction parameters input from the prediction parameter encoding unit 111. The predicted image generation unit 101 generates a predicted image P of the PU using one of a plurality of prediction methods for the read reference picture block. The predicted image generation unit 101 outputs the generated PU predicted image P to the subtraction unit 102.

  The operation of the predicted image generation unit 101 is the same as that of the already described predicted image generation unit 308, and a description thereof will be omitted.

  The predicted image generated by the predicted image generation unit 101 is output to the subtraction unit 102 and the addition unit 106.

  The subtraction unit 102 generates a residual signal by subtracting the signal value of the predicted image P of the PU input from the predicted image generation unit 101 from the pixel value of the corresponding PU of the image T. The subtraction unit 102 outputs the generated residual signal to the transform / quantization unit 103.

  Transform / quantization section 103 performs frequency conversion on the residual signal input from subtraction section 102 and calculates a conversion coefficient. The transform / quantization unit 103 quantizes the calculated transform coefficient to obtain a quantized coefficient. Transform / quantization section 103 outputs the obtained quantized coefficient to entropy encoding section 104 and inverse quantization / inverse transform section 105.

  The entropy coding unit 104 receives a quantization coefficient from the transform / quantization unit 103 and a coding parameter from the prediction parameter coding unit 111. The input coding parameters include codes such as a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, and a merge index merge_idx.

  The entropy coding unit 104 generates a coded stream Te by entropy coding the input quantization coefficients and coding parameters, and outputs the generated coded stream Te to the outside.

  The inverse quantization / inverse transform unit 105 inversely quantizes the quantized coefficient input from the transform / quantization unit 103 to obtain a transform coefficient. The inverse quantization / inverse transform unit 105 performs an inverse frequency transform on the obtained transform coefficient to calculate a residual signal. The inverse quantization / inverse transform unit 105 outputs the calculated residual signal to the addition unit 106.

  The addition unit 106 adds the signal value of the PU predicted image P input from the predicted image generation unit 101 and the signal value of the residual signal input from the inverse quantization / inverse transformation unit 105 for each pixel, and performs decoding. Generate an image. The adding unit 106 stores the generated decoded image in the reference picture memory 109.

  The loop filter 107 applies a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image generated by the adding unit 106.

  The prediction parameter memory 108 stores the prediction parameter generated by the coding parameter determination unit 110 at a position predetermined for each picture to be coded and each CU.

  The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each picture to be encoded and each CU.

  The encoding parameter determination unit 110 selects one set from a plurality of sets of encoding parameters. The coding parameter is the above-described prediction parameter or a parameter to be coded which is generated in association with the prediction parameter.

  The encoding parameter determination unit 110 calculates a magnitude of the information amount and a cost value indicating an encoding error for each of the plurality of sets. The encoding parameter determination unit 110 selects a set of encoding parameters that minimizes the calculated cost value. As a result, the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the unselected set of encoding parameters. The coding parameter determination unit 110 stores the determined coding parameter in the prediction parameter memory 108.

  The prediction parameter coding unit 111 derives a coding format from the parameters input from the coding parameter determination unit 110, and outputs the format to the entropy coding unit 104. Deriving a format for encoding is, for example, deriving a difference vector from a motion vector and a prediction vector. The prediction parameter coding unit 111 derives parameters necessary for generating a predicted image from the parameters input from the coding parameter determination unit 110, and outputs the parameters to the predicted image generation unit 101. The parameter required to generate the predicted image is, for example, a motion vector in sub-block units.

  The inter prediction parameter coding unit 112 derives an inter prediction parameter such as a difference vector based on the prediction parameter input from the coding parameter determination unit 110. The inter-prediction parameter encoding unit 112 derives parameters necessary for generating a predicted image to be output to the predicted image generation unit 101, and the inter-prediction parameter decoding unit 303 (see FIG. 4 and the like) derives the inter-prediction parameters. Some of the configurations are the same as the configurations to be performed. The configuration of the inter prediction parameter coding unit 112 will be described later.

  The intra prediction parameter coding unit 113 derives a coding format (for example, mpm_idx, rem_intra_luma_pred_mode, etc.) from the intra prediction mode IntraPredMode input from the coding parameter determination unit 110.

(Configuration of inter prediction parameter coding unit)
Next, the configuration of the inter prediction parameter coding unit 112 will be described. The inter prediction parameter coding unit 112 is a unit corresponding to the inter prediction parameter decoding unit 303 in FIG. 9, and the configuration is shown in FIG.

  The inter prediction parameter coding unit 112 includes an inter prediction parameter coding control unit 1121, an AMVP prediction parameter derivation unit 1122, a subtraction unit 1123, a sub block prediction parameter derivation unit 1125, and the like. The inter prediction parameter coding unit 112 outputs the motion vector (mvLX, subMvLX), the reference picture index refIdxLX, the PU partition mode part_mode, the inter prediction identifier inter_pred_idc, or information indicating these to the predicted image generation unit 101. Further, the inter prediction parameter coding unit 112 sets the PU partition mode part_mode, the merge flag merge_flag, the merge index merge_idx, the inter prediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx, the difference vector mvdLX, and the sub block prediction mode flag Output to encoding section 104.

  The inter prediction parameter coding control unit 1121 includes a merge index deriving unit 11211 and a vector candidate index deriving unit 11212. The merge index deriving unit 11211 compares the motion vector and the reference picture index input from the coding parameter determination unit 110 with the motion vector and the reference picture index of the PU of the merge candidate read from the prediction parameter memory 108, and performs merging. The index merge_idx is derived and output to the entropy coding unit 104. A merge candidate is a reference PU (for example, a reference PU adjacent to the lower left end, the upper left end, and the upper right end of an encoding target block) within a predetermined range from the encoding target CU to be encoded. This is the PU for which processing has been completed. The vector candidate index deriving unit 11212 derives a predicted vector index mvp_LX_idx.

  When the encoding parameter determination unit 110 determines to use the sub-block prediction mode, the sub-block prediction parameter derivation unit 1125 performs spatial sub-block prediction, temporal sub-block prediction, affine prediction, and matching motion derivation according to the value of subPbMotionFlag. A motion vector and a reference picture index for any sub-block prediction are derived. As described in the description of the image decoding apparatus, the motion vector and the reference picture index are derived by reading the motion vector of the adjacent PU and the reference picture block and the reference picture index from the prediction parameter memory 108. The sub-block prediction parameter deriving unit 1125 may perform the same processing as the processing performed by the above-described sub-block prediction parameter deriving unit 3037.

  The AMVP prediction parameter deriving unit 1122 has a configuration similar to that of the above-described AMVP prediction parameter deriving unit 3032 (see FIG. 9).

  That is, when the prediction mode predMode indicates the inter prediction mode, the motion vector mvLX is input from the coding parameter determination unit 110 to the AMVP prediction parameter derivation unit 1122. The AMVP prediction parameter deriving unit 1122 derives a prediction vector mvpLX based on the input motion vector mvLX. The AMVP prediction parameter derivation unit 1122 outputs the derived prediction vector mvpLX to the subtraction unit 1123. Note that the reference picture index refIdx and the prediction vector index mvp_LX_idx are output to the entropy coding unit 104.

  The subtraction unit 1123 generates a difference vector mvdLX by subtracting the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 1122 from the motion vector mvLX input from the encoding parameter determination unit 110. The difference vector mvdLX is output to entropy coding section 104.

  Note that a part of the image encoding device 11 and the image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the predicted image generation unit 308, the inverse quantization / inverse transform Unit 311, addition unit 312, predicted image generation unit 101, subtraction unit 102, transformation / quantization unit 103, entropy encoding unit 104, inverse quantization / inverse transformation unit 105, loop filter 107, encoding parameter determination unit 110, The prediction parameter coding unit 111 may be realized by a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read and executed by a computer system. Here, the “computer system” is a computer system built in one of the image encoding device 11 and the image decoding device 31, and includes hardware such as an OS and peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" is a medium that holds the program dynamically for a short time, such as a communication line for transmitting the program through a communication line such as a network such as the Internet or a telephone line, In this case, a program holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client, may be included. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, or may be for realizing the above-mentioned functions in combination with a program already recorded in a computer system.

  Further, part or all of the image encoding device 11 and the image decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image encoding device 11 and the image decoding device 31 may be individually implemented as a processor, or a part or all thereof may be integrated and implemented as a processor. The method of circuit integration is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where a technology for forming an integrated circuit that replaces the LSI appears due to the advance of the semiconductor technology, an integrated circuit based on the technology may be used.

  As described above, one embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention. It is possible to

[Application example]
The above-described image encoding device 11 and image decoding device 31 can be used by being mounted on various devices that transmit, receive, record, and reproduce moving images. Note that the moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.

  First, the fact that the above-described image encoding device 11 and image decoding device 31 can be used for transmitting and receiving moving images will be described with reference to FIG.

  FIG. 35A is a block diagram illustrating a configuration of a transmission device PROD_A on which the image encoding device 11 is mounted. As shown in (a) of FIG. 35, the transmission device PROD_A modulates a carrier wave with the coding unit PROD_A1 that obtains coded data by coding a moving image and the coded data obtained by the coding unit PROD_A1. And a transmitting unit PROD_A3 for transmitting the modulated signal obtained by the modulating unit PROD_A2. The above-described image encoding device 11 is used as the encoding unit PROD_A1.

  The transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 for externally inputting the moving image, and a supply source of the moving image to be input to the encoding unit PROD_A1. , An image processing unit A7 for generating or processing an image. FIG. 35 (a) illustrates a configuration in which the transmitting device PROD_A is provided with all of these components, but a portion thereof may be omitted.

  Note that the recording medium PROD_A5 may be a recording of a moving image that is not encoded, or may record a moving image encoded by a recording encoding method different from the transmission encoding method. It may be something. In the latter case, a decoding unit (not shown) that decodes the encoded data read from the recording medium PROD_A5 in accordance with the encoding method for recording may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.

  FIG. 35B is a block diagram illustrating a configuration of the receiving device PROD_B including the image decoding device 31. As shown in FIG. 35 (b), the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiving unit PROD_B1, A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2. The above-described image decoding device 31 is used as the decoding unit PROD_B3.

  The receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside, as a supply destination of the moving image output by the decoding unit PROD_B3. PROD_B6 may be further provided. FIG. 35 (b) illustrates a configuration in which the receiving device PROD_B includes all of them, but a part of the configuration may be omitted.

  Note that the recording medium PROD_B5 may be for recording a moving image that is not encoded, or may be encoded using a recording encoding method different from the transmission encoding method. You may. In the latter case, an encoding unit (not shown) that encodes the moving image obtained from the decoding unit PROD_B3 according to the encoding method for recording may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

  The transmission medium for transmitting the modulated signal may be wireless or wired. The transmission mode for transmitting the modulated signal may be broadcast (here, a transmission mode in which the transmission destination is not specified in advance), or communication (here, transmission in which the transmission destination is specified in advance). (Which refers to an embodiment). That is, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

  For example, a terrestrial digital broadcast station (such as a broadcasting facility) / a receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting. A broadcasting station (broadcasting facility or the like) / receiving station (television receiver or the like) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

  Servers (workstations, etc.) / Clients (television receivers, personal computers, smartphones, etc.) for VOD (Video On Demand) services and video sharing services using the Internet are transmission devices that transmit and receive modulated signals by communication. This is an example of PROD_A / reception device PROD_B (usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. In addition, the smartphone includes a multifunctional mobile phone terminal.

  The client of the moving image sharing service has a function of decoding encoded data downloaded from the server and displaying the encoded data on a display, and a function of encoding a moving image captured by a camera and uploading the encoded moving image to the server. That is, the client of the video sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

  Next, the fact that the above-described image encoding device 11 and image decoding device 31 can be used for recording and reproduction of a moving image will be described with reference to FIG.

  FIG. 36A is a block diagram illustrating a configuration of a recording device PROD_C in which the above-described image encoding device 11 is mounted. As shown in (a) of FIG. 36, the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and encoded data obtained by the encoding unit PROD_C1 on a recording medium PROD_M. And a writing unit PROD_C2 for writing. The above-described image encoding device 11 is used as the encoding unit PROD_C1.

  The recording medium PROD_M may be (1) a type built in the recording device PROD_C such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or (2) an SD memory. It may be of a type connected to the recording device PROD_C, such as a card or a USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registered) Such as a trademark, for example, may be loaded into a drive device (not shown) built in the recording device PROD_C.

  In addition, the recording device PROD_C includes a camera PROD_C3 that captures a moving image, an input terminal PROD_C4 for externally inputting the moving image, and a reception terminal that receives the moving image, as a supply source of the moving image to be input to the encoding unit PROD_C1. A unit PROD_C5 and an image processing unit PROD_C6 for generating or processing an image may be further provided. FIG. 36A illustrates a configuration in which all of these components are included in the recording device PROD_C, but some of them may be omitted.

  The receiving unit PROD_C5 may receive a moving image that has not been encoded, or may receive encoded data encoded using a transmission encoding method different from the recording encoding method. May be used. In the latter case, a transmission decoding unit (not shown) for decoding encoded data encoded by the transmission encoding method may be interposed between the receiving unit PROD_C5 and the encoding unit PROD_C1.

  Such a recording device PROD_C includes, for example, a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, and the like (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main moving image supply source). . In addition, a camcorder (in this case, the camera PROD_C3 is a main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images), and a smartphone ( In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main source of the moving image) is an example of such a recording device PROD_C.

  (B) of FIG. 36 is a block diagram illustrating a configuration of a playback device PROD_D including the above-described image decoding device 31. As shown in FIG. 36 (b), the playback device PROD_D reads the encoded data written on the recording medium PROD_M, and reads the encoded data read by the read unit PROD_D1 to decode the moving image. And a decoding unit PROD_D2 for obtaining the same. The above-described image decoding device 31 is used as the decoding unit PROD_D2.

  Note that the recording medium PROD_M may be (1) a type built in the playback device PROD_D, such as an HDD or SSD, or (2) a type such as an SD memory card or a USB flash memory. It may be of a type that is connected to the playback device PROD_D, or (3) may be a device such as a DVD or BD that is loaded into a drive device (not shown) built in the playback device PROD_D. Good.

  Also, the playback device PROD_D includes a display PROD_D3 for displaying a moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image, as a supply destination of the moving image output by the decoding unit PROD_D2. PROD_D5 may be further provided. FIG. 36B illustrates a configuration in which the playback device PROD_D includes all of them, but a part of the configuration may be omitted.

  Note that the transmission unit PROD_D5 may transmit an uncoded moving image, or may transmit coded data coded by a transmission coding method different from the recording coding method. May be used. In the latter case, an encoding unit (not shown) for encoding a moving image using a transmission encoding method may be interposed between the decoding unit PROD_D2 and the transmission unit PROD_D5.

  Such a playback device PROD_D includes, for example, a DVD player, a BD player, an HDD player, and the like (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main moving image supply destination). . In addition, a television receiver (in this case, the display PROD_D3 is a main supply destination of a moving image), a digital signage (also referred to as an electronic signboard or an electronic bulletin board, etc.) Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main destination of the moving image), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the video A main supply destination of an image), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main supply destination of a moving image) and the like are also examples of such a playback device PROD_D.

(Hardware realization and software realization)
Also, each block of the image decoding device 31 and the image encoding device 11 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or a CPU (Central Processing Unit). May be implemented by software.

  In the latter case, each of the above devices includes a CPU that executes instructions of a program for realizing each function, a ROM (Read Only Memory) storing the program, a RAM (Random Access Memory) that expands the program, the program and various A storage device (recording medium) such as a memory for storing data is provided. An object of an embodiment of the present invention is to record a program code (executable program, intermediate code program, source program) of a control program of each of the above-described devices, which is software for realizing the above-described functions, in a computer-readable manner. The present invention can also be achieved by supplying a medium to each of the above-described devices and causing a computer (or a CPU or an MPU) to read and execute a program code recorded on a recording medium.

  Examples of the recording medium include tapes such as a magnetic tape and a cassette tape, magnetic disks such as a floppy (registered trademark) disk / hard disk, and CD-ROMs (Compact Disc Read-Only Memory) / MO disks (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (Blu-ray Disc: registered trademark) and other discs including optical discs, IC cards (including memory cards) / Electrically Erasable and Programmable Read-Only Memory (EEPROM) / Electrically Erasable and Programmable Read-Only Memory (registered trademark) / Flash ROM and other semiconductor memories or PLDs (Programmable logic devices) ) Or a logic circuit such as an FPGA (Field Programmable Gate Array).

  Further, each of the devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit a program code. For example, the Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, virtual private network (Virtual Private Network) Network), a telephone line network, a mobile communication network, a satellite communication network, and the like. Further, the transmission medium constituting the communication network may be any medium capable of transmitting the program code, and is not limited to a specific configuration or a specific type. For example, even if a cable such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc., infrared rays such as IrDA (Infrared Data Association) and remote control , BlueTooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It can also be used wirelessly. Note that the embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately changed within the scope of the claims are also included in the technical scope of the present invention.

  The embodiments of the present invention can be suitably applied to an image decoding device that decodes coded data obtained by coding image data, and an image coding device that generates coded data obtained by coding image data. it can. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.

11 ... Image coding device (moving image coding device)
31 ... Image decoding device (moving image decoding device)
302... Prediction parameter decoding unit (prediction image generation device)
309: Inter prediction image generation unit (prediction image generation unit)

Claims (12)

An image decoding apparatus that decodes encoded data obtained by encoding a target image,
A prediction image generation unit that generates a prediction image for each sub-block,
The predicted image generation unit includes:
When a difference between a motion vector of the target sub-block and a motion vector of a sub-block adjacent to the target sub-block is equal to or smaller than a predetermined value, a predicted image generation process using the OBMC process is performed. Image decoding device. The predicted image generation unit includes:
When the difference between the motion vector of the target sub-block and the motion vector of the sub-block adjacent to the target sub-block is not 0 and is smaller than a predetermined value, the predicted image generation process using the OBMC process is performed. The image decoding device according to claim 1, wherein the image decoding is performed. The predicted image generation unit includes:
When the motion vector of the target sub-block is the same as the motion vector of the sub-block included in the block including the target sub-block and adjacent to the target sub-block, a predicted image using the OBMC process Perform generation processing,
The image decoding apparatus according to claim 1, wherein the OBMC process is performed on a sub-block obtained by combining the target sub-block and a sub-block adjacent to the target sub-block. The predicted image generation unit includes:
When the motion vector of the target sub-block is the same as the motion vector of the sub-block included in the block including the target sub-block and horizontally adjacent to the target sub-block, the OBMC process is performed. Perform a predicted image generation process using
The image decoding apparatus according to claim 3, wherein the OBMC processing is performed on a sub-block obtained by combining the target sub-block and a sub-block horizontally adjacent to the target sub-block. An image decoding apparatus that decodes encoded data obtained by encoding a target image,
A prediction image generation unit that generates a prediction image for each block,
The predicted image generation unit includes:
When the target sub-block is a sub-block of a predetermined size or less, and the target sub-block is a bidirectional prediction sub-block, the motion information of the adjacent sub-block adjacent to the target sub-block with respect to the target sub-block. An image decoding apparatus, wherein the same motion information as above is assigned to generate the predicted image. The predicted image generation unit includes:
The image according to claim 5, wherein the prediction image is generated by allocating an average vector of a motion vector of the target sub-block and a motion vector of the adjacent sub-block to the target sub-block. Decoding device. An image encoding device that encodes a residual between a predicted image and an encoding target image,
A prediction image generation unit that generates a prediction image for each sub-block,
The predicted image generation unit includes:
When a difference between a motion vector of the target sub-block and a motion vector of a sub-block adjacent to the target sub-block is equal to or smaller than a predetermined value, a predicted image generation process using the OBMC process is performed. Image coding device. The predicted image generation unit includes:
When the difference between the motion vector of the target block and the motion vector of a sub-block adjacent to the target sub-block is not 0 and is equal to or smaller than a predetermined value, a predicted image generation process using the OBMC process is performed. The image encoding device according to claim 7, wherein: The predicted image generation unit includes:
When the motion vector of the target sub-block is the same as the motion vector of the sub-block included in the block including the target sub-block and adjacent to the target sub-block, a predicted image using the OBMC process The image encoding device according to claim 7, wherein the image encoding device performs a generation process. The predicted image generation unit includes:
When the motion vector of the target sub-block is the same as the motion vector of the sub-block included in the block including the target sub-block and horizontally adjacent to the target sub-block, the OBMC process is performed. The image encoding apparatus according to claim 9, wherein a prediction image generation process is performed using the image encoding. An image encoding device that encodes a residual between a predicted image and an encoding target image,
A prediction image generation unit that generates a prediction image for each sub-block,
The predicted image generation unit includes:
When the target sub-block is a block having a predetermined size or less and the target sub-block is a bidirectional prediction sub-block, the motion information of the adjacent sub-block adjacent to the target sub-block is included in the target sub-block. An image encoding device, wherein the same motion information is assigned to generate the predicted image. The predicted image generation unit includes:
The image according to claim 11, wherein the prediction image is generated by allocating an average vector of a motion vector of the target sub-block and a motion vector of the adjacent sub-block to the target sub-block. Encoding device.

Images