JP7048503B2 - Decoding device, coding device, decoding method, and coding method - Google Patents

Description

The present invention relates to a motion vector generator, a predictive image generator, a motion image decoding device, and a motion image coding device.

In order to efficiently transmit or record a moving image, a moving image coding device that generates coded data by encoding the moving image, and a moving image that generates a decoded image by decoding the coded data. An image decoding device is used.

Specific examples of the moving image coding method include the methods proposed by H.264 / AVC and HEVC (High-Efficiency Video Coding).

In such a moving image coding method, the image (picture) constituting the moving image is a slice obtained by dividing the image and a coding unit (coding unit) obtained by dividing the slice. : CU)), and managed by a hierarchical structure consisting of a prediction unit (PU) and a conversion unit (TU), which are blocks obtained by dividing the coding unit, and coded for each CU / It is decrypted.

Further, in such a moving image coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and the predicted image is obtained from the input image (original image). The predicted residual obtained by subtraction (sometimes referred to as a "difference image" or "residual image") is encoded. Examples of the method for generating the predicted image include inter-screen prediction (inter-prediction) and in-screen prediction (intra-prediction).

Further, Non-Patent Document 1 is mentioned as a technique for coding and decoding moving images in recent years. Non-Patent Document 1 discloses a technique for reducing a code amount related to transmission of a motion vector by searching for a motion vector by matching in a coding device and a decoding device.

Video / JVET, "Algorithm Description of Joint Exploration TestModel 1 (JEM 1)", INTERNATIONAL ORGANIZATION FOR STANDARDIZATION ORGANISATION INTERNATIONALE DE NORMALISATION ISO / IEC JTC1 / SC29 / WG11 CODING OF MOVING PICTURES AND AUDIO, ISO / IEC JTC1 / SC29 / WG11 / N15790, October 2015, Geneva, CH.

In recent moving image coding and decoding techniques, the first problem that the amount of coded data is increased has arisen. Further, in the matching technique disclosed in Non-Patent Document 1, there is a second problem that the amount of processing for the motion vector search required for predictive image generation increases.

The present invention is to provide an image decoding device, an image coding device, and a predictive image generation device capable of solving at least one of the above-mentioned first and second problems.

In order to solve the above problems, the motion vector generator according to one aspect of the present invention generates a motion vector that is referred to for generating a predicted image used for encoding or decoding a motion image. In the apparatus, the motion vector is included in the prediction block with reference to the first motion vector search unit that searches for the motion vector for each prediction block by the matching process and the motion vector selected by the first motion vector search unit. Each of the plurality of sub-blocks is provided with a second motion vector search unit that searches for a motion vector by matching processing, and the motion vector generator is the first motion vector search unit for a certain target prediction block. In, the configuration includes an operation of performing a local search by template matching and performing a local search using bilateral matching in the second motion vector search unit for the target prediction block.

In order to solve the above problems, the motion vector generator according to one aspect of the present invention generates a motion vector that is referred to for generating a predicted image used for encoding or decoding a motion image. In the apparatus, the motion vector is included in the prediction block with reference to the first motion vector search unit that searches for the motion vector for each prediction block by the matching process and the motion vector selected by the first motion vector search unit. Each of the plurality of subblocks is provided with a motion vector derivation unit for deriving a motion vector by matching processing, and the first motion vector search unit performs a local search after performing an initial vector search for a prediction block. The motion vector is searched for by performing the above, and the motion vector derivation unit has a configuration in which the motion vector derivation method is different depending on the position of the subblock to be derived in the prediction block.

In order to solve the above problems, the motion vector generator according to one aspect of the present invention generates a motion vector that is referred to for generating a predictive image used for encoding or decoding a motion image. In the motion vector generator that derives the motion vector for each prediction block by AMVP (Adaptive Motion Vector Prediction), the prediction vector candidate used for deriving the motion vector is derived and stored in the prediction vector candidate list. The prediction vector candidate derivation unit includes a prediction vector candidate derivation unit, and the prediction vector candidate derivation unit performs a local search after performing an initial vector search on a target block for one of the prediction vector candidates, and performs a matching process to perform a motion vector. Is derived, and the same prediction vector candidate as the initial vector searched by the initial vector search is not stored in the prediction vector candidate list.

It is a motion vector generator that generates a motion vector that is referenced to generate a motion vector used for encoding or decoding a motion image, and is a motion vector that derives a motion vector for each prediction block by AMVP (Adaptive Motion Vector Prediction). The vector generator includes a prediction vector candidate derivation unit that derives a prediction vector candidate used for deriving the motion vector and stores it in the prediction vector candidate list. The prediction vector candidate derivation unit is one of the prediction vector candidates. In addition to deriving one by the matching process, the prediction vector candidate derived by the matching process is always stored at the top of the prediction vector candidate list.

According to the above configuration, it is possible to solve at least one of the above-mentioned first and second problems.

It is a figure which shows the hierarchical structure of the data of the coded stream which concerns on this embodiment. It is a figure which shows the pattern of a PU division mode. (A) to (h) show the partition shape when the PU division modes are 2Nx2N, 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and NxN, respectively. It is a conceptual diagram which shows an example of a reference picture and a reference picture list. It is a block diagram which shows the structure of the image coding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the image decoding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the inter prediction image generation part of the image coding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the merge prediction parameter derivation part which concerns on this embodiment. It is a schematic diagram which shows the structure of the AMVP prediction parameter derivation part which concerns on this embodiment. It is a flowchart which shows the operation of the motion vector decoding process of the image decoding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the inter prediction parameter coding part of the image coding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the inter prediction image generation part which concerns on this embodiment. It is a schematic diagram which shows the structure of the inter prediction parameter decoding unit which concerns on this embodiment. It is a figure which shows the example which derives the motion vector spMvLX [xi] [yi] of each subblock constituting PU (width nPbW) which is the object for predicting a motion vector. (A) is a figure for explaining bilateral matching. (B) is a diagram for explaining template matching. It is a flowchart which shows the outline of the motion prediction mode determination flow. It is a sequence diagram which shows the flow of the motion prediction mode determination flow. It is a flowchart which shows the flow of the pattern match vector derivation process. (A) to (d) are diagrams for explaining the pattern match vector derivation process. (A) and (b) are diagrams for explaining a motion search pattern. It is a figure which shows the harmful effect when performing template matching in a subblock. At the sub-block level, it is a flowchart showing the flow of the pattern match vector derivation process when the pattern matching is fixed to bilateral matching. It is a flowchart which shows the flow of the pattern matching vector derivation process at the time of performing the performance estimation of bilateral matching. It is a figure for demonstrating the method of using bilateral matching for the subblock which does not touch the upper end or the left end of the target block. It is a flowchart which shows the flow of the process which determines the pattern matching or bilateral matching according to the position in a subblock. It is a flowchart which shows the flow of the process which uses the motion vector of the peripheral sub-block according to the position of a sub-block. (A) to (e) are diagrams showing an example of a method of deriving a motion vector of a target subblock using a motion vector of a peripheral block. It is a flowchart which shows the flow of the prediction vector candidate derivation process. It is a figure for demonstrating the derivation process of a vector candidate. It is a flowchart which shows the flow of the pattern match vector candidate derivation process. It is a figure which shows the example of the pseudo code for making the prediction vector candidate list. (A) and (b) are diagrams showing a block as a template used when deriving a pattern match vector. It is a flowchart which shows the flow of the prediction vector candidate derivation process. It is a figure which shows the example of the pseudo code for making the prediction vector candidate list. It is a flowchart which shows the flow of the prediction vector candidate derivation process. It is a figure which shows the example of the pseudo code for making the prediction vector candidate list. It is a flowchart which shows the flow of the prediction vector candidate derivation process. It is a figure which shows the example of the pseudo code for making the prediction vector candidate list. It is a figure which showed the structure of the transmission device equipped with the image coding device, and the receiving device equipped with an image decoding device which concerns on this embodiment. (A) shows a transmitting device equipped with an image coding device, and (b) shows a receiving device equipped with an image decoding device. It is a figure which showed the structure of the recording apparatus equipped with the image coding apparatus which concerns on this embodiment, and the reproduction apparatus equipped with the image decoding apparatus. (A) shows a recording device equipped with an image coding device, and (b) shows a playback device equipped with an image decoding device. It is a schematic diagram which shows the structure of the image transmission system which concerns on this embodiment.

[Embodiment 1]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 40 is a schematic diagram showing the configuration of the image transmission system 1 according to the present embodiment.

The image transmission system 1 is a system that transmits a code obtained by encoding a coded image, decodes the transmitted code, and displays the image. The image transmission system 1 includes an image coding device (moving image coding device, predicted image generation device) 11, a network 21, an image decoding device (moving image decoding device, predicted image generation device) 31, and an image display device 41. It is composed.

An image T showing an image of a single layer or a plurality of layers is input to the image coding device 11. A layer is a concept used to distinguish a plurality of pictures when there is one or more pictures constituting a certain time. For example, encoding the same picture with a plurality of layers having different image quality and resolution results in scalable coding, and coding a picture with different viewpoints with a plurality of layers results in view scalable coding. When prediction (interlayer prediction, interview prediction) is performed between pictures of a plurality of layers, the coding efficiency is greatly improved. In addition, even when prediction is not performed (simulcast), the coded data can be collected.

The network 21 transmits the coded stream Te generated by the image coding device 11 to the image decoding device 31. The network 21 is an internet (internet), a wide area network (WAN: Wide Area Network), a small-scale network (LAN: Local Area Network), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, but may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Further, the network 21 may be replaced with a storage medium such as a DVD (Digital Versatile Disc) or BD (Blue-ray Disc) on which a coded stream Te is recorded.

The image decoding device 31 decodes each of the coded streams Te transmitted by the network 21 and generates one or a plurality of decoded images Td respectively.

The image display device 41 displays all or a part of one or a plurality of decoded images Td generated by the image decoding device 31. The image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Further, in spatial scalable coding and SNR scalable coding, when the image decoding device 31 and the image display device 41 have high processing power, an extended layer image with high image quality is displayed and only lower processing power is obtained. Displays a base layer image that does not require as high processing power and display power as the extended layer.

<Operator>
The operators used herein are described below.

>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, and | = is a sum operation (OR) with another condition.

x? y: z is a ternary operator that takes y when x is true (other than 0) and z when x is false (0).

Clip3 (a, b, c) is a function that clips c to a value greater than or equal to a and less than or equal to b. Is a function that returns c (where a <= b).

<Structure of coded stream Te>
Prior to the detailed description of the image coding device 11 and the image decoding device 31 according to the present embodiment, the data structure of the coded stream Te generated by the image coding device 11 and decoded by the image decoding device 31 will be described. ..

FIG. 1 is a diagram showing a hierarchical structure of data in the coded stream Te. The coded stream Te typically includes a sequence and a plurality of pictures constituting the sequence. 1 (a) to 1 (f) of FIG. 1 are a coded video sequence that defines a sequence SEQ, a coded picture that defines a picture PICT, a coded slice that defines a slice S, and a coded slice that defines slice data, respectively. It is a figure which shows the coding unit (CU) included in the data, the coding tree unit included in the coding slice data, and the coding tree unit.

(Coded video sequence)
The coded video sequence defines a set of data that the image decoder 31 refers to in order to decode the sequence SEQ to be processed. As shown in FIG. 1A, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an addition. Includes SEI (Supplemental Enhancement Information). Here, the value shown after # indicates the layer ID. FIG. 1 shows an example in which coded data of # 0 and # 1, that is, layer 0 and layer 1 exist, but the type of layer and the number of layers do not depend on this.

A video parameter set VPS is a set of coding parameters common to a plurality of moving images in a moving image composed of a plurality of layers, and a set of coding parameters related to the multiple layers included in the moving image and individual layers. The set is defined.

The sequence parameter set SPS defines a set of coding parameters that the image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are specified. There may be a plurality of SPS. In that case, select one of multiple SPS from PPS.

The picture parameter set PPS defines a set of coding parameters that the image decoding device 31 refers to in order to decode each picture in the target sequence. For example, it includes a reference value of the quantization width used for decoding a picture (pic_init_qp_minus26) and a flag (weighted_pred_flag) indicating the application of weighted prediction. There may be a plurality of PPS. In that case, select one of a plurality of PPSs from each picture in the target sequence.

(Encoded picture)
The coded picture defines a set of data referred to by the image decoding device 31 in order to decode the picture PICT to be processed. As shown in FIG. 1 (b), the picture PICT includes slices S0 to SNS-1 (NS is the total number of slices contained in the picture PICT).

In the following, when it is not necessary to distinguish each of slices S0 to SNS-1, the subscript of the code may be omitted. The same applies to the data included in the coded stream Te described below and having a subscript.

(Coded slice)
The coded slice defines a set of data referred to by the image decoding device 31 in order to decode the slice S to be processed. As shown in FIG. 1 (c), the slice S includes the slice header SH and the slice data SDATA.

The slice header SH includes a group of coding parameters referred to by the image decoding device 31 for determining the decoding method of the target slice. The slice type specification information (slice_type) that specifies the slice type is an example of the coding parameter included in the slice header SH.

The slice types that can be specified by the slice type specification information are (1) I slice that uses only intra prediction at the time of coding, (2) unidirectional prediction at the time of coding, or P slice that uses intra prediction. (3) B slices using unidirectional prediction, bidirectional prediction, or intra prediction at the time of coding can be mentioned.

The slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the coded video sequence.

(Coded slice data)
The coded slice data defines a set of data referred to by the image decoding device 31 in order to decode the slice data SDATA to be processed. The slice data SDATA includes a coding tree unit (CTU) as shown in FIG. 1 (d). A CTU is a block of fixed size (for example, 64x64) that constitutes a slice, and is sometimes called a maximum coding unit (LCU).

(Coded tree unit)
As shown in FIG. 1 (e), a set of data referred to by the image decoding device 31 for decoding the coded tree unit to be processed is defined. The coded tree unit is divided by a recursive quadtree division. A node with a tree structure obtained by recursive quadtree division is called a coding node (CN). The middle node of the quadtree is a coding node, and the coding tree unit itself is also defined as the top-level coding node. The CTU includes a split flag (cu_split_flag), and when cu_split_flag is 1, it is split into four coding nodes CN. When cu_split_flag is 0, the coding node CN is not divided and has one coding unit (CU: Coding Unit) as a node. The coding unit CU is the terminal node of the coding node and is not further divided. The coding unit CU is a basic unit of coding processing.

When the size of the coding tree unit CTU is 64x64 pixels, the size of the coding unit can be any of 64x64 pixels, 32x32 pixels, 16x16 pixels, and 8x8 pixels.

(Encoding unit)
As shown in FIG. 1 (f), a set of data referred to by the image decoding device 31 for decoding the coding unit to be processed is defined. Specifically, the coding unit is composed of a prediction tree, a conversion tree, and a CU header CUH. The CU header defines the prediction mode, division method (PU division mode), and the like.

In the prediction tree, prediction information (reference picture index, motion vector, etc.) of each prediction unit (PU) obtained by dividing the coding unit into one or a plurality is defined. In other words, the prediction unit is one or more non-overlapping regions that make up the coding unit. The prediction tree also includes one or more prediction units obtained by the above division. In the following, the prediction unit obtained by further dividing the prediction unit is referred to as a “sub-block”. The subblock is composed of a plurality of pixels. If the predictor unit and the subblock are the same size, then there is only one subblock in the predictor unit. If the predictor unit is larger than the size of the subblock, the predictor unit is divided into subblocks. For example, when the prediction unit is 8x8 and the subblock is 4x4, the prediction unit is divided into four subblocks consisting of two horizontal divisions and two vertical divisions.

The prediction process may be performed for each prediction unit (subblock).

Roughly speaking, there are two types of division in the prediction tree: intra-prediction and inter-prediction. The intra prediction is a prediction within the same picture, and the inter prediction refers to a prediction process performed between pictures different from each other (for example, between display times and between layer images).

In the case of intra prediction, there are two division methods: 2Nx2N (same size as the coding unit) and NxN.

In the case of inter-prediction, the division method is encoded by the PU division mode (part_mode) of the coded data, 2Nx2N (same size as the coding unit), 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and There are NxN and so on. Note that 2NxN and Nx2N indicate a 1: 1 symmetric division, and 2NxnU, 2NxnD and nLx2N and nRx2N indicate a 1: 3 and 3: 1 asymmetric division. The PUs included in the CU are expressed as PU0, PU1, PU2, and PU3 in that order.

2 (a) to 2 (h) specifically show the shape of the partition (position of the boundary of the PU division) in each PU division mode. (A) of FIG. 2 shows a partition of 2Nx2N, and (b), (c), and (d) show a partition (horizontal partition) of 2NxN, 2NxnU, and 2NxnD, respectively. (E), (f), and (g) indicate partitions (vertical partitions) in the case of Nx2N, nLx2N, and nRx2N, respectively, and (h) indicates an NxN partition. The horizontally long partition and the vertically long partition are collectively called a rectangular partition, and 2Nx2N and NxN are collectively called a square partition.

Further, in the conversion tree, the coding unit is divided into one or a plurality of conversion units, and the position and size of each conversion unit are defined. In other words, the conversion unit is one or more non-overlapping regions constituting the coding unit. The conversion tree also includes one or more conversion units obtained from the above divisions.

There are two types of division in the conversion tree: one that allocates an area of the same size as the coding unit as the conversion unit, and one that recursively divides into quadtrees, similar to the above-mentioned division of CU.

The conversion process is performed for each conversion unit.

(Prediction parameter)
The prediction image of the prediction unit (PU) is derived by the prediction parameters associated with the PU. Prediction parameters include prediction parameters for intra-prediction or prediction parameters for inter-prediction. Hereinafter, the prediction parameters of the inter-prediction (inter-prediction parameters) will be described. The inter-prediction parameter is composed of the prediction list utilization flags predFlagL0 and predFlagL1, the reference picture indexes refIdxL0 and refIdxL1, and the motion vectors mvL0 and mvL1. The prediction list usage flags predFlagL0 and predFlagL1 are flags indicating whether or not the reference picture list called the L0 list and the L1 list is used, respectively, and the reference picture list corresponding to the case where the value is 1 is used. In the present specification, when "a flag indicating whether or not it is XX" is described, it is assumed that a flag other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. Treat 1 as true and 0 as false (same below). However, in an actual device or method, other values can be used as true values and false values.

The syntax elements for deriving the inter-prediction parameters contained in the coded data include, for example, PU split mode part_mode, merge flag merge_flag, merge index merge_idx, inter-prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, There is a difference vector mvdLX.

(Reference picture list)
The reference picture list is a list composed of reference pictures stored in the reference picture memory 306. FIG. 3 is a conceptual diagram showing an example of a reference picture and a reference picture list. In FIG. 3A, the rectangle is a picture, the arrow is a reference relationship of the picture, the horizontal axis is time, I, P, and B in the rectangle are intra pictures, single prediction pictures, bi-prediction pictures, and the numbers in the rectangle are. Indicates the decoding order. As shown in the figure, the decoding order of the pictures is I0, P1, B2, B3, B4, and the display order is I0, B3, B2, B4, P1. FIG. 3B shows an example of a reference picture list. The reference picture list is a list representing candidates for reference pictures, and one picture (slice) may have one or more reference picture lists. In the example of the figure, the target picture B3 has two reference picture lists, L0 list RefPicList0 and L1 list RefPicList1. When the target picture is B3, the reference pictures are I0, P1, and B2, and the reference picture has these pictures as elements. In each prediction unit, the reference picture index refIdxLX specifies which picture in the reference picture list RefPicListX is actually referenced. The figure shows an example in which refIdxL0 and refIdxL1 refer to reference pictures P1 and B2.

(Merge prediction and AMVP prediction)
Prediction parameter decoding (encoding) methods include a merge mode and an AMVP (Adaptive Motion Vector Prediction) mode. The merge flag merge_flag is a flag for identifying these. The merge prediction mode is a mode used to derive the prediction list utilization flag predFlagLX (or inter prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the motion vector mvLX from the prediction parameters of the neighboring PUs that have already been processed without including them in the coded data. .. The AMVP mode is a mode in which the inter-prediction identifier inter_pred_idc, the reference picture index refIdxLX, and the motion vector mvLX are included in the coded data. The motion vector mvLX is encoded as a prediction vector index mvp_LX_idx that identifies the prediction vector mvpLX and a difference vector mvdLX.

The inter-prediction identifier inter_pred_idc is a value indicating the type and number of reference pictures, and takes any value of PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate that the reference pictures managed by the reference picture list of the L0 list and the L1 list are used, respectively, and indicate that one reference picture is used (single prediction). PRED_BI indicates that two reference pictures are used (bipred), and the reference pictures managed by the L0 list and the L1 list are used. The prediction vector index mvp_LX_idx is an index indicating the prediction vector, and the reference picture index refIdxLX is an index indicating the reference pictures managed by the reference picture list. LX is a description method used when L0 prediction and L1 prediction are not distinguished, and by replacing LX with L0 and L1, the parameters for the L0 list and the parameters for the L1 list are distinguished.

The merge index merge_idx is an index indicating which of the prediction parameter candidates (merge candidates) derived from the PU for which processing has been completed is used as the prediction parameter of the PU to be decoded.

(Motion vector)
The motion vector mvLX indicates the amount of deviation between blocks on two different pictures. The prediction vector and difference vector related to the motion vector mvLX are called the prediction vector mvpLX and the difference vector mvdLX, respectively.

(Inter prediction identifier inter_pred_idc and prediction list usage flag predFlagLX)
The relationship between the inter-prediction identifier inter_pred_idc and the prediction list usage flags predFlagL0 and predFlagL1 is as follows and can be converted to each other.

inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
As the inter-prediction parameter, the prediction list use flag may be used, or the inter-prediction identifier may be used. Further, the determination using the prediction list utilization flag may be replaced with the determination using the inter-prediction identifier. On the contrary, the determination using the inter-prediction identifier may be replaced with the determination using the prediction list utilization flag.

(Judgment of bipred biPred)
The bipred BiPred flag biPred can be derived depending on whether the two prediction list usage flags are both 1. For example, it can be derived by the following formula.

biPred = (predFlagL0 == 1 && predFlagL1 == 1)
The flag biPred can also be derived by whether or not the inter-prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived by the following formula.

biPred = (inter_pred_idc == PRED_BI)? 1: 0
The above equation can also be expressed by the following equation.

biPred = (inter_pred_idc == PRED_BI)
Note that PRED_BI can use a value of 3, for example.

(Configuration of image decoder)
Next, the configuration of the image decoding device 31 according to the present embodiment will be described. FIG. 5 is a schematic view showing the configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, and a reverse. It includes a quantization / inverse DCT unit 311 and an addition unit 312.

Further, the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit (motion vector generator) 303 and an intra prediction parameter decoding unit 304. The prediction image generation unit 308 includes an inter prediction image generation unit 309 and an intra prediction image generation unit 310.

The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, separates and decodes each code (syntax element). The separated codes include prediction information for generating a prediction image, residual information for generating a difference image, and the like.

The entropy decoding unit 301 outputs a part of the separated codes to the prediction parameter decoding unit 302. Some of the separated codes are, for example, the prediction mode predMode, the PU split mode part_mode, the merge flag merge_flag, the merge index merge_idx, the interprediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx, and the difference vector mvdLX. The control of which code is decoded is performed based on the instruction of the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantization coefficient to the inverse quantization / inverse DCT unit 311. This quantization coefficient is a coefficient obtained by performing DCT (Discrete Cosine Transform) on the residual signal and quantizing it in the coding process.

The inter-prediction parameter decoding unit 303 decodes the inter-prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301.

The inter-prediction parameter decoding unit 303 outputs the decoded inter-prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307. The details of the inter-prediction parameter decoding unit 303 will be described later.

The intra prediction parameter decoding unit 304 decodes the intra prediction parameter by referring to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. The intra prediction parameter is a parameter used in the process of predicting the CU in one picture, for example, the intrapred mode. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.

The intra prediction parameter decoding unit 304 may derive an intra prediction mode that differs depending on the luminance and the color difference. In this case, the intra prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the luminance prediction mode IntraPredModeC as the color difference prediction parameter. The luminance prediction mode IntraPredModeY is 35 modes, and corresponds to planar prediction (0), DC prediction (1), and direction prediction (2 to 34). The color difference prediction mode IntraPredModeC uses any one of planar prediction (0), DC prediction (1), direction prediction (2-34), and LM mode (35). The intra prediction parameter decoding unit 304 decodes a flag indicating whether or not IntraPredModeC is in the same mode as the luminance mode, and if it shows that the flag is in the same mode as the luminance mode, assigns IntraPredModeY to IntraPredModeC and the flag is the luminance mode. If it is shown that the mode is different from the mode, the planar prediction (0), DC prediction (1), direction prediction (2-34), and LM mode (35) may be decoded as IntraPredModeC.

The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the addition unit 312.

The reference picture memory 306 stores the decoded image of the CU generated by the addition unit 312 at a position predetermined for each of the picture to be decoded and the CU.

The prediction parameter memory 307 stores the prediction parameters in a predetermined position for each picture and prediction unit (or subblock, fixed size block, pixel) to be decoded. Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode pred Mode separated by the entropy decoding unit 301. .. The stored inter-prediction parameters include, for example, the prediction list utilization flag predFlagLX (inter-prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the motion vector mvLX.

The prediction mode predMode input from the entropy decoding unit 301 is input to the prediction image generation unit 308, and the prediction parameter is input from the prediction parameter decoding unit 302. Further, the prediction image generation unit 308 reads the reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of the PU using the input prediction parameter and the read reference picture in the prediction mode indicated by the prediction mode predMode.

Here, when the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the reference picture read out, and the prediction image of the PU is predicted by the inter prediction. To generate.

The inter-prediction image generation unit 309 uses a motion vector as a reference from the reference picture indicated by the reference picture index refIdxLX to the reference picture list (L0 list or L1 list) in which the prediction list usage flag predFlagLX is 1. The reference picture block at the position indicated by mvLX is read from the reference picture memory 306. The inter-predicted image generation unit 309 performs prediction based on the read reference picture block and generates a predicted image of the PU. The inter-prediction image generation unit 309 outputs the generated prediction image of the PU to the addition unit 312.

When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs the intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the reference picture read out. Specifically, the intra prediction image generation unit 310 reads from the reference picture memory 306 an adjacent PU that is a picture to be decoded and is within a predetermined range from the PU to be decoded. When the decoding target PU moves sequentially in the order of so-called raster scan, the predetermined range is, for example, one of the adjacent PUs on the left, upper left, upper, and upper right, and differs depending on the intra prediction mode. The order of raster scan is the order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.

The intra prediction image generation unit 310 predicts the read adjacent PU in the prediction mode indicated by the intra prediction mode IntraPredMode, and generates a prediction image of the PU. The intra prediction image generation unit 310 outputs the generated prediction image of the PU to the addition unit 312.

When the intra prediction parameter decoding unit 304 derives an intra prediction mode different in luminance and color difference, the intra prediction image generation unit 310 determines the planar prediction (0), DC prediction (1), and direction according to the luminance prediction mode IntraPredModeY. A predicted image of the luminance PU is generated by any of the predictions (2 to 34), and the planar prediction (0), DC prediction (1), direction prediction (2 to 34), and LM mode are generated according to the color difference prediction mode IntraPredModeC. A predicted image of the PU of color difference is generated by any of (35).

The inverse quantization / inverse DCT unit 311 inversely quantizes the quantization coefficient input from the entropy decoding unit 301 to obtain the DCT coefficient. The inverse quantization / inverse DCT unit 311 performs an inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient and calculates a residual signal. The inverse quantization / inverse DCT unit 311 outputs the calculated residual signal to the addition unit 312.

The addition unit 312 adds the prediction image of the PU input from the inter-prediction image generation unit 309 or the intra-prediction image generation unit 310 and the residual signal input from the inverse quantization / inverse DCT unit 311 for each pixel. Generate a decrypted image of PU. The addition unit 312 stores the generated PU decoded image in the reference picture memory 306, and outputs the decoded image Td in which the generated PU decoded image is integrated for each picture to the outside.

(Structure of inter-prediction parameter decoding unit)
Next, the configuration of the inter-prediction parameter decoding unit 303 will be described.

FIG. 12 is a schematic diagram showing the configuration of the inter-prediction parameter decoding unit 303 according to the present embodiment. The inter prediction parameter decoding unit 303 includes an inter prediction parameter decoding control unit 3031, an AMVP prediction parameter derivation unit 3032, an addition unit 3035, a merge prediction parameter derivation unit 3036, and a subblock prediction parameter derivation unit (motion vector derivation unit) 3037. It is composed.

The inter-prediction parameter decoding control unit 3031 instructs the entropy decoding unit 301 to decode the code (syntax element) related to the inter-prediction, and the code (syntax element) included in the coded data, for example, the PU division mode part_mode. , Merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX.

The inter-prediction parameter decoding control unit 3031 first extracts the merge flag merge_flag. When the inter-prediction parameter decoding control unit 3031 expresses that a certain syntax element is extracted, it means that the entropy decoding unit 301 is instructed to decode the certain syntax element and the corresponding syntax element is read from the coded data. do.

When the merge flag merge_flag indicates 0, that is, the AMVP prediction mode, the inter-prediction parameter decoding control unit 3031 extracts the AMVP prediction parameter from the coded data using the entropy decoding unit 301. AMVP prediction parameters include, for example, the inter prediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx, and the difference vector mvdLX. The AMVP prediction parameter derivation unit 3032 derives the prediction vector mvpLX from the prediction vector index mvp_LX_idx. Details will be described later. The inter-prediction parameter decoding control unit 3031 outputs the difference vector mvdLX to the addition unit 3035. The addition unit 3035 adds the prediction vector mvpLX and the difference vector mvdLX to derive a motion vector.

When the merge flag merge_flag indicates 1, that is, the merge prediction mode, the inter-prediction parameter decoding control unit 3031 extracts the merge index merge_idx as a prediction parameter related to the merge prediction. The inter-prediction parameter decoding control unit 3031 outputs the extracted merge index merge_idx to the merge prediction parameter derivation unit 3036 (details will be described later), and outputs the sub-block prediction mode flag subPbMotionFlag to the sub-block prediction parameter derivation unit 3037. The sub-block prediction parameter derivation unit 3037 divides the PU into a plurality of sub-blocks according to the value of the sub-block prediction mode flag subPbMotionFlag, and derives a motion vector in sub-block units. That is, in the sub-block prediction mode, the prediction block is predicted in small block units of 4x4 or 8x8. In the image coding device 11 described later, the subblock prediction mode is used as opposed to the method of dividing the CU into a plurality of partitions (PUs such as 2NxN, Nx2N, NxN) and encoding the syntax of the prediction parameters for each partition. Then, since a plurality of sub-blocks are grouped into a set and the syntax of the prediction parameter is coded for each set, the motion information of many sub-blocks can be coded with a small amount of code.

More specifically, the sub-block prediction parameter derivation unit 3037 performs a sub-block prediction in the sub-block prediction mode, a space-time sub-block prediction unit 30371, an affine prediction unit 30372, and a matching prediction unit (first motion vector search unit). , Second motion vector search unit) 30373.

(Subblock prediction mode flag)
Here, a method of deriving the subblock prediction mode flag subPbMotionFlag indicating whether or not the prediction mode of a certain PU is the subblock prediction mode in the image coding device 11 (details will be described later) will be described. The image coding device 11 derives the subblock prediction mode flag subPbMotionFlag based on which of the spatial subblock prediction SSUB, the time subblock prediction TSUB, the affine prediction AFFINE, and the matching prediction MAT, which will be described later, is used. For example, when the prediction mode selected by a certain PU is N (for example, N is a label indicating the selected merge candidate), the subblock prediction mode flag subPbMotionFlag may be derived by the following formula.

subPbMotionFlag = (N == TSUB) || (N == SSUB) || (N == AFFINE) || (N == MAT)
Here, || indicates the logical sum (the same applies hereinafter).

Further, as described below, the above equation may be appropriately changed depending on the type of mode of subblock prediction performed by the image coding device 11. That is, when the image coding device 11 is configured to perform spatial subblock prediction SSUB and affine prediction AFFINE, the subblock prediction mode flag subPbMotionFlag may be derived as follows.

subPbMotionFlag = (N == SSUB) || (N == AFFINE)
In addition, when prediction is performed by each prediction mode included in subblock prediction (for example, spatiotemporal subblock prediction, affine prediction, matching prediction), subPbMotionFlag is included in the processing of the prediction mode corresponding to each subblock prediction. May be set to 1.

Also, for example, if the CU size is 8x8 (logarithmic CU size log2CbSize == 3) and the division type is other than 2Nx2N and the PU is a small size, the PU can be a subblock with the number of divisions 1. .. In this case, the subblock prediction mode subPbMotionFlag may be derived as follows.

subPbMotionFlag | = (log2CbSize == 3 && PartMode! = 2Nx2N)
Note that | = means that the subPbMotionFlag may be derived by a sum operation (OR) with another condition. That is, the subPbMotionFlag may be derived by the sum operation of the determination of the prediction mode N and the determination of the small PU size as follows (the same applies hereinafter).

subPbMotionFlag = (N == TSUB) || (N == SSUB) || (N == AFFINE) || (N == MAT) || (log2CbSize == 3 && PartMode! = 2Nx2N)
Further, for example, the case where the CU size is 8x8 (log2CbSize == 3) and the division type is 2NxN, Nx2N, NxN may be included in the subblock prediction. That is, the subPbMotionFlag may be derived as follows.

subPbMotionFlag | = (log2CbSize == 3 && (PartMode == 2NxN || PartMode == Nx2N || PartMode == NxN))
Further, for example, the case where the CU size is 8x8 (log2CbSize == 3) and the division type is NxN may be included in the subblock prediction. That is, the subPbMotionFlag may be derived as follows.

subPbMotionFlag | = (log2CbSize == 3 && PartMode == NxN)
Further, as a case of determining the sub-block prediction, a case where the width or height of the PU is 4 may be included. That is, the subblock prediction mode flag subPbMotionFlag may be derived as follows.

subPbMotionFlag | = (nPbW == 4 || nPbH == 4)
The sub-block prediction parameter derivation unit 3037 of the image decoding device 31 derives the sub-block prediction mode from the subPbMotionFlag by the reverse method of the above.

(Sub-block prediction unit)
Next, the sub-block prediction unit will be described.

(Space-time subblock prediction unit 30371)
The spatiotemporal subblock prediction unit 30371 is based on the motion vector of the PU on the reference image (for example, the immediately preceding picture) adjacent to the target PU in time, or the motion vector of the PU spatially adjacent to the target PU. The motion vector of the subblock obtained by dividing is derived. Specifically, by scaling the motion vector of the PU on the reference image to match the reference picture referenced by the target PU, the motion vector of each subblock in the target PU spMvLX [xi] [yi] (xi = xPb). + nSbW * i, yj = yPb + nSbH * j, i = 0, 1, 2, ..., nPbW / nSbW -1, j = 0, 1, 2, ..., nPbH / nSbH -1) Derived (time subblock prediction). Here, (xPb, yPb) is the upper left coordinate of the target PU, nPbW, nPbH is the size of the target PU, and nSbW, nSbH is the size of the subblock.

In addition, by calculating the weighted average according to the distance between the motion vector of the PU adjacent to the target PU and the subblock obtained by dividing the target PU, the motion vector spMvLX [ xi] [yi] (xi = xPb + nSbW * i, yj = yPb + nSbH * j, i = 0, 1, 2, ・ ・ ・, nPbW / nSbW ―― 1, j = 0, 1, 2, ・ ・·, NPbH / nSbH-1) may be derived (spatial subblock prediction).

The above-mentioned candidate TSUB for time subblock prediction and candidate SSUB for spatial subblock prediction are selected as one mode (merge candidate) of the merge mode.

(Affine prediction department)
The affine prediction unit 30372 derives the affine prediction parameters of the target PU. In this embodiment, motion vectors (mv0_x, mv0_y) (mv1_x, mv1_y) of two control points (V0, V1) of the target PU are derived as affine prediction parameters. Specifically, the motion vector of each control point may be derived by predicting from the motion vector of the adjacent PU of the target PU, and further, the prediction vector and the coded data derived as the motion vector of the control point may be derived. The motion vector of each control point may be derived from the sum of the difference vectors derived from.

FIG. 13 shows an example of deriving the motion vector spMvLX of each subblock constituting the target PU (nPbW × nPbH) from the motion vector (mv0_x, mv0_y) of the control point V0 and the motion vector (mv1_x, mv1_y) of V1. It is a figure. As shown in FIG. 13, the motion vector spMvLX of each subblock is derived as a motion vector for each point located at the center of each subblock.

The affine prediction unit 30372 uses the motion vector spMvLX [xi] [yi] (xi = xPb + nSbW * i, yj = yPb + nSbH * j, i) of each subblock in the target PU based on the affine prediction parameters of the target PU. = 0, 1, 2, ···, nPbW / nSbW − 1, j = 0, 1, 2, ···, nPbH / nSbH − 1) is derived using the following equation.

spMvLX [xi] [yi] [0] = mv0_x + (mv1_x --mv0_x) / nPbW * (xi + nSbW / 2)-(mv1_y --mv0_y) / nPbH * (yi + nSbH / 2)
spMvLX [xi] [yi] [1] = mv0_y + (mv1_y --mv0_y) / nPbW * (xi + nSbW / 2) + (mv1_x --mv0_x) / nPbH * (yi + nSbH / 2)
Here, xPb and yPb are the upper left coordinates of the target PU, nPbW and nPbH are the width and height of the target PU, and nSbW and nSbH are the width and height of the subblock.

(Matching prediction unit 30373)
The matching prediction unit 30373 derives the motion vector spMvLX of the subblock constituting the PU by performing the matching process of any of a plurality of matching methods (bilateral matching or template matching). FIG. 14 is a diagram for explaining (a) bilateral matching and (b) template matching. The matching prediction mode is selected as one merge candidate (matching candidate) in the merge mode.

The matching prediction unit 30373 derives a motion vector by matching regions in a plurality of reference images, assuming that the object moves at a constant velocity. In bilateral matching, matching between reference images A and B is performed on the assumption that an object passes through a certain area of the reference image A, a target PU of the target picture Cur_Pic, and a certain area of the reference image B in a constant velocity motion. To derive the motion vector of the target PU. In template matching, assuming that the motion vector of the target PU is equal to the motion vector of the target PU, the motion vector is derived by matching the motion vector of the target PU adjacent region Temp_Cur and the reference block adjacent region Temp_L0 on the reference picture. In the matching prediction unit, the target PU is divided into multiple sub-blocks, and the motion vector spMvLX [xi] [yi] (xi =) of the sub-blocks is performed by performing bilateral matching or template matching described later in each divided sub-block. xPb + nSbW * i, yj = yPb + nSbH * j, i = 0, 1, 2, ・ ・ ・, nPbW / nSbW -1 、 j = 0, 1, 2, ・ ・ ・, nPbH / nSbH ―― 1) Is derived.

As shown in FIG. 14A, in bilateral matching, two reference images are referred to in order to derive the motion vector of the subblock Cur_block in the target picture Cur_Pic. More specifically, first, when the coordinates of the subblock Cur_block are expressed as (xCur, yCur), it is a region in the reference image (referred to as reference picture A) specified by the reference picture index Ref0.
(XPos0, yPos0) = (xCur + MV0_x, yCur + MV0_y)
Block_A with the upper left coordinates (xPos0, yPos0) specified by and the area within the reference image (referred to as reference picture B) specified by the reference picture index Ref1.
(XPos1, yPos1) = (xCur + MV1_x, xCur + MV1_y) = (xCur --MV0_x * TD1 / TD0, yCur --MV0_y * TD1 / TD0)
Block_B with the upper left coordinates (xPos1, yPos1) specified by is set. Here, TD0 and TD1 represent the distance between the target picture Cur_Pic and the reference picture A, and the distance between the target picture Cur_Pic and the reference picture B, respectively, as shown in FIG. 14 (a). ing.

Next, (MV0_x, MV0_y) is determined so that the matching cost between Block_A and Block_B is minimized. The motion vector derived in this way (MV0_x, MV0_y) is a motion vector given to the subblock.

On the other hand, FIG. 14B is a diagram for explaining template matching among the above matching processes.

As shown in FIG. 14 (b), in template matching, one reference picture is referred to in order to derive the motion vector of the subblock Cur_block in the target picture Cur_Pic.

More specifically, first, it is a region in the reference image (referred to as reference picture A) specified by the reference picture index Ref0.
(XPos, yPos) = (xCur + MV0_x, yCur + MV0_y)
The reference block Block_A with the upper left coordinates (xPos, yPos) identified by is identified. Here, (xCur, yCur) is the upper left coordinate of the subblock Cur_block.

Next, the template area Temp_Cur adjacent to the subblock Cur_block in the target picture Cur_Pic and the template area Temp_L0 adjacent to Block_A in the reference picture A are set. In the example shown in FIG. 14 (b), the template region Temp_Cur is composed of a region adjacent to the upper side of the subblock Cur_block and a region adjacent to the left side of the subblock Cur_block. Further, the template area Temp_L0 is composed of an area adjacent to the upper side of Block_A and an area adjacent to the left side of Block_A.

Next, it is determined that the matching cost between Temp_Cur and TempL0 is minimized (MV0_x, MV0_y), and it becomes the motion vector spMvLX given to the subblock.

FIG. 7 is a schematic diagram showing the configuration of the merge prediction parameter derivation unit 3036 according to the present embodiment. The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361, a merge candidate selection unit 30632, and a merge candidate storage unit 30363. The merge candidate storage unit 30363 stores the merge candidates input from the merge candidate derivation unit 30361. The merge candidate includes the prediction list usage flag predFlagLX, the motion vector mvLX, and the reference picture index refIdxLX. In the merge candidate storage unit 30363, an index is assigned to the stored merge candidates according to a predetermined rule.

The merge candidate derivation unit 30361 derives the merge candidate by using the motion vector of the adjacent PU that has already been decoded and the reference picture index refIdxLX as they are. Alternatively, affine prediction may be used to derive merge candidates. This method will be described in detail below. The merge candidate derivation unit 30361 may use the affine prediction for the spatial merge candidate derivation process, the time merge candidate derivation process, the join merge candidate derivation process, and the zero merge candidate derivation process, which will be described later. The affine prediction is performed in sub-block units, and the prediction parameters are stored in the prediction parameter memory 307 for each sub-block. Alternatively, the affine prediction may be performed on a pixel-by-pixel basis.

(Spatial merge candidate derivation process)
As the spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads and reads the prediction parameters (prediction list usage flag predFlagLX, motion vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule. The predicted parameters are derived as merge candidates. The prediction parameter to be read is a prediction parameter related to each of the PUs within a predetermined range from the decoding target PU (for example, all or a part of the PUs in contact with the left lower end, the upper left end, and the upper right end of the decoding target PU). be. The merge candidate derived by the merge candidate derivation unit 30361 is stored in the merge candidate storage unit 30363.

(Time merge candidate derivation process)
As the time merge derivation process, the merge candidate derivation unit 30361 reads the prediction parameter of the PU in the reference image including the lower right coordinate of the decoding target PU from the prediction parameter memory 307 and sets it as a merge candidate. The reference image may be specified by, for example, the reference picture index refIdxLX specified in the slice header, or the reference picture index refIdxLX of the PU adjacent to the decoding target PU may be used. The merge candidate derived by the merge candidate derivation unit 30361 is stored in the merge candidate storage unit 30363.

(Join merge candidate derivation process)
As a merge merge derivation process, the merge candidate derivation unit 30361 uses the motion vectors and reference picture indexes of two different derived merge candidates that have already been derived and stored in the merge candidate storage unit 30363 as motion vectors of L0 and L1, respectively. Derivation of join merge candidates by combining. The merge candidate derived by the merge candidate derivation unit 30361 is stored in the merge candidate storage unit 30363.

(Zero merge candidate derivation process)
As the zero merge candidate derivation process, the merge candidate derivation unit 30361 derives a merge candidate in which the reference picture index refIdxLX is 0 and the X component and the Y component of the motion vector mvLX are both 0. The merge candidate derived by the merge candidate derivation unit 30361 is stored in the merge candidate storage unit 30363.

Among the merge candidates stored in the merge candidate storage unit 30363, the merge candidate selection unit 30632 selects the merge candidate to which the index corresponding to the merge index merge_idx input from the inter-prediction parameter decoding control unit 3031 is assigned to the target PU. Select as an inter-prediction parameter for. The merge candidate selection unit 30362 stores the selected merge candidate in the prediction parameter memory 307 and outputs the selected merge candidate to the prediction image generation unit 308.

FIG. 8 is a schematic diagram showing the configuration of the AMVP prediction parameter derivation unit 3032 according to the present embodiment. The AMVP prediction parameter derivation unit 3032 includes a vector candidate derivation unit (prediction vector candidate derivation unit) 3033, a vector candidate selection unit 3034, and a vector candidate storage unit 3035. The vector candidate derivation unit 3033 reads the motion vector mvLX of the already processed PU stored in the prediction parameter memory 307 based on the reference picture index refIdx, derives the prediction vector candidate, and makes a prediction vector for the vector candidate storage unit 3035. Store in the candidate list mvpListLX [].

The vector candidate selection unit 3034 selects the motion vector mvpListLX [mvp_LX_idx] indicated by the prediction vector index mvp_LX_idx among the prediction vector candidates of the prediction vector candidate list mvpListLX [] as the prediction vector mvpLX. The vector candidate selection unit 3034 outputs the selected prediction vector mvpLX to the addition unit 3035.

The prediction vector candidate is a PU for which decoding processing has been completed, and is derived by scaling the motion vector of a PU (for example, an adjacent PU) in a predetermined range from the decoding target PU. The adjacent PU includes a PU spatially adjacent to the decoding target PU, for example, the left PU, the upper PU, and a region temporally adjacent to the decoding target PU, for example, the same position as the decoding target PU, and is displayed. Includes regions obtained from predictive parameters of PUs at different times.

The addition unit 3035 calculates the motion vector mvLX by adding the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 3032 and the difference vector mvdLX input from the inter-prediction parameter decoding control unit 3031. The addition unit 3035 outputs the calculated motion vector mvLX to the prediction image generation unit 308 and the prediction parameter memory 307.

(Inter prediction image generation unit 309)
FIG. 11 is a schematic diagram showing the configuration of the inter-predictive image generation unit 309 included in the predictive image generation unit 308 according to the present embodiment. The inter-prediction image generation unit 309 includes a motion compensation unit (prediction image generation device) 3091 and a weight prediction unit 3094.

(Motion compensation)
The motion compensation unit 3091 has a reference picture index refIdxLX from the reference picture memory 306 based on the inter-prediction parameters (prediction list utilization flag predFlagLX, reference picture index refIdxLX, motion vector mvLX) input from the inter-prediction parameter decoding unit 303. In the reference picture specified in, an interpolated image (motion compensation image) is generated by reading a block at a position shifted by the motion vector mvLX from the position of the PU to be decoded. Here, when the accuracy of the motion vector mvLX is not an integer accuracy, a filter called a motion compensation filter for generating pixels at decimal positions is applied to generate a motion compensation image.

(Weight prediction)
The weight prediction unit 3094 generates a prediction image of the PU by multiplying the input motion compensation image predSamplesLX by a weight coefficient. When one of the prediction list usage flags (predFlagL0 or predFlagL1) is 1 (in the case of simple prediction), and when weight prediction is not used, the input motion compensation image predSamplesLX (LX is L0 or L1) is the number of pixel bits bitDepth. Perform the following formula processing to match.

predSamples [X] [Y] = Clip3 (0, (1 << bitDepth) --1, (predSamplesLX [X] [Y] + offset1) >> shift1)
Here, shift1 = 14 --bitDepth, offset1 = 1 << (shift1-1). If both of the reference list usage flags (predFlagL0 and predFlagL1) are 1 (in the case of biprediction BiPred) and weight prediction is not used, the input motion compensation images predSamplesL0 and predSamplesL1 are averaged and the number of pixel bits. Perform the following formula processing to match.

predSamples [X] [Y] = Clip3 (0, (1 << bitDepth) --1, (predSamplesL0 [X] [Y] + predSamplesL1 [X] [Y] + offset2) >> shift2)
Here, shift2 = 15-bitDepth, offset2 = 1 << (shift2-1).

Further, in the case of simple prediction, when weight prediction is performed, the weight prediction unit 3094 derives the weight prediction coefficient w0 and the offset o0 from the coded data, and performs the processing of the following equation.

predSamples [X] [Y] = Clip3 (0, (1 << bitDepth) ―― 1, ((predSamplesLX [X] [Y] * w0 + 2 ^ (log2WD ―― 1)) >> log2WD) + o0)
Here, log2WD is a variable indicating a predetermined shift amount.

Further, in the case of the biprediction BiPred, when the weight prediction is performed, the weight prediction unit 3094 derives the weight prediction coefficients w0, w1, o0, and o1 from the coded data, and performs the processing of the following equation.

predSamples [X] [Y] = Clip3 (0, (1 << bitDepth) --1, (predSamplesL0 [X] [Y] * w0 + predSamplesL1 [X] [Y] * w1 + ((o0 + o1 + 1)) << log2WD)) >> (log2WD + 1))
<Motion vector decoding process>
Hereinafter, the motion vector decoding process according to the present embodiment will be specifically described with reference to FIG. 9.

As is clear from the above description, the motion vector decoding process according to the present embodiment is a process of decoding a syntax element related to inter-prediction (also referred to as a motion syntax decoding process) and a process of deriving a motion vector (also called a motion vector decoding process). Motion vector derivation process) and included.

(Motion syntax decoding process)
FIG. 9 is a flowchart showing the flow of the inter-prediction syntax decoding process performed by the inter-prediction parameter decoding control unit 3031. In the following description in the description of FIG. 9, each process is performed by the inter-prediction parameter decoding control unit 3031 unless otherwise specified.

First, in step S101, the merge flag merge_flag is decoded, and in step S102,
merge_flag! = 0 (whether merge_flag is 0)
Is judged.

When merge_flag! = 0 is true (Y in S102), the merge index merge_idx is decoded in S103, and the motion vector derivation process (S111) in the merge mode is executed.

If merge_flag! = 0 is false (N in S102), the inter-prediction identifier inter_pred_idc is decoded in S104.

When inter_pred_idc is other than PRED_L1 (PRED_L0 or PRED_BI), the reference picture index refIdxL0, the difference vector parameter mvdL0, and the prediction vector index mvp_L0_idx are decoded in S105, S106, and S107, respectively.

When inter_pred_idc is other than PRED_L0 (PRED_L1 or PRED_BI), the reference picture index refIdxL1, the difference vector parameter mvdL1, and the prediction vector index mvp_L1_idx are decoded in S108, S109, and S110. Subsequently, the motion vector derivation process (S112) in the AMVP mode is executed.

(Another expression of motion vector derivation process)
The above motion vector derivation process can be expressed as follows. FIG. 15 is a flowchart showing an outline of the motion prediction mode determination flow. The motion prediction mode determination flow is executed by the inter prediction parameter decoding unit 303. The motion prediction mode is a mode for determining a method for deriving a motion vector used for motion compensation prediction.

As shown in FIG. 15, in the motion prediction mode determination flow, first, the inter-prediction parameter decoding control unit 3031 determines whether or not the merge mode is used (S1501), and if it is not the merge mode (NO in S1501), the AMVP mode. Will be. On the other hand, when it is determined that the mode is the merge mode (YES in S1501), it is determined whether or not the mode is the matching mode (S1502). Then, when it is determined that the mode is the matching mode (YES in S1502), the matching mode is set, and when it is determined that the mode is not the matching mode (NO in S1502), the merge mode is set. The matching mode is also called FRUC (Frame Rate Up Conversion) merge mode, and performs the above-mentioned template matching or bilateral matching.

Next, the details of the motion prediction mode determination flow will be described with reference to FIG. FIG. 16 is a sequence diagram showing the flow of the motion prediction mode determination flow.

First, the inter-prediction parameter decoding control unit 3031 decodes the merge flag merge_flag in step S101, and determines merge_flag == 1? In step S102.

When merge_flag == 1 is true (YES in S102), the parameter fruc_merge_idx indicating the matching mode is decoded in step S103, and fruc_merge_idx! = 0 is determined in step S104.

When fruc_merge_idx! = 0 is true (YES in S104), the matching mode is selected as the motion vector derivation method. In step S105, the matching prediction unit 30373 derives a pattern match vector by bilateral matching when fruc_merge_idx is 1, and derives a pattern match vector by template matching when fruc_merge_idx is 2.

In the above, the parameter fruc_merge_idx indicating the matching mode serves as a flag indicating whether or not to use the matching mode and a parameter indicating the matching method in the matching mode, but the present invention is not limited thereto. That is, instead of the parameter fruc_merge_idx indicating the matching mode, the flag fruc_merge_flag indicating whether or not to use the matching mode and the parameter fruc_merge_param indicating the matching method may be used. In this case, the determination of fruc_merge_idx! = 0 is equivalent to fruc_merge_flag! = 0, and the determination of fruc_merge_idx == 1 is equivalent to fruc_merge_param! = 0. Note that fruc_merge_param is decoded when fruc_merge_flag is 1.

If fruc_merge_idx! = 0 is false in step S104, the merge prediction parameter derivation unit 3036 decodes the merge index merge_idx in S111. Subsequently, in S112, the merge candidate mergeCand is derived, and in S113, the motion vector mvLX is derived by the following equation.

mvLX = mergeCand [merge_idx]
On the other hand, if merge_flag == 1 is false (NO in S102) in step S102, the AMVP mode is selected. More specifically, the AMVP prediction parameter derivation unit 3032 decodes the difference vector mvdLX in S121 and decodes the prediction vector index mvp_LX_idx in S122. Further, in S123, the prediction vector candidate pmvCand is derived. Subsequently, in S124, the motion vector mvLX is derived by the following equation.

mvLX = pmvCand [mvp_LX_idx] + mvdLX
(Motion vector derivation processing by matching processing)
Hereinafter, the flow of the motion vector derivation (pattern matching vector derivation) process in the matching mode will be described with reference to FIGS. 17 and 18. FIG. 17 is a flowchart showing the flow of the pattern match vector derivation process. FIG. 18 is a diagram for explaining the pattern match vector derivation process.

FIG. 17 shows details of the process in step S105 in the sequence diagram shown in FIG. The process shown in FIG. 17 is executed by the matching prediction unit 30373.

Of the steps shown in FIG. 17, steps S1051 to S1054 are block searches executed at the block level. That is, using pattern matching, a motion vector is derived for the entire block (CU or PU). Specifically, as shown in FIG. 18A, the motion vector in the entire target block is derived. In other words, the motion vector is derived for each target block.

Further, steps S1055 to S1060 are subblock searches executed at the subblock level. That is, using pattern matching, a motion vector is derived in units of sub-blocks constituting the block. In particular. As shown in FIG. 18B, a motion vector is derived for each subblock in the target block. The size of the sub-block is 1/8 in each of the vertical and horizontal directions with respect to the target block. However, the minimum size of the subblock is 4 × 4 pixels.

First, in step S1051, fruc_merge_idx == 1 is determined. When fruc_merge_idx == 1 is false (NO in S1051), in step S1052, a template for performing template matching is acquired. More specifically, the template for template matching is obtained from the peripheral area of the block. Specifically, as shown in FIG. 18C, the template is acquired from the upper adjacent area or the left adjacent area of the target block. The size (thickness) of the template is 4 pixels. Then, the process proceeds to step S1053.

Further, when fruc_merge_idx == 1 is true (YES in S1051), the process proceeds to step S1053. In step S1053, a block-level initial vector in the target block is derived (initial vector search). The initial vector is a motion vector that is the basis of the search, and from limited motion vector candidates (spatial merge candidate, time merge candidate, join merge candidate, zero vector, ATMVP vector of the target block, etc.), matching cost. Is derived from the vector that minimizes, and is used as the initial vector. The ATMVP vector is a motion vector derived from the block on the reference image in sub-block units, which is indicated by the temporal vector derived from the motion vector around the target block.

In step S1054, a block-level local search (local search) in the target block is performed. In the local search, the local region centered on the initial vector derived in step S1051 is further searched, the vector having the minimum matching cost is searched, and the motion vector of the final target block is used. The local search may be a step search or a raster search. The details of the local search will be described later.

Subsequently, the following processing is performed for each subblock included in the target block (steps S1055 to S1060).

First, in step S1056, fruc_merge_idx == 1 is determined. When fruc_merge_idx == 1 is false (NO in S1056), in step S1057, a template for performing template matching is acquired. More specifically, the template for template matching is obtained from the peripheral area of the subblock. Specifically, as shown in FIG. 18 (c), the template is acquired from the upper adjacent area or the left adjacent area of the target subblock. Then, the process proceeds to step S1058. Further, when fruc_merge_idx == 1 is true (YES in S1056), the process proceeds to step S1058.

In step S1053, the initial vector of the subblock in the target block is derived (initial vector search). Specifically, vector candidates (movement vector of the target block, zero vector, center collate vector of the subblock, lower right colocate vector of the subblock, ATMVP vector of the subblock, upper adjacent vector of the subblock, sub). Of the vectors on the left side of the block, etc.), the vector that minimizes the matching cost is used as the initial vector of the subblock. The vector candidates used for the initial vector search of the subblock are not limited to the above-mentioned vectors.

Next, in step S1059, a step search (local search) centered on the initial vector of the subblock selected in S1058 is performed. Then, the matching cost of the vector candidate near the initial vector of the subblock is derived, and the minimum vector is derived as the motion vector of the subblock.

Then, when the processing is completed for all the sub-blocks included in the target block, the pattern match vector derivation processing is completed.

When fruc_merge_idx is 1 in both the initial vector search and the local search, the matching cost is derived by bilateral matching. If fruc_merge_idx is 2, the matching cost is derived by template matching.

(Local search algorithm)
Next, the algorithm of the local search will be described with reference to FIG. FIG. 19 is a diagram for explaining a motion search pattern. The number of steps (stepIter, maximum number of rounds) indicating how many times the method used for motion search (stepMethod) is repeated is set to a predetermined value. As will be described later, the maximum number of rounds at the subblock level, stepIterSubPU, may be less than the maximum number of rounds at the block level, stepIterPU.

The matching prediction unit 30373 considers the search candidate point that gives the smallest matching cost among the search candidate points whose matching cost is evaluated in the motion search as the optimum search point, and selects the motion vector bestMV of the search candidate point. .. Examples of the function used for deriving the matching cost include SAD (Sum of Absolute Difference, absolute value error sum), STD (Hadamard transform absolute value error sum), SSD (Sum of Square difference), and the like. ..

The motion vector local search performed by the matching prediction unit 30373 is not limited to this, but is limited to diamond search (stepMethod = DIAMOND), cross search (stepMethod = CROSS), and raster search (raster type search, stepMethod = RASTER). ) And other motion search algorithms are used.

In the following, the case of PU as the target block will be described as an example.

<Step search>
First, as an example of the step search, the diamond search will be described with reference to FIGS. 19 (a) and 19 (b). 19 (a) and 19 (b) are diagrams showing a motion search pattern when the diamond search is applied. FIG. 19 shows an example of a search range of 7PU (horizontal) × 5PU (vertical). In addition, the offset candidate (offsetCand) that is added to the coordinates of the search start point to set the search candidate point is
offsetCand [8] = {(0, 2), (1, 1), (2, 0), (1, -1), (0, -2), (-1, -1), (-2, 0), (-1, 1)}
The case of a diamond search such as eight represented by is illustrated. The matching prediction unit 30373 selects the coordinates of eight search candidate points by adding each value (offsetCand [Idx]) of the offset candidate (offsetCand) to the coordinate (position) startMV of the search start point. Each search candidate point selected in this way corresponds to eight directions from the search start direction nDirectStart = 0 to the search end direction nDirectEnd = 7. As the number of offset candidates, 8 is usually used in the diamond search, but other values, for example, any value larger than 8 or any value smaller than 8 may be used. However, as the number of offset candidates increases, the time required for motion search processing and the number of operations increase, so it is desirable to select an appropriate value.

In FIG. 19, a white diamond indicates the starting point of the initial vector startMV at each search count, a black diamond indicates the end point of the optimum vector bestMV in each search round, and a black circle indicates a search candidate point at each search count. It is shown and the white circles indicate the points that have been searched for each number of searches.

When the matching prediction unit 30373 performs a motion search to which a step search is applied, the search round numIter is initialized to 0 before the search is started. Then, at the start of each search round, the minimum cost minCost is set as the matching cost of the search start point, and the initial value (-1) is set in the best candidate index bestIdx.

minCost = mcost (startMV)
bestIdx = -1
Here, mcost (X) is a function for deriving the matching cost in the search vector X.

The matching prediction unit 30373 selects and evaluates a search candidate point centered on the search start point (here, P0) in each search round (here, the 0th search, numIter = 0). Here, eight points arranged in a diamond shape are selected as search candidate points (here, points 0 to 7 in the first stage of FIG. 19A). That is, at the start of the search, the matching prediction unit 30373 sets a search vector (initial vector startMV = P0) having the search start point P0 as the start point and each search candidate point as the end point.

The matching prediction unit 30373 evaluates the matching cost for the search candidate points 0 to 7 shown in the first stage of FIG. 19A. For Idx = nDirectStart..nDirectEnd (here, nDirectStart = 0, nDirectEnd = 7), motion vector candidate candMV is derived in order by the following formula, and the matching cost in each candMV is evaluated.

candMV = startMV + offsetCand [Idx]
For example, when the search candidate point (including the search start point P0) that gives the lowest matching cost is the point 2 shown in the first stage of FIG. 19 (a), that is, the search candidate point candMV [Idx] with Idx = 2. If the matching cost candCost (candCost = mcost (candMV [Idx])) of is less than the minimum cost minCost (candCost <minCost), the matching predictor 30373 updates the best-first search candidate index bestIdx to Idx to optimize it. Update cost minCost, optimal vector bestMV. This can be expressed as:

bestIdx = Idx
minCost = candCost
bestMV = candMV [Idx]
In each round, search candidate points are set around the search start point, the matching cost for the set search candidate points is derived and evaluated, and the search candidate point that is the optimum matching cost is selected. The processing is referred to as "step round processing" here. In the step search, this "step round process" is repeatedly executed. In each step round process, the number of search rounds numIter is incremented by 1.

If the optimal vector bestMV is updated in this process at the end of each step round process and at the end of evaluation of all search candidate points (here, bestIdx> = 0), and the number of search rounds numIter When is less than the predetermined maximum number of rounds stepIter (here, numIter <stepIter), the matching prediction unit 30373 performs the next step round processing. The search candidate point selected at this time is used as the search start point of the next round. That is, in the next step round process, the matching prediction unit 30373 selects the point 2 shown in the first stage of FIG. 19A as the end point of the optimum vector bestMV for the prediction block PU.

startMV = bestMV (here P (1))
Whether or not the optimal vector bestMV has been updated is determined by whether or not the optimal vector bestMV is different from the search start point, and whether or not the bestIdx has been updated to a value other than the initial value (-1). , MinCost can also be determined by whether or not it has been updated to a value other than the initial cost of the starting point. If the search start index nDirectStart and search end index nDirectEnd to be used in the next round are determined by the following formula depending on the position of the optimal vector bestMV (optimal candidate index Idx), the already searched search points can be obtained. Efficient search is possible without searching again.

nStep = 2-(bestIdx & 1)
nDirectStart = bestIdx --nStep
nDirectEnd = bestIdx --nStep
Next, as shown in the second stage of FIG. 19 (a), in the first search (numIter = 1), the matching prediction unit 30373 is the end point of the optimum vector bestMV in the first stage of FIG. 19 (a). The point 2 selected as is the starting point of the initial vector startMV (search start point P1) in this search, and is a plurality of points arranged in a diamond shape around the search start point P1 and is still selected as a search candidate point. The non-existing points are selected in order as search candidate points (points 0 to 4 in the second stage of FIG. 19A), and the matching cost is evaluated. That is, the search candidate points indicated by Idx = nDirectStart..nDirectEnd (here, nDirectStart = 0, nDirectEnd = 4) are evaluated. For example, when the search candidate point (including the search start point P1) that gives the lowest matching cost is the point 1 shown in the second stage of FIG. 19 (a), that is, the search candidate point candMV [Idx] with Idx = 1. ] Matching cost candCost (candCost = mcost (candMV [Idx])) is less than the minimum cost minCost (candCost <minCost), the matching predictor 30373 updates the best vector bestMV as in the previous round. ..

Since the optimum vector bestMV was updated in this processing, the matching prediction unit 30373 executes the next step round processing. Again, as the starting point of the optimal vector bestMV for the prediction block PU, the point 1 shown in the second stage of FIG. 19A, which is the optimal search candidate point of this round, is selected.

Subsequently, as shown in the third stage of FIG. 19 (a), in the second search (numIter = 2), the matching prediction unit 30373 determines the point 1 shown in the second stage of FIG. 19 (a). A plurality of points arranged in a diamond shape around the search start point P2 as the starting point of the initial vector startMV (search start point P2) in this search, and are not yet selected as search candidate points. A point existing in the range is selected as a search candidate point (points 0 to 2 in the third stage of FIG. 19A) (that is, nDirectStart = 0, nDirectEnd = 2).

When the search candidate point giving the smallest matching cost in the third stage of FIG. 19 (a) is equal to or greater than the cost of the search start point P2 shown in the third stage of FIG. 19 (a), the optimum vector bestMV is updated. do not. At this point, the general step search process (diamond search) is completed.

The matching prediction unit 30373 may newly perform another step search. In FIG. 19B, the matching prediction unit 30373 shows an example in which a cross search (stepMethod = CROSS) is performed once in order to search around the search start point P2 in more detail in the next step round process. ing.

In the cross search, the following values are used as offset candidates (offsetCand).

offsetCand [4] = {(0, 1), (1, 0), (0, -1), (-1, 0)
In the cross search, the matching prediction unit 30373 is a point at a position of up, down, left, right (cross) about the search start point (the search start point P2 of the third stage in FIG. 19A), and is the above-mentioned diamond. In the search, points that are not selected as search candidate points are selected as search candidate points. For example, the search candidate points (including the search start point P2) that give the lowest matching cost among the search candidate points 0 to 3 on the top, bottom, left, and right of the search start point P2 are the points 1 shown in FIG. 19 (b). If so, the matching prediction unit 30373 selects point 1 shown in FIG. 19 (b) as the end point of the optimum vector bestMV for the prediction block PU.

In this way, the matching prediction unit 30373 selects a search candidate point that gives the lowest matching cost by using a certain step search (for example, diamond search) one or more times, and then performs a certain step search (for example, cross search). ) May be used once or multiple times to select the motion vector in more detail.

The motion search pattern in the example shown in FIG. 19A is an example of motion search using a diamond search, and the matching prediction unit 30373 may use a diamond search having another motion search pattern. For example, in the 0th search, a point that exists equidistant from the search start point is selected as a search candidate point, and the nth search is performed to search for a point whose distance from the search start point is 2n-1. Diamond search may be applied.

<Raster search>
Subsequently, the raster search will be described. When the matching prediction unit 30373 performs a motion search to which a raster search is applied, the matching prediction unit 30373 comprehensively selects search points within the search range at regular intervals and selects these matching costs in the order of raster scan. evaluate. Here, raster scan starts from the upper left of the search range, examines the pixels from the left side to the right until it reaches the right end, and when it reaches the right end, it goes down one line and goes from the left end to the right again. It is a comprehensive search method that examines pixels in order.

The matching prediction unit 30373 selects the search vector that gives the smallest matching cost among the matching costs calculated for each of the search vectors having the end points set in the raster scan order.

Raster scan is a block with a size of blkW x blkH, where Y coordinate y and X coordinate x are set to the initial values, then x is scanned from the initial value to the closing price, and when x reaches the closing price. , X is returned to the initial value, y is increased, and x is scanned again from the initial value to the closing price in the updated y. In pseudo code, the loop of x is done in the following double loop such that the loop of y is inside the loop of y.

for (y = 0; y <blkH; y ++) {// Loop for y
for (x = 0; x <blkW; x ++) {// Processing in loop raster scan for x
}
}
An extended raster scan may be used instead of the raster scan. The extended raster scan scans each point in the block in a predetermined scan order like a raster scan. For example, a spiral scan that scans spirally from the center to the periphery.

(Configuration of image coding device)
Next, the configuration of the image coding device 11 according to the present embodiment will be described. FIG. 4 is a block diagram showing the configuration of the image coding device 11 according to the present embodiment. The image coding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, an entropy coding unit 104, an inverse quantization / inverse DCT unit 105, an addition unit 106, a loop filter 107, and a prediction parameter memory. (Prediction parameter storage unit, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, and prediction parameter coding unit 111 are included. The prediction parameter coding unit 111 includes an inter prediction parameter coding unit (motion vector generator) 112 and an intra prediction parameter coding unit 113.

For each picture of the image T, the prediction image generation unit 101 generates a prediction image P of the prediction unit PU for each coding unit CU which is a region in which the picture is divided. Here, the prediction image generation unit 101 reads out the decoded block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter coding unit 111. The prediction parameter input from the prediction parameter coding unit 111 is, for example, a motion vector in the case of inter-prediction. The prediction image generation unit 101 reads out the block at the position on the reference image indicated by the motion vector starting from the target PU. In the case of intra prediction, the prediction parameter is, for example, an intra prediction mode. The pixel value of the adjacent PU used in the intra prediction mode is read from the reference picture memory 109, and the predicted image P of the PU is generated. The prediction image generation unit 101 generates a prediction image P of the PU for the read reference picture block by using one of the plurality of prediction methods. The predicted image generation unit 101 outputs the generated predicted image P of the PU to the subtraction unit 102.

The predicted image generation unit 101 has the same operation as the predicted image generation unit 308 described above. For example, FIG. 6 is a schematic diagram showing the configuration of the inter-predictive image generation unit 1011 included in the predictive image generation unit 101. The inter-prediction image generation unit 1011 includes a motion compensation unit 10111 and a weight prediction unit 10112. Since the motion compensation unit 10111 and the weight prediction unit 10112 have the same configurations as the motion compensation unit 3091 and the weight prediction unit 3094 described above, the description thereof will be omitted here.

The prediction image generation unit 101 uses the parameters input from the prediction parameter coding unit to generate the prediction image P of the PU based on the pixel value of the reference block read from the reference picture memory. The predicted image generated by the predicted image generation unit 101 is output to the subtraction unit 102 and the addition unit 106.

The subtraction unit 102 generates a residual signal by subtracting the signal value of the predicted image P of the PU input from the predicted image generation unit 101 from the pixel value of the corresponding PU of the image T. The subtraction unit 102 outputs the generated residual signal to the DCT / quantization unit 103.

The DCT / quantization unit 103 performs DCT on the residual signal input from the subtraction unit 102, and calculates the DCT coefficient. The DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain the quantization coefficient. The DCT / quantization unit 103 outputs the obtained quantization coefficient to the entropy coding unit 104 and the inverse quantization / inverse DCT unit 105.

A quantization coefficient is input to the entropy coding unit 104 from the DCT / quantization unit 103, and a coding parameter is input from the prediction parameter coding unit 111. The coded parameters to be input include, for example, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, and a merge index merge_idx.

The entropy coding unit 104 entropy-codes the input quantization coefficient and coding parameter to generate a coded stream Te, and outputs the generated coded stream Te to the outside.

The dequantization / inverse DCT unit 105 dequantizes the quantization coefficient input from the DCT / quantization unit 103 to obtain the DCT coefficient. The inverse quantization / inverse DCT unit 105 performs an inverse DCT on the obtained DCT coefficient and calculates a residual signal. The inverse quantization / inverse DCT unit 105 outputs the calculated residual signal to the addition unit 106.

The addition unit 106 adds the signal value of the predicted image P of the PU input from the prediction image generation unit 101 and the signal value of the residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and decodes the signal. Generate an image. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image generated by the addition unit 106.

The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 at positions predetermined for each of the picture to be coded and the CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a position predetermined for each of the picture to be encoded and the CU.

The coding parameter determination unit 110 selects one set from the plurality of sets of coding parameters. The coding parameter is the above-mentioned prediction parameter or a parameter to be coded generated in connection with the prediction parameter. The prediction image generation unit 101 generates a prediction image P of the PU using each of these sets of coding parameters.

The coding parameter determination unit 110 calculates a cost value indicating the magnitude of the amount of information and the coding error for each of the plurality of sets. The cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy-coding the quantization error and the coding parameter. The squared error is the sum of the squared values of the residual values of the residual signal calculated by the subtraction unit 102 between the pixels. The coefficient λ is a real number greater than the preset zero. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated cost value. As a result, the entropy coding unit 104 outputs the selected set of coding parameters to the outside as a coding stream Te, and does not output the set of unselected coding parameters. The coding parameter determination unit 110 stores the determined coding parameter in the prediction parameter memory 108.

The prediction parameter coding unit 111 derives a format for coding from the parameters input from the coding parameter determination unit 110 and outputs the encoding format to the entropy coding unit 104. Derivation of the form for encoding is, for example, deriving a difference vector from a motion vector and a prediction vector. Further, the prediction parameter coding unit 111 derives the parameters necessary for generating the prediction image from the parameters input from the coding parameter determination unit 110, and outputs the parameters to the prediction image generation unit 101. The parameters required to generate the predicted image are, for example, motion vectors in sub-block units.

The inter-prediction parameter coding unit 112 derives an inter-prediction parameter such as a difference vector based on the prediction parameter input from the coding parameter determination unit 110. In the inter-prediction parameter coding unit 112, the inter-prediction parameter decoding unit 303 (see FIG. 5, etc.) derives the inter-prediction parameter as a configuration for deriving the parameters necessary for generating the prediction image to be output to the prediction image generation unit 101. Includes a configuration that is partially the same as the configuration to be used. The configuration of the inter-prediction parameter coding unit 112 will be described later.

The intra prediction parameter coding unit 113 derives a coding format (for example, MPM_idx, rem_intra_luma_pred_mode, etc.) from the intra prediction mode IntraPredMode input from the coding parameter determination unit 110.

(Structure of inter-prediction parameter coding unit)
Next, the configuration of the inter-prediction parameter coding unit 112 will be described. The inter-prediction parameter coding unit 112 is a means corresponding to the inter-prediction parameter decoding unit 303 of FIG. 12, and the configuration is shown in FIG.

The inter-prediction parameter coding unit 112 includes an inter-prediction parameter coding control unit 1121, an AMVP prediction parameter derivation unit 1122, a subtraction unit 1123, a subblock prediction parameter derivation unit 1125, and a division mode derivation unit and a merge flag derivation unit (not shown). , Inter-prediction identifier derivation unit, reference picture index derivation unit, vector difference derivation unit, etc. are included. Divided mode derivation part, merge flag derivation part, inter prediction identifier derivation part, reference picture index derivation part, vector difference derivation part are PU division mode part_mode, merge flag merge_flag, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, difference vector, respectively. Derive mvdLX. The inter-prediction parameter coding unit 112 outputs a motion vector (mvLX, subMvLX), a reference picture index refIdxLX, a PU division mode part_mode, an inter-prediction identifier inter_pred_idc, or information indicating these to the prediction image generation unit 101. The inter prediction parameter coding unit 112 entropy the PU split mode part_mode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, difference vector mvdLX, and subblock prediction mode flag subPbMotionFlag. It is output to the coding unit 104.

The inter-prediction parameter coding control unit 1121 includes a merge index derivation unit 11211 and a vector candidate index derivation unit 11212. The merge index derivation unit 11211 compares the motion vector and the reference picture index input from the coding parameter determination unit 110 with the motion vector and the reference picture index of the merge candidate PU read from the prediction parameter memory 108, and merges them. The index merge_idx is derived and output to the entropy coding unit 104. The merge candidate is a reference PU within a predetermined range from the coding target CU to be encoded (for example, a reference PU in contact with the left lower end, the upper left end, and the upper right end of the coding target block), and is encoded. It is a PU that has been processed. The vector candidate index derivation unit 11212 derives the prediction vector index mvp_LX_idx.

In the subblock prediction parameter derivation unit 1125, when the coding parameter determination unit 110 decides to use the subblock prediction mode, any of spatial subblock prediction, time subblock prediction, affine prediction, and matching prediction is performed according to the value of subPbMotionFlag. Derivation of the motion vector and reference picture index of the subblock prediction. As described in the description of the image decoding device, the motion vector and the reference picture index read out the motion vector of the adjacent PU, the reference picture block, and the reference picture index from the prediction parameter memory 108 and derive them.

The AMVP prediction parameter derivation unit 1122 has the same configuration as the AMVP prediction parameter derivation unit 3032 (see FIG. 12) described above.

That is, when the prediction mode predMode indicates the inter prediction mode, the motion vector mvLX is input from the coding parameter determination unit 110 to the AMVP prediction parameter derivation unit 1122. The AMVP prediction parameter derivation unit 1122 derives the prediction vector mvpLX based on the input motion vector mvLX. The AMVP prediction parameter derivation unit 1122 outputs the derived prediction vector mvpLX to the subtraction unit 1123. The reference picture index refIdx and the prediction vector index mvp_LX_idx are output to the entropy coding unit 104.

The subtraction unit 1123 generates a difference vector mvdLX by subtracting the prediction vector mvpLX input from the AMVP prediction parameter derivation unit 1122 from the motion vector mvLX input from the coding parameter determination unit 110. The difference vector mvdLX is output to the entropy coding unit 104.

In addition, a part of the image coding device 11 and the image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, the inverse quantization / inverse DCT. Unit 311, addition unit 312, prediction image generation unit 101, subtraction unit 102, DCT / quantization unit 103, entropy coding unit 104, dequantization / inverse DCT unit 105, loop filter 107, coding parameter determination unit 110, The prediction parameter coding unit 111 may be realized by a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The "computer system" referred to here is a computer system built in either the image coding device 11 or the image decoding device 31, and includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In that case, a program may be held for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

[Embodiment 2]
As described above, in the first embodiment, the motion vector is derived in sub-block units in the pattern match vector derivation process. Further, template matching or bilateral matching is used for each subblock (see steps S1055 to S1060 in FIG. 17). However, since the subblock that does not touch the upper end or the left end of the target block cannot acquire the template, template matching cannot be performed. Specifically, it will be described with reference to FIG. FIG. 20 is a diagram showing an adverse effect when template matching is performed in a subblock. As shown in FIG. 20, among the subblocks included in the target block, the subblocks that are not in contact with the upper end or the left end of the target block are not in contact with the upper adjacent region or the left adjacent region of the target block. I can't get the template from. Therefore, in these sub-blocks, the block-level motion vector is used as it is, and the coding efficiency is lowered.

Therefore, in the present embodiment, the coding efficiency is prevented from being lowered by performing the following processing on the subblock for which the template cannot be acquired. As described above, the pattern matching derivation process is executed by the matching prediction unit 30373.

(Embodiment 2-1)
In the present embodiment, the matching prediction unit 30373 performs template matching at the block level and bilateral matching at the sub-block level when fruc_merge_idx is 2, that is, when template matching is indicated. As a result, it is possible to avoid that the template cannot be acquired and template matching cannot be performed, and it is possible to prevent a decrease in coding efficiency.

The flow of the pattern match vector derivation process in this embodiment will be described with reference to FIG. 21. FIG. 21 is a flowchart showing the flow of the pattern match vector derivation process in the present embodiment. The steps for performing the same processing as in the flowchart of FIG. 17 described above will be assigned the same step numbers, and the description thereof will be omitted.

As shown in FIG. 21, the processing at the block level is the same as the pattern match vector derivation processing shown in FIG. In the present embodiment, in the block search, the type of matching is determined by referring to the parameter indicating the matching mode. Specifically, template matching or bilateral matching is performed according to the value of fruc_merge_idx (fruc_merge_param). Further, after entering the loop of the subblock (S1055), the matching prediction unit 30373 proceeds to step S1058 without referring to the parameter indicating the matching mode (without determining whether or not fruc_merge_idx == 1). That is, the process proceeds to step S1058 without performing the processes of steps S1055 to 1057 of FIG. Then, at the sub-block level, bilateral matching is performed.

The above configuration includes an operation of performing a local search by template matching in a block search and performing a local search by using bilateral matching in a sub-block search.

The block search having the above configuration searches for a motion vector by performing a local search by bilateral matching or template matching, and the subblock search is a local search by template matching in the block search or bilateral matching. It is characterized in that the matching process is performed using only bilateral matching regardless of whether or not the local search is performed by.

The block search having the above configuration searches for a motion vector by performing a local search by bilateral matching or template matching based on a parameter indicating a matching mode, and a sub-block search is a parameter indicating a matching mode. Therefore, the matching process is performed using bilateral matching.

(Other configurations)
In the above-described embodiment, the matching prediction unit 30373 is configured to perform bilateral matching at the subblock level. The present invention is not limited to this, and for example, the bilateral matching may be performed only when it is estimated that the prediction accuracy is improved by performing the bilateral matching.

With reference to FIG. 22, the flow of the pattern match vector derivation process in the above configuration will be described. FIG. 22 is a flowchart showing the flow of the pattern match vector derivation process in the case of performing bilateral matching performance estimation. The steps for performing the same processing as in the flowchart of FIG. 17 described above will be assigned the same step numbers, and the description thereof will be omitted.

As shown in FIG. 22, the processing at the block level is the same as the pattern match vector derivation processing shown in FIG. In the present embodiment, in the processing at the sub-block level, the matching prediction unit 30373 estimates the performance of bilateral matching and sets the value of bilateralAvailable before entering the loop of the sub-block (S2201). Then, when it is estimated that the performance is improved by using the bilateral matching, that is, the prediction accuracy is improved by using the bilateral matching (YES in S2202), the subblock loop is entered and the processing after step S1055 is performed. Run. If it is determined in step S2202 that no performance is obtained in bilateral matching (NO in S2202), the motion vector derived at the block level is used as it is.

(Performance estimation method for bilateral matching)
The performance estimation of bilateral matching can be performed by, for example, the following method. The performance estimation of bilateral matching is executed by the matching prediction unit 30373.

(Estimation method 1)
When there are two reference pictures having the same absolute value of the distance between the pictures and the target picture, it is estimated that the performance is improved by using bilateral matching. That is, when the distance between the pictures of the two reference pictures of the target picture, TD0 and TD1, is 1: 1, it is estimated that the performance is improved by using bilateral matching. This is due to the following reasons. When TD0 and TD1 are different, the magnitude of the difference between the target picture and the two reference pictures is different. This adversely affects the search accuracy. On the other hand, if the absolute values of TD0 and TD1 are the same, the above difference does not occur and the search accuracy is not adversely affected.

The distances TD0 and TD1 between pictures may be derived as the difference between the picture sequence numbers POC between the target picture and the reference picture. Specifically, it is as follows.

TD0 = POC (CurrPic)-POC (Ref0)
TD1 = POC (CurrPic)-POC (Ref1)
Then, if the distances TD0 and TD1 between pictures are 1: 1, that is, if the absolute values match, it is estimated that the performance is improved by using bilateral matching. That is, it is as follows. Here, bilateralAvailable is a flag indicating whether or not it is estimated that the performance is improved by bilateral matching. If bilateral matching is estimated to improve performance, bilateralAvailable is true, otherwise bilateralAvailable is false.

| TD0 | == | TD1 |? BilateralAvailable = true: bilateralAvailable = false
(Estimation method 2)
If the distances TD0 and TD1 from the target picture are smaller than the threshold value, it may be estimated that the performance is improved. The larger the distance between pictures, the larger the difference between pictures, and the worse the search accuracy. Therefore, it is possible to exclude the case where the search system is bad by presuming that the performance is improved by using bilateral matching only when the distance between pictures is smaller than the threshold value.

Specifically, it can be determined by the following formula.

(| TDO | <= TH && | TD1 | <= TH)?
bilateralAvailable = true: bilateralAvailable = false
TH: Threshold. For example, "2" can be set as the threshold value.

(Estimation method 3)
The estimation method 1 and the estimation method 2 described above may be combined. That is, when the absolute values of the distances TD0 and TD1 between pictures match and are smaller than the threshold value, it may be estimated that the performance is improved by using bilateral matching.

Specifically, it can be determined by the following formula.

(| TD0 | == | TD1 | && | TDO | <= TH)?
bilateralAvailable = true: bilateralAvailable = false
It should be noted that TD1 may be used instead of TD0 for comparison with the threshold value TH.

(Embodiment 2-2)
In the present embodiment, the matching prediction unit 30373 derives a motion vector for a subblock that is not in contact with the upper end or the left end of the target block among the subblocks in the target block by a method other than template matching. As described above, since a subblock that does not touch the upper end or the left end of the target block cannot obtain a template, it can be appropriately encoded by using a method other than template matching. It is possible to prevent a decrease in the efficiency of the conversion.

(Method using bilateral matching)
First, for the subblocks that are not in contact with the upper end or the left end of the target block, a method using bilateral matching can be considered. That is, as shown in FIG. 23, when the parameter fruc_merge_idx indicating the matching mode is 2, that is, when template matching is indicated, the subblocks that are not in contact with the upper end or the left end of the target block are templates. Matching is performed, and bilateral matching is performed for other subblocks.

A specific flow of the pattern matching derivation process will be described with reference to FIG. 24. FIG. 24 is a flowchart showing the flow of processing for determining the motion vector derivation method to pattern matching or bilateral matching according to the position of the subblock in the block. The steps for performing the same processing as in the flowchart of FIG. 17 described above will be assigned the same step numbers, and the description thereof will be omitted.

As shown in FIG. 24, the processing at the block level is the same as the pattern matching vector derivation processing shown in FIG. In the present embodiment, after entering the loop of the subblock (S1055), the matching prediction unit 30373 determines in step S1056c whether or not fruc_merge_idx == 1 and also determines x! = 0 && y! = 0. I do. That is, it is determined whether or not the position of the sub-block is in contact with the upper end or the left end of the target block. Then, when the position of the subblock is in contact with the upper end or the left end of the target block (NO in S1056c), the template is acquired in step S1057, and the motion vector is derived by template matching in step S1058 and step S1059. Otherwise, if it is not in contact with the upper end or the left end of the target block (YES in S1056c), the process proceeds to step S1058 to perform bilateral matching.

In the present embodiment, in the sub-block search, the method for deriving the motion vector is selected according to the position of the sub-block to be derived in the block. That is, the method of deriving the motion vector differs depending on the position in the block.

(Method using motion vector of peripheral subblock)
For a subblock that does not touch the upper end or the left end of the target block, a method of using the motion vector of the peripheral subblock of the subblock can be considered.

A specific flow of the pattern matching derivation process will be described with reference to FIG. 25. FIG. 25 is a flowchart showing a flow of processing using motion vectors of peripheral subblocks according to the position of the subblock. The steps for performing the same processing as in the flowchart of FIG. 17 described above will be assigned the same step numbers, and the description thereof will be omitted.

As shown in FIG. 25, the processing at the block level is the same as the pattern match vector derivation processing shown in FIG. In the present embodiment, after entering the loop of the subblock (S1055), the matching prediction unit 30373 determines in step S1056 whether or not fruc_merge_idx == 1. Then, when fruc_merge_idx == 1 is true (YES in S1056), the motion vector for bilateral matching is derived in step S1058 and step S1059 using the motion vector derived at the block level. On the other hand, when fruc_merge_idx == 1 is false (NO in S1056), the determination of x == 0 || y == 0 is performed in step S2501. That is, it is determined whether or not the subblock is in contact with the upper end or the left end of the target block. If x == 0 || y == 0 is true (YES in S2501), that is, if the subblock touches the upper end or the left end of the target block, the template is acquired in step S1057, and step S1058, In step S1059, a motion vector for template matching is derived. On the other hand, when x == 0 || y == 0 is false (NO in S2501), that is, when the subblock does not touch the upper end or the left end of the target block, the motion vector of the peripheral block is used in step S2502. The motion vector of the subblock is derived.

A derivation method using motion vectors of peripheral blocks will be described with reference to FIG. 26. FIG. 26 is a diagram showing an example of a method of deriving a motion vector of a target subblock using a motion vector of a peripheral block.

FIG. 26A shows an example of using the motion vector of the subblock adjacent to the left of the target subblock. As shown in FIG. 26 (a), the motion vector of the subblock adjacent to the left of the target subblock can be used as the peripheral subblock.

FIG. 26B shows an example of using the motion vector of the subblock adjacent to the target subblock. As shown in FIG. 26 (b), the motion vector of the subblock adjacent to the target subblock can be used as the peripheral subblock.

FIG. 26 (c) shows an example in which the motion vector of the subblock adjacent to the left of the target subblock and the motion vector of the subblock adjacent on the target subblock are used. As shown in (c) of FIG. 26, the average of the motion vector of the subblock adjacent to the left of the target subblock as the peripheral subblock and the motion vector of the subblock adjacent on the target subblock. Can be the motion vector of the target subblock.

FIG. 26 (d) shows the motion vector of the subblock adjacent to the left of the target subblock, the motion vector of the subblock adjacent to the target subblock, and the motion of the subblock on the upper right of the target subblock. An example using a vector is shown. As shown in FIG. 26 (d), the motion vector of the subblock adjacent to the left of the target subblock as the peripheral subblock, the motion vector of the subblock adjacent on the target subblock, and the target. The median value with the motion vector of the subblock on the upper right of the subblock can be used as the motion vector of the target subblock. Specifically, the motion vector mv of the target subblock can be derived by the following equation.

mv.x = MEDIAN (mvA.x, mvB.x, mvC.x)
mv.y = MEDIAN (mvA.y, mvB.y, mvC.y)
Here, mvA: motion vector of the left subblock, mvB: motion vector of the upper subblock, mvC: motion vector of the upper right subblock, MEDIAN (a, b, c): center of arguments a, b, c. It means to return a value.

If there is no subblock from which the motion vector can be acquired in the upper right of the target subblock, the motion vector of the subblock may be derived as "0", or any of the remaining two subblocks. The median value may be derived by preparing two motion vectors of the subblock.

FIG. 26 (e) shows the motion vector of the subblock adjacent to the left of the target subblock, the motion vector of the subblock adjacent on the target subblock, and the block level of the target block to which the target subblock belongs. An example using the motion vector of is shown. As shown in FIG. 26 (e), the motion vector of the subblock adjacent to the left of the target subblock as the peripheral subblock, the motion vector of the subblock adjacent to the target subblock, and the block. The median value with the motion vector of the level can be the motion vector of the target subblock. Specifically, the motion vector mv of the target subblock can be derived by the following equation.

mv.x = MEDIAN (mvA.x, mvB.x, blkMv.x)
mv.y = MEDIAN (mvA.y, mvB.y, blkMv.y)
Here, mvA: motion vector of the left subblock, mvB: motion vector of the upper subblock, blkMv: motion vector of the block level, MEDIAN (a, b, c): median of the arguments a, b, c. It means to return.

[Embodiment 3]
(Embodiment 3-1)
First, the details of the prediction vector candidate pmvCand derivation process in step S123 of FIG. 16 will be described with reference to FIGS. 27 and 28. FIG. 27 is a flowchart showing the flow of the prediction vector candidate derivation process. FIG. 28 is a diagram for explaining a vector candidate derivation process. As shown in FIG. 27, in the prediction vector candidate derivation process, the vector candidate derivation unit 3033 first derives the left adjacency vector candidate mvLXA from the left adjoining region of the target block in step S1231. Specifically, the motion vector of candidate 1 or candidate 2 in the left adjacent region shown in FIG. 28 is the left adjacent vector candidate mvLXA. The numbers in the figure indicate the scan order. In the derivation process of the left adjacent vector candidate mvLXA, the left adjacent region of the target block is scanned from the lower side to the upper side, and the motion vector for the same reference image as the reference image of the target block is used as the prediction vector candidate.

Next, in step S1232, the vector candidate derivation unit 3033 derives the upper adjacent vector candidate mvLXB from the upper adjacent region of the target block. Specifically, the motion vector of the candidate 1, candidate 2, or candidate 3 in the upper adjacent region shown in FIG. 28 is the upper adjacent vector candidate mvLXB. In the derivation process of the upper adjacent vector candidate mvLXB, the upper adjacent region of the target block is scanned from the right side to the left side, and the motion vector for the same reference image as the reference image of the target block is used as the prediction vector candidate.

Next, in step S1233, the vector candidate derivation unit 3033 derives the time vector candidate mvLXCol from the block at the same position in the reference image (Collocated picture) that is temporally different from the current image including the target block. Specifically, the motion vector of candidate 1 or candidate 2 shown in FIG. 28 is the time vector candidate mvLXCol. In the derivation process of the time vector candidate mvLXCol, the block at the same position as the target block on the reference image is scanned from the lower right side to the upper left side, and the motion vector for the same reference image as the reference image of the target block is used as the prediction vector candidate. ..

Next, in step S1234, the vector candidate derivation unit 3033 derives a pattern match vector candidate by using pattern matching.

Then, in step S1235, the vector candidate derivation unit 3033 creates a prediction vector candidate list from the vector candidates derived in steps S1231 to S1234.

Next, the details of the pattern match vector candidate derivation process in step S1234 will be described with reference to FIG. 29. FIG. 29 is a flowchart showing the flow of the pattern match vector candidate derivation process.

As shown in FIG. 29, in the pattern match vector candidate derivation process, the vector candidate derivation unit 3033 first acquires a template in step 12341. Next, in step S12342, a block-level initial vector search is performed. Finally, in step S12343, a block-level local search is performed. The initial vector search and the local search are the same as the above-mentioned search in the description of FIG.

Next, the process of creating the prediction vector candidate list will be described with reference to FIG. FIG. 30 is a diagram showing an example of a pseudo code for creating a prediction vector candidate list.

In FIG. 30, mvpListLX shows a list of predictive vector candidates. mvLXPM shows patternon match vector candidates. availableFlagLXA is a flag indicating whether mvLXA could be derived. availableFlagLXB is a flag indicating whether mvLXB could be derived. availableFlagLXCol is a flag indicating whether or not mvLXCol could be derived. availableFlagLXPM is a flag indicating whether mvLXPM could be derived.

As shown in FIG. 30, in creating the predicted vector candidate list, the vector candidate derivation unit 3033 first finds the left adjacent vector in the predicted vector candidate list if the left adjacent vector candidate can be derived (if (availableFlagLXA)). Store candidates (mvpListLX [i ++] = mvLXA). Next, if the upper adjacent vector candidate has been derived and the derived upper adjacent vector candidate is different from the left adjacent vector candidate (if (availableFlagLXB && (mvLXA! = MvLXB))), the upper adjacent vector candidate is selected. Store in the prediction vector candidate list (mvpListLX [i ++] = mvLXB).

On the other hand, if the left adjacent vector candidate cannot be derived and the upper adjacent vector candidate can be derived (else if (availableFlagLXB)), the upper adjacent vector candidate is stored in the prediction vector candidate list (mvpListLX [i ++] =. mvLXB).

If the time vector candidate can be derived (availableFlagLXCol), the time vector candidate is stored in the prediction vector candidate list (mvpListLX [i ++] = mvLXCol). In this embodiment, the maximum size of the prediction vector candidate list is "2". Therefore, when the size of the prediction vector candidate list is smaller than the maximum size "2" (if (i <2)), the time vector is used. Store the candidates in the prediction vector candidate list.

Next, if the pattern match vector candidate has been derived and the beginning of the prediction vector candidate list is different from the pattern match vector candidate (availableFlagLXPM && mvpListLX [0]! = MvLXPM), all the elements of the prediction vector candidate list are one. It is lowered one by one, the part exceeding the maximum size (“2” in this embodiment) is deleted, and the pattern matching vector candidate is stored at the beginning of the prediction vector candidate list (mvpListLX [0] = mvLXPM).

Finally, if the prediction vector candidates are not stored up to the maximum size of the prediction vector candidate list, the zero vector (0, 0) is stored at the position.

(Embodiment 3-2)
In the above-described embodiment 3-1 in the creation of the prediction vector candidate list, the left adjacent vector candidate, the upper adjacent vector candidate, and the time vector candidate are used as the prediction vector candidate, and the pattern matching vector candidate is used as the prediction vector candidate. .. Then, the block referred to for deriving the left adjacent vector candidate, the upper adjacent vector candidate, and the time vector candidate is the same as the block referred to for deriving the initial vector when deriving the pattern match vector candidate. As shown in FIGS. 31A and 31B, the block for deriving the left adjacent vector candidate mvLXA, the block for deriving the upper adjacent vector candidate mvLXB, and the block for deriving the time vector candidate mvLXCol are , All are blocks used to derive the initial vector when deriving the pattern match vector.

Therefore, the same prediction vector candidate as the initial vector at the time of deriving the pattern match vector candidate may be stored in the prediction vector candidate list as any of the left adjacent vector candidate, the right adjacent vector candidate, and the time vector candidate.

When deriving a pattern match vector candidate, it is useless to store the initial vector in the prediction vector candidate list even though the initial vector is modified to derive the pattern match vector candidate and stored in the prediction vector candidate list. Is.

Therefore, in the present embodiment, the motion vector selected as the initial vector at the time of deriving the pattern match vector candidate is not stored in the prediction vector candidate list. As a result, vector candidates similar to the pattern match vector candidate are not stored in the prediction vector candidate list, and the coding efficiency can be improved.

A specific process of the present embodiment will be described with reference to FIGS. 32 and 33. FIG. 32 is a flowchart showing the flow of the prediction vector candidate derivation process. FIG. 33 is a diagram showing an example of a pseudo code for creating a prediction vector candidate list. The steps for performing the same processing as the processing in the step described with reference to FIG. 27 are designated by the same step numbers, and the description thereof will be omitted.

As shown in FIG. 32, in the prediction vector candidate derivation process in the present embodiment, the vector candidate derivation unit 3033 first performs the pattern match vector candidate derivation process in step S1234 in the process shown in FIG. 27, followed by Steps S1231, S1232, S1233, and S1235a are performed in this order. Further, in the pattern matching vector candidate derivation process in step S1234, the initial vector startMv is saved. Further, in the prediction vector candidate list creation process in step S1235a, the prediction vector candidate list is created by also using the judgment as to whether or not it is different from the initial vector startMv. A specific process for creating a prediction vector candidate list will be described with reference to FIG. 33.

As shown in FIG. 33, in the creation of the prediction vector candidate list in the present embodiment, the vector candidate derivation unit 3033 first, if the pattern matching vector candidate can be derived (if (availableFlagLXPM)), puts it in the prediction vector candidate list. Store the pattern match vector candidate (mvpListLX [i ++] = mvLXPM).

Also, if "the left adjacent vector candidate has been derived" and "the pattern matching vector candidate cannot be derived, or the left adjacent vector candidate is different from the initial vector" (if (availableFlagLXA && (! AvailableFlagLXPM | |) mvLXA! = startMv))), store left adjacent vector candidates in the predicted vector candidate list (mvpListLX [i ++] = mvLXA). Further, "upper adjacent vector candidate can be derived", "upper adjacent vector candidate and left adjacent vector candidate are different", and "pattern match vector candidate cannot be derived, or upper adjacent vector candidate". Is different from the initial vector "(if (availableFlagLXB && (mvLXA! = mvLXB) && (! AvailableFlagLXPM | | mvLXB! = StartMv))), store the top adjacent vector candidates in the predicted vector candidate list (mvpListLX [i ++] = mvLXB).

On the other hand, "the left adjacent vector candidate has not been derived", "the upper adjacent vector candidate has been derived", and "the pattern match vector candidate cannot be derived, or the upper adjacent vector candidate is the initial". If it is different from the vector (else if (availableFlagLXB && (! AvailableFlagLXPM | | mvLXB! = StartMv))), store the top adjacent vector candidates in the predicted vector candidate list (mvpListLX [i ++] = mvLXB).

Also, if "time vector candidates have been derived" and "pattern match vector candidates cannot be derived, or time vector candidates are different from the initial vector" (if (availableFlagLXCol && (! AvailableFlagLXPM | | mvLXCol!)! = startMv))), store the time vector candidates in the prediction vector candidate list (mvpListLX [i ++] = mvLXCol). In this embodiment, the maximum size of the prediction vector candidate list is "2", so when the size of the prediction vector candidate list is smaller than the maximum size "2" (if (i <2)), the time vector is set. Store in the prediction vector candidate list.

Finally, if the prediction vector candidates are not stored up to the maximum size of the prediction vector candidate list, the zero vector (0, 0) is stored at the position.

(Embodiment 3-3)
In the above-described embodiment 3-1, in order to create a prediction vector candidate list, a pattern matching vector candidate must always be derived regardless of whether or not it is used as a prediction vector. This is because it is determined whether or not the head of the prediction vector candidate list matches the pattern match vector candidate, and it is determined whether to shift the prediction vector candidate of the prediction vector candidate list one by one to the back.

Therefore, in the present embodiment, it is not determined whether or not the pattern match vector candidate matches the value at the beginning of the prediction vector candidate list, and the pattern match vector candidate is stored at the beginning of the prediction vector candidate list. Then, when the pattern match vector candidate is not used as the prediction vector, the pattern match vector candidate is not derived.

As a result, when the leading vector of the prediction vector candidate list is not used, in other words, when the pattern match vector candidate is not used as the prediction vector, the process of deriving the pattern match vector candidate can be omitted, so that the processing amount is reduced. Can be done.

A specific process of the present embodiment will be described with reference to FIGS. 34 and 35. FIG. 34 is a flowchart showing the flow of the prediction vector candidate derivation process. FIG. 35 is a diagram showing an example of a pseudo code for creating a prediction vector candidate list. The steps for performing the same processing as the processing in the step described with reference to FIG. 27 are designated by the same step numbers, and the description thereof will be omitted.

As shown in FIG. 34, in the prediction vector candidate derivation process in the present embodiment, the vector candidate derivation unit 3033 first determines in step S3401 whether or not the prediction vector is the head of the prediction vector candidate list (mvp_LX_idx == 0). Whether or not) is determined. If it is the head of the prediction vector candidate list (YES in S3401), the process proceeds to step S1234, and the pattern matching vector candidate derivation process proceeds. After that, the process proceeds to step S1235 to proceed to the prediction vector candidate list creation process.

On the other hand, if it is not the head of the prediction vector candidate list (NO in S3401), the process proceeds to steps S1231, S1232, S1233, and S1236. In step S1236, a zero vector candidate derivation process is performed. After that, the process proceeds to step S1235, and the process proceeds to the process of creating the prediction vector candidate list (in this case, the head of the prediction vector candidate list is empty).

A specific process for creating a prediction vector candidate list will be described with reference to FIG. 35. As shown in FIG. 35, in the creation of the prediction vector candidate list in the present embodiment, the vector candidate derivation unit 3033 first predicts when the prediction vector is the head of the prediction vector candidate list (if (mvp_LX_idx == 0)). The pattern matching vector candidate is stored at the beginning of the vector candidate list (mvpListLX [0] = mvLXPM).

On the other hand, when the prediction vector is not the head of the prediction vector candidate list, the prediction is made based on whether or not the left adjacent vector candidate, the upper adjacent vector candidate, and the time vector candidate can be derived, as in the pseudo code example shown in FIG. Create a vector candidate list.

(Embodiment 3-4)
In the above-described embodiment 3-3, the pattern match vector candidate is always stored at the top of the prediction vector candidate list. However, even when the pattern match vector candidate cannot be derived, such as when the template cannot be obtained, it is useless to always leave the head of the prediction vector candidate list free for the pattern match vector candidate.

Therefore, in the present embodiment, it is determined whether or not the pattern match vector candidate can be derived, and if it can be derived, the pattern match vector candidate is stored at the beginning of the prediction vector candidate list as shown in the third embodiment. If it is not possible to derive pattern match vector candidates, other prediction vector candidates are stored at the beginning of the prediction vector candidate list.

As a result, even though the pattern match vector candidate cannot be derived, the head of the prediction vector candidate list is not left open, the prediction accuracy can be improved, and the coding efficiency can be improved.

A specific process of the present embodiment will be described with reference to FIGS. 36 and 37. FIG. 36 is a flowchart showing the flow of the prediction vector candidate derivation process. FIG. 37 is a diagram showing an example of a pseudo code for creating a prediction vector candidate list. The steps for performing the same processing as the processing in the step described with reference to FIG. 27 are designated by the same step numbers, and the description thereof will be omitted.

As shown in FIG. 36, in the prediction vector candidate derivation process in the present embodiment, the vector candidate derivation unit 3033 first determines in step S3601 whether or not the pattern match vector candidate can be derived. Then, if possible, set the pattern match vector candidate derivation flag PMMVCandAvailable to true. If it is not possible, set it to false.

Next, in step S3602, if the vector candidate derivation unit 3033 can derive the pattern match vector candidate (PMMCandAvailable: true) and the prediction vector is at the top of the prediction vector candidate list (mvp_LX_idc == 0). Judge whether or not. If the pattern match vector candidate can be derived and the prediction vector is at the top of the prediction vector candidate list (YES in S3602), the process proceeds to step S1234.

On the other hand, if it is impossible to derive the pattern match vector candidate, or the prediction vector is not at the top of the prediction vector candidate list (NO in S3602), the process proceeds to step S1231. Subsequent processing is the same as the flowchart shown in FIG. 34.

A specific process for creating a prediction vector candidate list will be described with reference to FIG. 37. As shown in FIG. 37, in the creation of the prediction vector candidate list in the present embodiment, the vector candidate derivation unit 3033 can derive the pattern match vector candidate (PMMVCandAvailable == true), and the prediction vector is the prediction vector candidate list. In the case of the head, the pattern match vector candidate is stored in the first place in the prediction vector candidate list. If the pattern matching vector candidate cannot be derived, or if the prediction vector is not the head of the prediction vector candidate list, first, the position to store the vector candidate in the prediction vector candidate list is determined. If the pattern matching vector candidate cannot be derived, the vector candidate is stored from the beginning of the predicted vector candidate list (i = 0 at i = PMMVCandAvailable? 1: 0). If the pattern matching vector candidate can be derived, the vector candidate is stored from the second list of the predicted vector candidate list (i = 1 in i = PMMVCandAvailable? 1: 0).

As a method of determining whether or not the pattern match vector candidate can be derived, for example, it can be determined whether or not the template can be acquired. Whether or not the template can be obtained can be disabled if the coordinates of the template are outside the image. Specifically, it is as follows.
(xCurr == 0 && yCurr == 0)?
PMMVCandAvailable = false: PMMVCandAvailable = true
Also, when the coordinates of the template are different slices and the coordinates of the template are different tiles, it can be determined that the template cannot be acquired.

[Additional notes]
Further, a part or all of the image coding device 11 and the image decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image coding device 11 and the image decoding device 31 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Although one embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like are made without departing from the gist of the present invention. It is possible to do.

[Application example]
The image coding device 11 and the image decoding device 31 described above can be mounted on and used in various devices for transmitting, receiving, recording, and reproducing moving images. The moving image may be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like.

First, it will be described with reference to FIG. 38 that the above-mentioned image coding device 11 and image decoding device 31 can be used for transmission and reception of moving images.

FIG. 38A is a block diagram showing a configuration of a transmission device PROD_A equipped with an image coding device 11. As shown in FIG. 38 (a), the transmission device PROD_A modulates the carrier wave with the coding unit PROD_A1 that obtains the coded data by coding the moving image and the coded data obtained by the coding unit PROD_A1. It includes a modulation unit PROD_A2 for obtaining a modulation signal, and a transmission unit PROD_A3 for transmitting the modulation signal obtained by the modulation unit PROD_A2. The image coding device 11 described above is used as the coding unit PROD_A1.

The transmitter PROD_A has a camera PROD_A4 for capturing a moving image, a recording medium PROD_A5 for recording a moving image, an input terminal PROD_A6 for inputting a moving image from the outside, and a video input terminal PROD_A6 as a source of the moving image to be input to the coding unit PROD_A1. , An image processing unit A7 for generating or processing an image may be further provided. In FIG. 38 (a), the configuration in which the transmitting device PROD_A includes all of them is illustrated, but a part thereof may be omitted.

The recording medium PROD_A5 may be a recording of an unencoded moving image, or a moving image encoded by a recording coding method different from the transmission coding method. It may be a thing. In the latter case, a decoding unit (not shown) that decodes the coded data read from the recording medium PROD_A5 according to the coding method for recording may be interposed between the recording medium PROD_A5 and the coding unit PROD_A1.

FIG. 38B is a block diagram showing a configuration of a receiving device PROD_B equipped with an image decoding device 31. As shown in FIG. 38 (b), the receiving device PROD_B has a receiving unit PROD_B1 that receives a modulated signal, a demodulation unit PROD_B2 that obtains coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and demodulation unit PROD_B2. It includes a decoding unit PROD_B3 that obtains a moving image by decoding the coded data obtained by the unit PROD_B2. The image decoding device 31 described above is used as the decoding unit PROD_B3.

The receiving device PROD_B is a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording a moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. It may also have PROD_B6. In FIG. 38 (b), the configuration in which the receiving device PROD_B includes all of them is illustrated, but some of them may be omitted.

The recording medium PROD_B5 may be used for recording an unencoded moving image, or may be encoded by a recording coding method different from the transmission coding method. You may. In the latter case, it is preferable to interpose a coding unit (not shown) that encodes the moving image acquired from the decoding unit PROD_B3 according to the coding method for recording between the decoding unit PROD_B3 and the recording medium PROD_B5.

The transmission medium for transmitting the modulated signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the destination is not specified in advance) or communication (here, transmission in which the destination is specified in advance). It may refer to an aspect). That is, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by radio broadcasting. Further, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

In addition, servers (workstations, etc.) / clients (television receivers, personal computers, smartphones, etc.) such as VOD (Video On Demand) services and video sharing services using the Internet are transmitters that send and receive modulated signals via communication. This is an example of PROD_A / receiver PROD_B (usually, in LAN, either wireless or wired is used as a transmission medium, and in WAN, wired is used as a transmission medium). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multifunctional mobile phone terminals.

The client of the video sharing service has a function of decoding the coded data downloaded from the server and displaying it on the display, as well as a function of encoding the moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, it will be described with reference to FIG. 39 that the above-mentioned image coding device 11 and image decoding device 31 can be used for recording and reproducing moving images.

FIG. 39A is a block diagram showing a configuration of a recording device PROD_C equipped with the above-mentioned image coding device 11. As shown in FIG. 39 (a), the recording device PROD_C uses the coding unit PROD_C1 for obtaining coded data by coding the moving image and the coded data obtained by the coding unit PROD_C1 for the recording medium PROD_M. It has a writing unit PROD_C2 for writing. The image coding device 11 described above is used as the coding unit PROD_C1.

The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of a type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration). It may be loaded into a drive device (not shown) built in the recording device PROD_C, such as (trademark).

Further, the recording device PROD_C has a camera PROD_C3 that captures a moving image, an input terminal PROD_C4 for inputting a moving image from the outside, and a reception for receiving the moving image as a source of the moving image to be input to the coding unit PROD_C1. The unit PROD_C5 and the image processing unit PROD_C6 for generating or processing an image may be further provided. In FIG. 39 (a), a configuration in which the recording device PROD_C is provided with all of these is illustrated, but some of them may be omitted.

The receiving unit PROD_C5 may receive an unencoded moving image, or receives coded data encoded by a coding method for transmission different from the coding method for recording. It may be something to do. In the latter case, a transmission decoding unit (not shown) that decodes the coded data encoded by the transmission coding method may be interposed between the receiving unit PROD_C5 and the coding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, and the like (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of moving images). .. In addition, a camcorder (in this case, the camera PROD_C3 is the main source of the moving image), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main source of the moving image), and a smartphone (this). In this case, the camera PROD_C3 or the receiver PROD_C5 is the main source of moving images) is also an example of such a recording device PROD_C.

FIG. 39B is a block diagram showing the configuration of the reproduction device PROD_D equipped with the above-mentioned image decoding device 31. As shown in FIG. 39B, the reproduction device PROD_D obtains a moving image by decoding the read unit PROD_D1 that reads the coded data written in the recording medium PROD_M and the coded data read by the read unit PROD_D1. It has a decryption unit PROD_D2. The image decoding device 31 described above is used as the decoding unit PROD_D2.

The recording medium PROD_M may be of a type built in the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory. It may be of a type connected to the playback device PROD_D, or may be loaded into a drive device (not shown) built in the playback device PROD_D, such as (3) DVD or BD. good.

Further, the playback device PROD_D has a display PROD_D3 for displaying a moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. It may also have PROD_D5. In FIG. 39 (b), the configuration in which the reproduction device PROD_D includes all of them is illustrated, but a part thereof may be omitted.

The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits coded data encoded by a transmission coding method different from the recording coding method. It may be something to do. In the latter case, it is preferable to interpose a coding unit (not shown) that encodes the moving image by a coding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

Examples of such a playback device PROD_D include a DVD player, a BD player, an HDD player, and the like (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is the main supply destination of the moving image). .. In addition, a television receiver (in this case, display PROD_D3 is the main supply destination of moving images) and digital signage (also called electronic signage or electronic bulletin board, etc., and display PROD_D3 or transmitter PROD_D5 is the main supply destination of moving images. (First), desktop PC (in this case, output terminal PROD_D4 or transmitter PROD_D5 is the main supply destination of moving images), laptop or tablet PC (in this case, display PROD_D3 or transmitter PROD_D5 is video) An example of such a playback device PROD_D is a smartphone (in this case, the display PROD_D3 or the transmitter PROD_D5 is the main supply destination of the moving image), which is the main supply destination of the image.

(Hardware realization and software realization)
Further, each block of the image decoding device 31 and the image coding device 11 described above may be realized by hardware by a logic circuit formed on an integrated circuit (IC chip), or may be realized by a CPU (Central Processing Unit). It may be realized by software using.

In the latter case, each of the above devices is a CPU that executes an instruction of a program that realizes each function, a ROM (Read Only Memory) that stores the above program, a RAM (RandomAccess Memory) that expands the above program, the above program, and various data. It is equipped with a storage device (recording medium) such as a memory for storing the data. Then, an object of the embodiment of the present invention is a recording in which the program code (execution format program, intermediate code program, source program) of the control program of each of the above-mentioned devices, which is software for realizing the above-mentioned function, is readablely recorded by a computer. It can also be achieved by supplying the medium to each of the above devices and having the computer (or CPU or MPU) read and execute the program code recorded on the recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic discs such as floppy (registered trademark) discs / hard disks, and CD-ROMs (Compact Disc Read-Only Memory) / MO discs (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (registered trademark) and other optical discs, IC cards (including memory cards) / Cards such as optical cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory: registered trademark) / Semiconductor memories such as flash ROM, or PLD (Programmable logic device) ) And logic circuits such as FPGA (Field Programmable Gate Array) can be used.

Further, each of the above devices may be configured to be connectable to a communication network, and the above program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit the program code. For example, Internet, Intranet, Extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, Virtual Private network (Virtual Private). Network), telephone line network, mobile communication network, satellite communication network, etc. can be used. Further, the transmission medium constituting this communication network may be any medium as long as it can transmit the program code, and is not limited to a specific configuration or type. For example, even wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared ray such as IrDA (Infrared Data Association) and remote control. , BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It can also be used wirelessly. The embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the claims. That is, an embodiment obtained by combining technical means appropriately modified within the scope of the claims is also included in the technical scope of the present invention.
(Mutual reference of related applications)
This application claims the benefit of priority to Japanese Patent Application No. 2016-188790 filed on September 27, 2016, the entire contents of which are included in this document by reference to such application. ..

11 Image coding device (moving image coding device, predicted image generation device)
31 Image decoding device (moving image decoding device, predicted image generator)
112 Inter prediction parameter coding unit (motion vector generator)
1125, 3037 Subblock prediction parameter derivation part (motion vector derivation part)
303 Inter prediction parameter decoding unit (motion vector generator)
3033 Vector candidate derivation unit (prediction vector candidate derivation unit)
11253, 30373 Matching prediction unit (first motion vector search unit, second motion vector search unit)

Claims (5)

In a decoding device that decodes coded data
A memory for storing a first reference picture list representing a reference picture candidate corresponding to a target picture and a second reference picture list representing a reference picture candidate, and a memory.
Derivation to derive the motion vector of the subblock in the target picture using the first reference picture in the first reference picture list and the second reference picture in the second reference picture list. Department and
Equipped with
The derivation unit includes a distance between first pictures, which is a difference between the picture sequence number of the target picture and the picture sequence number of the first reference picture, the picture sequence number of the target picture, and the second reference picture. A decoding device that does not derive the motion vector of the subblock when the distance between the second pictures, which is the difference from the picture sequence number of the above, is different . The decoding device according to claim 1, further comprising a predictive image generation unit that generates a predictive image using the motion vector of the subblock . In a coding device that encodes image data
A memory for storing a first reference picture list representing a reference picture candidate corresponding to a target picture and a second reference picture list representing a reference picture candidate, and a memory.
Derivation to derive the motion vector of the subblock in the target picture using the first reference picture in the first reference picture list and the second reference picture in the second reference picture list. Department and
Equipped with
The derivation unit includes a distance between first pictures, which is a difference between the picture sequence number of the target picture and the picture sequence number of the first reference picture, the picture sequence number of the target picture, and the second reference picture. A coding device that does not derive the motion vector of the subblock when the distance between the second pictures, which is the difference from the picture sequence number of the above, is different . In the decoding method for decoding encoded data,
A step of storing a first reference picture list representing a reference picture candidate and a second reference picture list representing a reference picture candidate corresponding to the target picture, and a step of storing the second reference picture list.
A step of deriving a motion vector of a subblock in the target picture by using the first reference picture in the first reference picture list and the second reference picture in the second reference picture list.
Including
In the step of deriving, the distance between the first pictures, which is the difference between the picture sequence number of the target picture and the picture sequence number of the first reference picture, the picture sequence number of the target picture, and the second reference. A decoding method in which the motion vector of the subblock is not derived when the distance between the second pictures, which is the difference from the picture sequence number of the picture, is different . In the coding method for encoding image data,
A step of storing a first reference picture list representing a reference picture candidate and a second reference picture list representing a reference picture candidate corresponding to the target picture, and a step of storing the second reference picture list.
A step of deriving a motion vector of a subblock in the target picture by using the first reference picture in the first reference picture list and the second reference picture in the second reference picture list. ,
Including
In the step of deriving, the distance between the first pictures, which is the difference between the picture sequence number of the target picture and the picture sequence number of the first reference picture, the picture sequence number of the target picture, and the second reference. A coding method in which the motion vector of the subblock is not derived when the distance between the second pictures, which is the difference from the picture sequence number of the picture, is different .

Images