JP2022048142A - Image decoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce complexities of coding/decoding of a moving image.

SOLUTION: An image decoding device (31) that decodes a coding unit (CU) comprises: a CU decoding part (20) that decodes a parameter that is used in predicting and a conversion coefficient; an inverse-quantization/reverse-conversion part (311) that inverse-quantizes and reverse-converts the conversion coefficient for each conversion unit (TU) to generate a prediction error; a predicted image generation part (101) that generates a predicted image from the parameter that is used in predicting; and an addition part (312) that adds the prediction error and the predicted image so as to generate a decoded image. The CU decoding part determines whether a target TU size is below a maximum TU size and is larger than a minimum TU size and whether a prediction mode is an inter prediction or not; decodes a flag indicating whether or not the coefficient should be converted in unit smaller than TU, on the basis of the determined result; and when the flag indicates that the coefficient should be converted in the unit smaller than TU, decodes a mode showing a dividing direction, and divides the TU in a direction shown by the mode to convert a region with a size corresponding to a quarter of the TU.

SELECTED DRAWING: Figure 10

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a predictive image generation device, an image decoding device, and an image coding device.

An image coding device that generates coded data by encoding a moving image and an image decoding that generates a decoded image by decoding the coded data in order to efficiently transmit or record the moving image. The device is used.

Specific examples of the moving image coding method include the methods proposed by H.264 / AVC and HEVC (High-Efficiency Video Coding).

In such a moving image coding method, the image (picture) constituting the moving image is a slice obtained by dividing the image and a coding unit (coding unit) obtained by dividing the slice. : CU)), and managed by a hierarchical structure consisting of a prediction unit (PU) and a conversion unit (TU), which are blocks obtained by dividing the coding unit, and coded for each CU / It is decrypted.

Further, in such a moving image coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and the predicted image is obtained from the input image (original image). The predicted residual obtained by subtraction (sometimes referred to as a "difference image" or "residual image") is encoded. Examples of the method for generating the predicted image include inter-screen prediction (inter-prediction) and in-screen prediction (intra-prediction).

Further, Non-Patent Document 1 is mentioned as a technique for coding and decoding moving images in recent years.

Furthermore, in recent years, as a method of dividing a coding tree unit (CTU) that constitutes a slice, CTU is divided into a binary tree in addition to the QT division that divides a quad tree. BT division has been introduced. This BT division includes horizontal division and vertical division.

"Algorithm Description of Joint Exploration Test Model 2", JVET-B1002, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11, 20-26 February 2016

With the introduction of BT division in addition to QT division as the CTU division method, the division pattern to the CU increases, the amount of moving image coding / decoding processing increases, and the coding / decoding becomes complicated. Further, when the CU is further divided into PU, TU, etc., the amount of moving image coding / decoding processing increases only by the division pattern of PU, TU or the combination of PU, TU.

Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding device or an image coding device capable of reducing the complexity of coding / decoding of a moving image. It is in.

The image decoding device according to one aspect of the present invention includes a CU decoding unit that decodes parameters and conversion coefficients used for prediction in an image decoding device that decodes a coding unit (CU) in order to solve the above problems. An inverse quantization / inverse conversion unit that generates a prediction error by inversely quantizing and inversely converting the conversion coefficient for each conversion unit (TU), a prediction image generation unit that generates a prediction image from prediction parameters, and a prediction error and prediction. The CU decoding unit includes an addition unit that adds images to generate a decoded image, and the CU decoding unit has a target TU size that is equal to or less than the maximum TU size, is larger than the minimum TU size, and has a prediction mode of inter-prediction. A mode that indicates the division direction when it is determined whether or not there is, and based on the determination, a flag indicating whether or not to convert in a unit smaller than TU is decoded, and when the flag indicates conversion in a unit smaller than TU. Is decoded, the TU is divided in the direction indicated by the mode, and an area having a size of 1/4 of the TU is converted.

According to one aspect of the present invention, it contributes to reducing the complexity of coding / decoding of moving images.

It is a schematic diagram which shows the structure of the image transmission system which concerns on this embodiment. It is a figure which shows the hierarchical structure of the data of the coded stream which concerns on this embodiment. It is a figure which shows the pattern of a PU division mode. (A) to (h) show the partition shape when the PU division modes are 2Nx2N, 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and NxN, respectively. It is a conceptual diagram which shows an example of a reference picture and a reference picture list. It is a block diagram which shows the structure of the image coding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the image decoding apparatus which concerns on this embodiment. It is a schematic diagram which shows the structure of the inter prediction image generation part of the image coding apparatus which concerns on this embodiment. It is a figure which showed the structure of the transmission device equipped with the image coding device, and the receiving device equipped with an image decoding device which concerns on this embodiment. (A) shows a transmitting device equipped with an image coding device, and (b) shows a receiving device equipped with an image decoding device. It is a figure which showed the structure of the recording apparatus equipped with the image coding apparatus which concerns on this embodiment, and the reproduction apparatus equipped with the image decoding apparatus. (A) shows a recording device equipped with an image coding device, and (b) shows a playback device equipped with an image decoding device. It is a block diagram which showed the main part structure of the image decoding apparatus which concerns on this embodiment. It is a flowchart explaining the schematic operation of the CT information decoding unit which concerns on this embodiment. It is a figure which shows the example of the hierarchy of CU, and the division pattern corresponding to a hierarchy. It is a figure which shows the list of the division pattern example of TU corresponding to each division pattern of CU. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a figure which shows the list of the division pattern example of TU corresponding to each shape of CU. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a flowchart explaining the schematic operation of the TT information decoding unit which concerns on one Embodiment of this invention. It is a functional block diagram which shows the structural example of the inverse quantization / inverse DCT part which concerns on one Embodiment of this invention. In the figure, (a) is a table showing a transformation set, and (b) is a diagram showing a default function of each transformation basis. It is a reference table when selecting a conversion set in intra prediction. It is a functional block diagram which shows the structural example of the reverse DCT part which concerns on one Embodiment of this invention. It is a reference table when selecting a conversion set in inter-prediction. It is a figure which shows 5 types of shape examples of TU. It is a reference table when selecting a conversion set in inter-prediction. It is a reference table when selecting a conversion set in inter-prediction. It is a reference table when selecting a conversion set in inter-prediction. It is a reference table when selecting a conversion set in inter-prediction. It is a figure which shows the relationship between the number of pixels of 4 types of TU, and the shape of 5 types of TU. It is a reference table when selecting a conversion set in inter-prediction. It is a reference table when selecting a conversion set in inter-prediction. It is a figure which shows the relationship between 3 kinds of long sides of TU, and 5 kinds of shapes of TU. It is a reference table when selecting a conversion set in inter-prediction. It is a reference table when selecting a conversion set in inter-prediction. It is a reference table when selecting a conversion set in intra prediction. It is a flowchart explaining the schematic operation of the conversion set derivation part which concerns on one Embodiment of this invention. It is a reference table when selecting a conversion set in intra prediction. A reference table for selecting conversion sets. A reference table for selecting conversion sets. A reference table for selecting conversion sets. A reference table for selecting conversion sets. It is a flowchart explaining the schematic operation of the conversion set derivation part which concerns on one Embodiment of this invention. It is a flowchart explaining the schematic operation of the conversion set derivation part which concerns on one Embodiment of this invention.

(First Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic diagram showing a configuration of an image transmission system 1 according to the present embodiment.

The image transmission system 1 is a system that transmits a code obtained by encoding a coded image, decodes the transmitted code, and displays the image. The image transmission system 1 includes an image coding device 11, a network 21, an image decoding device 31, and an image display device 41.

An image T showing an image of a single layer or a plurality of layers is input to the image coding device 11. A layer is a concept used to distinguish a plurality of pictures when there is one or more pictures constituting a certain time. For example, encoding the same picture with a plurality of layers having different image quality and resolution results in scalable coding, and coding a picture with different viewpoints with a plurality of layers results in view scalable coding. When prediction (interlayer prediction, interview prediction) is performed between pictures of a plurality of layers, the coding efficiency is greatly improved. In addition, even when prediction is not performed (simulcast), the coded data can be collected.

The network 21 transmits the coded stream Te generated by the image coding device 11 to the image decoding device 31. The network 21 is an internet (internet), a wide area network (WAN: Wide Area Network), a small-scale network (LAN: Local Area Network), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, but may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Further, the network 21 may be replaced with a storage medium on which a coded stream Te such as a DVD (Digital Versatile Disc) or BD (Blue-ray Disc) is recorded.

The image decoding device 31 decodes each of the coded streams Te transmitted by the network 21 and generates one or a plurality of decoded images Td respectively.

The image display device 41 displays all or a part of one or a plurality of decoded images Td generated by the image decoding device 31. The image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Further, in spatial scalable coding and SNR scalable coding, when the image decoding device 31 and the image display device 41 have high processing power, an extended layer image with high image quality is displayed and only lower processing power is obtained. Displays a base layer image that does not require as high processing power and display power as the extended layer.

<Operator>
The operators used herein are described below.

>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, and | = is a sum operation (OR) with another condition.

x? y: z is a ternary operator that takes y when x is true (other than 0) and z when x is false (0).

Clip3 (a, b, c) is a function that clips c to a value greater than or equal to a and less than or equal to b. Is a function that returns c (where a <= b).

<Structure of coded stream Te>
Prior to the detailed description of the image coding device 11 and the image decoding device 31 according to the present embodiment, the data structure of the coded stream Te generated by the image coding device 11 and decoded by the image decoding device 31 will be described. ..

FIG. 2 is a diagram showing a hierarchical structure of data in the coded stream Te. The coded stream Te typically includes a sequence and a plurality of pictures constituting the sequence. 2 (a) to 2 (f) of FIG. 2 show a coded video sequence that defines a sequence SEQ, a coded picture that defines a picture PICT, a coded slice that defines slice S, and a coded slice that defines slice data, respectively. It is a figure which shows the coding unit (CU) included in the data, the coding tree unit included in the coding slice data, and the coding tree unit.

(Coded video sequence)
The coded video sequence defines a set of data that the image decoder 31 refers to in order to decode the sequence SEQ to be processed. As shown in FIG. 2A, the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and an addition. Includes SEI (Supplemental Enhancement Information). Here, the value shown after # indicates the layer ID. FIG. 2 shows an example in which coded data of # 0 and # 1, that is, layer 0 and layer 1 exist, but the type of layer and the number of layers do not depend on this.

A video parameter set VPS is a set of coding parameters common to a plurality of moving images in a moving image composed of a plurality of layers, and a set of coding parameters related to the plurality of layers included in the moving image and individual layers. The set is defined.

The sequence parameter set SPS defines a set of coding parameters that the image decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are specified. There may be a plurality of SPS. In that case, select one of multiple SPS from PPS.

The picture parameter set PPS defines a set of coding parameters that the image decoding device 31 refers to in order to decode each picture in the target sequence. For example, it includes a reference value of the quantization width used for decoding a picture (pic_init_qp_minus26) and a flag (weighted_pred_flag) indicating the application of weighted prediction. There may be a plurality of PPSs. In that case, select one of a plurality of PPSs from each picture in the target sequence.

(Encoded picture)
The coded picture defines a set of data referred to by the image decoding device 31 in order to decode the picture PICT to be processed. As shown in FIG. 2B, the picture PICT includes slices S0 to S _NS-1 (NS is the total number of slices contained in the picture PICT).

In the following, when it is not necessary to distinguish each of slices S0 to S _NS-1 , the subscript of the sign may be omitted. The same applies to the data included in the coded stream Te described below and having a subscript.

(Coded slice)
The coded slice defines a set of data referred to by the image decoding device 31 in order to decode the slice S to be processed. As shown in FIG. 2 (c), the slice S includes the slice header SH and the slice data SDATA.

The slice header SH includes a group of coding parameters referred to by the image decoding device 31 for determining the decoding method of the target slice. The slice type specification information (slice_type) that specifies the slice type is an example of the coding parameter included in the slice header SH.

The slice types that can be specified by the slice type specification information are (1) I slice that uses only intra prediction at the time of coding, (2) unidirectional prediction at the time of coding, or P slice that uses intra prediction. (3) B slices using unidirectional prediction, bidirectional prediction, or intra prediction at the time of coding can be mentioned.

The slice header SH may include a reference (pic_parameter_set_id) to the picture parameter set PPS included in the coded video sequence.

(Coded slice data)
The coded slice data defines a set of data referred to by the image decoding device 31 in order to decode the slice data SDATA to be processed. As shown in FIG. 2D, the slice data SDATA includes a coding tree unit (CTU). A CTU is a block of fixed size (for example, 64x64) that constitutes a slice, and is sometimes called a maximum coding unit (LCU).

(Coded tree unit)
As shown in FIG. 2 (e), a set of data referred to by the image decoding device 31 for decoding the coded tree unit to be processed is defined. The coded tree unit is divided by recursive quadtree division (QT division) or binary tree division (BT division). A node with a tree structure obtained by recursive quadtree division or binary tree division is called a coding node (CN). The intermediate node of the quadtree and the binary tree is a coding tree (CT), and the coding tree unit itself is defined as the top-level coding tree.

The CTU includes a QT split flag (cu_split_flag) indicating whether or not to perform QT split, and a BT split mode (split_bt_mode) indicating how to split the BT split. When cu_split_flag is 1, it is divided into four coding nodes CN. When cu_split_flag is 0, the coding node CN is not divided and has one coding unit (CU: Coding Unit) as a node. On the other hand, when split_bt_mode is 2, it is horizontally divided into two coding nodes CN. When split_bt_mode is 1, it is vertically split into two coding nodes CN. When split_bt_mode is 0, the coding node CN is not split and has one coding unit CU as a node. The coding unit CU is a terminal node (leaf node) of the coding node and is not further divided. The coding unit CU is a basic unit of coding processing.

When the size of the coding tree unit CTU is 64x64 pixels, the size of the coding unit is 64x64 pixels, 64x32 pixels, 32x64 pixels, 32x32 pixels, 64x16 pixels, 16x64 pixels, 32x16 pixels, 16x32 pixels, 16x16 pixels. , 64x8 pixels, 8x64 pixels, 32x8 pixels, 8x32 pixels, 16x8 pixels, 8x16 pixels, and 8x8 pixels can be taken.

(Encoding unit)
As shown in FIG. 2 (f), a set of data referred to by the image decoding device 31 for decoding the coding unit to be processed is defined. Specifically, the coding unit is composed of a prediction tree, a conversion tree, and a CU header CUH. The CU header defines the prediction mode, division method (PU division mode), and the like.

In the prediction tree, prediction information (reference picture index, motion vector, etc.) of each prediction unit (PU) obtained by dividing the coding unit into one or a plurality is defined. In other words, the prediction unit is one or more non-overlapping regions that make up the coding unit. The prediction tree also includes one or more prediction units obtained by the above division. In the following, the prediction unit obtained by further dividing the prediction unit is referred to as a “sub-block”. The subblock is composed of a plurality of pixels. If the predictor unit and the subblock are the same size, then there is only one subblock in the predictor unit. If the predictor unit is larger than the size of the subblock, the predictor unit is divided into subblocks. For example, when the prediction unit is 8x8 and the subblock is 4x4, the prediction unit is divided into four subblocks consisting of two horizontal divisions and two vertical divisions.

The prediction process may be performed for each prediction unit (subblock).

Roughly speaking, there are two types of division in the prediction tree: intra-prediction and inter-prediction. The intra prediction is a prediction within the same picture, and the inter prediction refers to a prediction process performed between pictures different from each other (for example, between display times and between layer images).

In the case of intra prediction, there are two division methods: 2Nx2N (same size as the coding unit) and NxN.

In the case of inter-prediction, the division method is encoded by the PU division mode (part_mode) of the coded data, 2Nx2N (same size as the coding unit), 2NxN, 2NxnU, 2NxnD, Nx2N, nLx2N, nRx2N, and There are NxN and so on. Note that 2NxN and Nx2N indicate a 1: 1 symmetric division, and 2NxnU, 2NxnD and nLx2N and nRx2N indicate a 1: 3 and 3: 1 asymmetric division. The PUs included in the CU are expressed as PU0, PU1, PU2, and PU3 in that order.

FIGS. 3A to 3H specifically illustrate the shape of the partition (position of the boundary of the PU division) in each PU division mode. (A) of FIG. 3 shows a partition of 2Nx2N, and (b), (c), and (d) show a partition (horizontal partition) of 2NxN, 2NxnU, and 2NxnD, respectively. (E), (f), and (g) indicate partitions (vertical partitions) in the case of Nx2N, nLx2N, and nRx2N, respectively, and (h) indicates an NxN partition. The horizontally long partition and the vertically long partition are collectively called a rectangular partition, and 2Nx2N and NxN are collectively called a square partition.

Further, in the transformation tree (TT: Transform Tree), the coding unit is divided into one or a plurality of transformation units (TU: Transform Unit), and the position and size of each transformation unit are defined. In other words, the conversion unit is one or more non-overlapping regions constituting the coding unit. The conversion tree also includes one or more conversion units obtained from the above divisions.

For division in the conversion tree, an area of the same size as the coding unit is allocated as the conversion unit, and the conversion unit is obtained by dividing the CU into quadtrees (TU division) in the same way as the above-mentioned division of CU. There is. The conversion process is performed for each conversion unit.

(Prediction parameter)
The prediction image of the prediction unit (PU) is derived by the prediction parameters associated with the PU. Prediction parameters include prediction parameters for intra-prediction or prediction parameters for inter-prediction. Hereinafter, the prediction parameters of the inter-prediction (inter-prediction parameters) will be described. The inter-prediction parameter is composed of the prediction list utilization flags predFlagL0 and predFlagL1, the reference picture indexes refIdxL0 and refIdxL1, and the motion vectors mvL0 and mvL1. The prediction list usage flags predFlagL0 and predFlagL1 are flags indicating whether or not the reference picture list called the L0 list and the L1 list is used, respectively, and the reference picture list corresponding to the case where the value is 1 is used. In the present specification, when "a flag indicating whether or not it is XX" is described, it is assumed that a flag other than 0 (for example, 1) is XX, 0 is not XX, and logical negation, logical product, etc. Treat 1 as true and 0 as false (same below). However, in an actual device or method, other values can be used as true values and false values.

The syntax elements for deriving the inter-prediction parameters contained in the coded data include, for example, PU split mode part_mode, merge flag merge_flag, merge index merge_idx, inter-prediction identifier inter_pred_idc, reference picture index refIdxLX, prediction vector index mvp_LX_idx, There is a difference vector mvdLX.

(Reference picture list)
The reference picture list is a list composed of reference pictures stored in the reference picture memory 306. FIG. 4 is a conceptual diagram showing an example of a reference picture and a reference picture list. In FIG. 4A, the rectangle is a picture, the arrow is a reference relationship of the picture, the horizontal axis is time, I, P, and B in the rectangle are intra pictures, single prediction pictures, bi-prediction pictures, and the numbers in the rectangle are. Indicates the decoding order. As shown in the figure, the decoding order of the pictures is I0, P1, B2, B3, B4, and the display order is I0, B3, B2, B4, P1. FIG. 4B shows an example of a reference picture list. The reference picture list is a list representing candidates for reference pictures, and one picture (slice) may have one or more reference picture lists. In the example of the figure, the target picture B3 has two reference picture lists, L0 list RefPicList0 and L1 list RefPicList1. When the target picture is B3, the reference pictures are I0, P1, and B2, and the reference picture has these pictures as elements. In each prediction unit, the reference picture index refIdxLX specifies which picture in the reference picture list RefPicListX is actually referenced. The figure shows an example in which reference pictures P1 and B2 are referenced by refIdxL0 and refIdxL1.

(Merge prediction and AMVP prediction)
Prediction parameter decoding (encoding) methods include a merge mode and an AMVP (Adaptive Motion Vector Prediction) mode. The merge flag merge_flag is a flag for identifying these. The merge prediction mode is a mode in which the prediction list utilization flag predFlagLX (or the inter prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the motion vector mvLX are not included in the coded data and are derived from the prediction parameters of the neighboring PUs that have already been processed. The AMVP mode is a mode in which the inter-prediction identifier inter_pred_idc, the reference picture index refIdxLX, and the motion vector mvLX are included in the coded data. The motion vector mvLX is encoded as a prediction vector index mvp_LX_idx that identifies the prediction vector mvpLX and a difference vector mvdLX.

The inter-prediction identifier inter_pred_idc is a value indicating the type and number of reference pictures, and takes any value of PRED_L0, PRED_L1, and PRED_BI. PRED_L0 and PRED_L1 indicate that the reference pictures managed by the reference picture list of the L0 list and the L1 list are used, respectively, and indicate that one reference picture is used (single prediction). PRED_BI indicates that two reference pictures are used (bipred), and the reference pictures managed by the L0 list and the L1 list are used. The prediction vector index mvp_LX_idx is an index indicating the prediction vector, and the reference picture index refIdxLX is an index indicating the reference pictures managed by the reference picture list. LX is a description method used when L0 prediction and L1 prediction are not distinguished, and by replacing LX with L0 and L1, the parameters for the L0 list and the parameters for the L1 list are distinguished.

The merge index merge_idx is an index indicating which of the prediction parameter candidates (merge candidates) derived from the PU for which processing has been completed is used as the prediction parameter of the PU to be decoded.

(Motion vector)
The motion vector mvLX indicates the amount of deviation between blocks on two different pictures. The prediction vector and difference vector related to the motion vector mvLX are called the prediction vector mvpLX and the difference vector mvdLX, respectively.

(Inter prediction identifier inter_pred_idc and prediction list usage flag predFlagLX)
The relationship between the inter-prediction identifier inter_pred_idc and the prediction list usage flags predFlagL0 and predFlagL1 is as follows and can be converted to each other.

inter_pred_idc = (predFlagL1 << 1) + predFlagL0
predFlagL0 = inter_pred_idc & 1
predFlagL1 = inter_pred_idc >> 1
As the inter-prediction parameter, the prediction list use flag may be used, or the inter-prediction identifier may be used. Further, the determination using the prediction list utilization flag may be replaced with the determination using the inter-prediction identifier. On the contrary, the determination using the inter-prediction identifier may be replaced with the determination using the prediction list utilization flag.

(Judgment of bipred biPred)
The bipred BiPred flag biPred can be derived depending on whether the two prediction list usage flags are both 1. For example, it can be derived by the following formula.

biPred = (predFlagL0 == 1 && predFlagL1 == 1)
The flag biPred can also be derived by whether or not the inter-prediction identifier is a value indicating that two prediction lists (reference pictures) are used. For example, it can be derived by the following formula.

biPred = (inter_pred_idc == PRED_BI)? 1: 0
The above equation can also be expressed by the following equation.

biPred = (inter_pred_idc == PRED_BI)
Note that PRED_BI can use a value of 3, for example.

(Configuration of image decoder)
Next, the configuration of the image decoding device 31 according to the present embodiment will be described. FIG. 5 is a schematic view showing the configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an entropy decoding unit 301, a prediction parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, and a reverse. It includes a quantized / inverse DCT unit 311 and an addition unit 312.

Further, the prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The prediction image generation unit 308 includes an inter prediction image generation unit 309 and an intra prediction image generation unit 310.

The entropy decoding unit 301 performs entropy decoding on the coded stream Te input from the outside, separates and decodes each code (syntax element). The separated codes include prediction information for generating a prediction image, residual information for generating a difference image, and the like.

The entropy decoding unit 301 outputs a part of the separated codes to the prediction parameter decoding unit 302. Some of the separated codes are, for example, the prediction mode predMode, the PU split mode part_mode, the merge flag merge_flag, the merge index merge_idx, the interprediction identifier inter_pred_idc, the reference picture index refIdxLX, the prediction vector index mvp_LX_idx, and the difference vector mvdLX. The control of which code is decoded is performed based on the instruction of the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantization coefficient to the inverse quantization / inverse DCT unit 311. This quantization coefficient is a coefficient obtained by performing DCT (Discrete Cosine Transform) on the residual signal and quantizing it in the coding process.

The inter-prediction parameter decoding unit 303 decodes the inter-prediction parameter with reference to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301.

The inter-prediction parameter decoding unit 303 outputs the decoded inter-prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307. The details of the inter-prediction parameter decoding unit 303 will be described later.

The intra prediction parameter decoding unit 304 decodes the intra prediction parameter by referring to the prediction parameter stored in the prediction parameter memory 307 based on the code input from the entropy decoding unit 301. The intra prediction parameter is a parameter used in the process of predicting the CU in one picture, for example, the intrapred mode. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.

The intra prediction parameter decoding unit 304 may derive an intra prediction mode that differs depending on the luminance and the color difference. In this case, the intra prediction parameter decoding unit 304 decodes the luminance prediction mode IntraPredModeY as the luminance prediction parameter and the luminance prediction mode IntraPredModeC as the color difference prediction parameter. The luminance prediction mode IntraPredModeY is 35 modes, and corresponds to planar prediction (0), DC prediction (1), and direction prediction (2 to 34). The color difference prediction mode IntraPredModeC uses any one of planar prediction (0), DC prediction (1), direction prediction (2-34), and LM mode (35). The intra prediction parameter decoding unit 304 decodes a flag indicating whether or not IntraPredModeC is in the same mode as the luminance mode, and if it shows that the flag is in the same mode as the luminance mode, assigns IntraPredModeY to IntraPredModeC and the flag is the luminance mode. If it is shown that the mode is different from the mode, the planar prediction (0), DC prediction (1), direction prediction (2-34), and LM mode (35) may be decoded as IntraPredModeC.

The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the addition unit 312.

The reference picture memory 306 stores the decoded image of the CU generated by the addition unit 312 at a position predetermined for each of the picture to be decoded and the CU.

The prediction parameter memory 307 stores the prediction parameters in a predetermined position for each picture and prediction unit (or subblock, fixed size block, pixel) to be decoded. Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode pred Mode separated by the entropy decoding unit 301. .. The stored inter-prediction parameters include, for example, the prediction list utilization flag predFlagLX (inter-prediction identifier inter_pred_idc), the reference picture index refIdxLX, and the motion vector mvLX.

The prediction mode predMode input from the entropy decoding unit 301 is input to the prediction image generation unit 308, and the prediction parameter is input from the prediction parameter decoding unit 302. Further, the prediction image generation unit 308 reads the reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of the PU using the input prediction parameter and the read reference picture in the prediction mode indicated by the prediction mode predMode.

Here, when the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the reference picture read out, and the prediction image of the PU is predicted by the inter prediction. To generate.

The inter-prediction image generation unit 309 uses a motion vector as a reference from the reference picture indicated by the reference picture index refIdxLX to the reference picture list (L0 list or L1 list) in which the prediction list usage flag predFlagLX is 1. The reference picture block at the position indicated by mvLX is read from the reference picture memory 306. The inter-predicted image generation unit 309 performs prediction based on the read reference picture block and generates a predicted image of the PU. The inter-prediction image generation unit 309 outputs the generated prediction image of the PU to the addition unit 312.

When the prediction mode predMode indicates the intra prediction mode, the intra prediction image generation unit 310 performs the intra prediction using the intra prediction parameters input from the intra prediction parameter decoding unit 304 and the reference picture read out. Specifically, the intra prediction image generation unit 310 reads from the reference picture memory 306 an adjacent PU that is a picture to be decoded and is within a predetermined range from the PU to be decoded. When the decoding target PU moves sequentially in the order of so-called raster scan, the predetermined range is, for example, one of the adjacent PUs on the left, upper left, upper, and upper right, and differs depending on the intra prediction mode. The order of raster scan is the order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.

The intra prediction image generation unit 310 predicts the read adjacent PU in the prediction mode indicated by the intra prediction mode IntraPredMode, and generates a prediction image of the PU. The intra prediction image generation unit 310 outputs the generated prediction image of the PU to the addition unit 312.

When the intra prediction parameter decoding unit 304 derives an intra prediction mode different in luminance and color difference, the intra prediction image generation unit 310 determines the planar prediction (0), DC prediction (1), and direction according to the luminance prediction mode IntraPredModeY. A predicted image of the luminance PU is generated by any of the predictions (2 to 34), and the planar prediction (0), DC prediction (1), direction prediction (2 to 34), and LM mode are generated according to the color difference prediction mode IntraPredModeC. A predicted image of the PU of color difference is generated by any of (35).

The inverse quantization / inverse DCT unit 311 inversely quantizes the quantization coefficient input from the entropy decoding unit 301 to obtain the DCT coefficient. The inverse quantization / inverse DCT unit 311 performs an inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient and calculates a residual signal. The inverse quantization / inverse DCT unit 311 outputs the calculated residual signal to the addition unit 312.

The addition unit 312 adds the prediction image of the PU input from the inter-prediction image generation unit 309 or the intra-prediction image generation unit 310 and the residual signal input from the inverse quantization / inverse DCT unit 311 for each pixel. Generate a decrypted image of PU. The addition unit 312 stores the generated PU decoded image in the reference picture memory 306, and outputs the decoded image Td in which the generated PU decoded image is integrated for each picture to the outside.

(Configuration of image coding device)
Next, the configuration of the image coding device 11 according to the present embodiment will be described. FIG. 6 is a block diagram showing the configuration of the image coding device 11 according to the present embodiment. The image coding device 11 includes a prediction image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, an entropy coding unit 104, an inverse quantization / inverse DCT unit 105, an addition unit 106, a loop filter 107, and a prediction parameter memory. (Prediction parameter storage unit, frame memory) 108, reference picture memory (reference image storage unit, frame memory) 109, coding parameter determination unit 110, and prediction parameter coding unit 111 are included. The prediction parameter coding unit 111 includes an inter prediction parameter coding unit 112 and an intra prediction parameter coding unit 113.

For each picture of the image T, the prediction image generation unit 101 generates a prediction image P of the prediction unit PU for each coding unit CU which is a region in which the picture is divided. Here, the prediction image generation unit 101 reads out the decoded block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter coding unit 111. The prediction parameter input from the prediction parameter coding unit 111 is, for example, a motion vector in the case of inter-prediction. The prediction image generation unit 101 reads out the block at the position on the reference image indicated by the motion vector starting from the target PU. In the case of intra prediction, the prediction parameter is, for example, an intra prediction mode. The pixel value of the adjacent PU used in the intra prediction mode is read from the reference picture memory 109, and the predicted image P of the PU is generated. The prediction image generation unit 101 generates a prediction image P of the PU for the read reference picture block by using one of the plurality of prediction methods. The predicted image generation unit 101 outputs the generated predicted image P of the PU to the subtraction unit 102.

The predicted image generation unit 101 has the same operation as the predicted image generation unit 308 described above. For example, FIG. 7 is a schematic diagram showing the configuration of the inter-predictive image generation unit 1011 included in the predictive image generation unit 101. The inter-prediction image generation unit 1011 includes a motion compensation unit 10111 and a weight prediction unit 10112. Since the motion compensation unit 10111 and the weight prediction unit 10112 have the same configurations as the motion compensation unit 3091 and the weight prediction unit 3094 described above, the description thereof will be omitted here.

The prediction image generation unit 101 uses the parameters input from the prediction parameter coding unit to generate the prediction image P of the PU based on the pixel value of the reference block read from the reference picture memory. The predicted image generated by the predicted image generation unit 101 is output to the subtraction unit 102 and the addition unit 106.

The subtraction unit 102 generates a residual signal by subtracting the signal value of the predicted image P of the PU input from the predicted image generation unit 101 from the pixel value of the corresponding PU of the image T. The subtraction unit 102 outputs the generated residual signal to the DCT / quantization unit 103.

The DCT / quantization unit 103 performs DCT on the residual signal input from the subtraction unit 102, and calculates the DCT coefficient. The DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain the quantization coefficient. The DCT / quantization unit 103 outputs the obtained quantization coefficient to the entropy coding unit 104 and the inverse quantization / inverse DCT unit 105.

A quantization coefficient is input to the entropy coding unit 104 from the DCT / quantization unit 103, and a coding parameter is input from the prediction parameter coding unit 111. The coded parameters to be input include, for example, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, and a merge index merge_idx.

The entropy coding unit 104 entropy-codes the input quantization coefficient and coding parameter to generate a coded stream Te, and outputs the generated coded stream Te to the outside.

The inverse quantization / inverse DCT unit 105 inversely quantizes the quantization coefficient input from the DCT / quantization unit 103 to obtain the DCT coefficient. The inverse quantization / inverse DCT unit 105 performs an inverse DCT on the obtained DCT coefficient and calculates a residual signal. The inverse quantization / inverse DCT unit 105 outputs the calculated residual signal to the addition unit 106.

The addition unit 106 adds the signal value of the predicted image P of the PU input from the prediction image generation unit 101 and the signal value of the residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and decodes the signal. Generate an image. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image generated by the addition unit 106.

The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 at positions predetermined for each of the picture to be coded and the CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a position predetermined for each of the picture to be encoded and the CU.

The coding parameter determination unit 110 selects one set from the plurality of sets of coding parameters. The coding parameter is the above-mentioned prediction parameter or a parameter to be coded generated in connection with the prediction parameter. The prediction image generation unit 101 generates a prediction image P of the PU using each of these sets of coding parameters.

The coding parameter determination unit 110 calculates a cost value indicating the magnitude of the amount of information and the coding error for each of the plurality of sets. The cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by the coefficient λ. The code amount is the amount of information of the coded stream Te obtained by entropy-coding the quantization error and the coding parameter. The squared error is the sum of the squared values of the residual values of the residual signal calculated by the subtraction unit 102 between the pixels. The coefficient λ is a real number greater than the preset zero. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated cost value. As a result, the entropy coding unit 104 outputs the selected set of coding parameters to the outside as a coding stream Te, and does not output the set of unselected coding parameters. The coding parameter determination unit 110 stores the determined coding parameter in the prediction parameter memory 108.

The prediction parameter coding unit 111 derives a format for coding from the parameters input from the coding parameter determination unit 110 and outputs the encoding format to the entropy coding unit 104. Derivation of the form for encoding is, for example, deriving a difference vector from a motion vector and a prediction vector. Further, the prediction parameter coding unit 111 derives the parameters necessary for generating the prediction image from the parameters input from the coding parameter determination unit 110, and outputs the parameters to the prediction image generation unit 101. The parameters required to generate the predicted image are, for example, motion vectors in sub-block units.

The inter-prediction parameter coding unit 112 derives an inter-prediction parameter such as a difference vector based on the prediction parameter input from the coding parameter determination unit 110. In the inter-prediction parameter coding unit 112, the inter-prediction parameter decoding unit 303 (see FIG. 6, etc.) derives the inter-prediction parameter as a configuration for deriving the parameters necessary for generating the prediction image to be output to the prediction image generation unit 101. Includes a configuration that is partially the same as the configuration to be used. The configuration of the inter-prediction parameter coding unit 112 will be described later.

The intra prediction parameter coding unit 113 derives a coding format (for example, MPM_idx, rem_intra_luma_pred_mode, etc.) from the intra prediction mode IntraPredMode input from the coding parameter determination unit 110.

(Main part configuration of image decoding device)
FIG. 10 shows a block diagram showing a configuration of a main part of the image decoding apparatus according to the present embodiment. In this figure, in order to simplify the figure, the illustration of some members included in the block diagram shown in FIG. 10 is omitted. Further, for convenience of explanation, the same reference numerals are given to the members having the same functions as the members shown in FIG. 5, and the description thereof will be omitted.

As shown in FIG. 10, the image decoding device 31 includes a decoding module 9, a CT information decoding unit 10, a predictive image generation unit 308, an inverse quantization / inverse DCT transform unit 311, a reference picture memory 306, an addition unit 312, and a loop filter. It includes a 305, a header decoding unit 19, and a CU decoding unit 20. The CU decoding unit 20 further includes a PU information decoding unit 12 and a TT information decoding unit 13 (divided information decoding unit, division unit), and the TT information decoding unit 13 further includes a TU decoding unit 22.

(Decryption module)
Hereinafter, the schematic operation of each module will be described. The decoding module 9 performs a decoding process for decoding a syntax value from a binary. More specifically, the decoding module 9 decodes and decodes the syntax value encoded by the entropy coding method such as CABAC based on the coded data supplied from the supply source and the syntax type. Returns the syntax value to the supplier.

In the example shown below, the sources of the coded data and the syntax type are the CT information decoding unit 10 and the CU decoding unit 20 (PU information decoding unit 12 and TT information decoding unit 13).

(Header decoder)
The header decoding unit 19 decodes the VPS (video parameter set), SPS, PPS, and slice header of the coded data input from the image coding device 11.

(CT information decoding unit)
The CT information decoding unit 10 uses the decoding module 9 to perform decoding processing of the coding tree unit and the coding tree for the coded data input from the image coding device 11. Specifically, the CT information decoding unit 10 decodes the CTU information and the CT information from the encoded data according to the following procedure.

First, the CT information decoding unit 10 decodes the tree unit header CTUH from the CTU information included in the CTU by using the decoding module 9. Next, the CT information decoding unit 10 decodes the QT division flag indicating whether or not the target CT is QT-divided and the BT division mode indicating the division method of the BT division of the target CT from the CT information included in the CT. , The target CT is recursively divided and decoded until the QT division flag and the BT division mode do not notify further division. Finally, the tree unit footer CTUF is decoded from the CTU information.

The tree unit header CTUH and the tree unit footer CTUF include coding parameters referenced by the image decoder 31 to determine how to decode the target coded tree unit. In addition to the QT division flag and the BT division mode, the CT information may include parameters applied by the target CT and the lower coding node.

(CU decoding unit)
The CU decoding unit 20 is composed of a PU information decoding unit 12 and a TT information decoding unit 13, and decodes the PUI information and the TTI information of the lowest-level coded tree CT (that is, CU).

(PU information decoding unit)
In the PU information decoding unit 12, the PU information (merge flag (merge_flag), merge index (merge_idx), prediction vector index (mvp_idx), reference image index (ref_idx), inter prediction identifier (inter_pred_flag), and difference vector (mvd) of each PU are used. ) Etc.) are decoded using the decoding module 9.

(TT information decoding unit)
The TT information decoding unit 13 decodes the TT information (TU split flag SP_TU (split_transform_flag), TU residual flag CBP_TU (cbf_cb, cbf_cr, cbf_luma), and TU) of the conversion tree TT using the decoding module 9.

Further, the TT information decoding unit 13 includes a TU decoding unit 22. The TU decoding unit 22 decodes the QP update information (quantization correction value) when the TU contains a residual. The QP update information is a value indicating a difference value from the quantized parameter predicted value qPpred, which is the predicted value of the quantized parameter QP. Further, the TU decoding unit 22 decodes the quantization predicted residual (residual_coding).

(Processing of CT information decoding)
The operation of CT information decoding by the CT information decoding unit 10 will be described in detail with reference to FIG. FIG. 11 is a flowchart illustrating a schematic operation of the CT information decoding unit 10 according to the embodiment of the present invention.

The CT information decoding S1400 by the CT information decoding unit 10 performs QT information decoding and BT information decoding. Hereinafter, the QT information decoding by the CT information decoding unit 10 and the BT information decoding will be described in order.

First, the CT information decoding unit 10 decodes CT information from the coded data and recursively decodes the coding tree CT (coding_quadtree).

(S1411) The CT information decoding unit 10 determines whether or not the decoded CT information has a QT division flag. When the CT information decoding unit 10 determines that the QT division flag is present, the CT information decoding unit 10 transitions to S1421. In other cases, the transition to S1422 occurs.

(S1421) When the CT information decoding unit 10 determines that the QT division flag is present, the CT information decoding unit 10 decodes the QT division flag (split_cu_flag) which is a syntax element.

(S1422) The CT information decoding unit 10 omits decoding of the QT division flag split_cu_flag from the coded data in other cases, that is, when the QT division flag split_cu_flag does not appear in the coded data (not present). , QT split flag split_cu_flag is set to 0 and derived (infer).

(S1431) When the QT split flag split_cu_flag is other than 0 (= 1), the CT information decoding unit 10 executes (S1441) described later, shifts to one level lower, and performs the subsequent processing (S1411). repeat. In other cases (when the QT split flag split_cu_flag is 0), the transition to S1451 is performed.

(S1441) The CT information decoding unit 10 performs QT division. Specifically, the CT information decoding unit 10 decodes four coded tree CTs in the CT layer one layer below. The CT information decoding unit 10 continues the QT information decoding started from S1411 even in the lower coded tree CT.

Subsequently, the CT information decoding unit 10 decodes the CT information from the coded data and recursively decodes the coding tree CT (coding_binarytree). At this time, the QT depth (before division) is first set to 0, and then the QT depth is incremented by 1 each time the QT division is performed.

(S1451) First, the CT information decoding unit 10 determines whether or not the decoded CT information has a BT division mode. When the CT information decoding unit 10 determines that there is a BT division mode, the CT information decoding unit 10 transitions to S1461. In other cases, the transition to S1462 occurs.

(S1461) When the CT information decoding unit 10 determines that there is a BT division mode, it decodes the BT division mode split_bt_mode, which is a syntax element.

(S1462) The CT information decoding unit 10 omits decoding of the BT division mode split_bt_mode from the coded data in other cases, that is, when the BT division mode split_bt_mode does not appear in the coded data, and the BT division mode. Derived with split_bt_mode set to 0.

(S1471) When the BT division mode split_bt_mode is other than 0 (= 1 or 2), the CT information decoding unit 10 executes (S1481) described later, shifts to one level lower, and after (S1451). Repeat the process. In other cases (when the BT division mode split_bt_mode is 0), the CT information decoding unit 10 does not divide the target coding tree and ends the process.

(S1481) The CT information decoding unit 10 performs BT division. Specifically, in the BT layer one layer below, the CT information decoding unit 10 performs horizontal division when the BT division mode is 1, decodes two coded tree CTs, and when the BT division mode is 2. Performs vertical division and decodes the two coded tree CTs. The CT information decoding unit 10 continues the BT information decoding started from S1451 even in the lower coded tree CT. At this time, the BT depth before division is first set to 0, and then the BT depth is incremented by 1 each time the BT division is performed.

In the CT information decoding described above, the QT tree and the BT tree are in different layers (there is a hierarchical relationship between the QT tree and the BT tree. That is, the QT tree recursively between the QT trees. It is split and the BT tree is recursively split between the BT trees. The BT tree can exist as a subordinate node of the QT tree, but the QT tree does not exist in the subordinate node of the BT tree). Decryption method. With this method, QT division cannot be performed after BT division, but it is not necessary to determine the presence or absence of the QT division flag after BT division. However, this embodiment is not limited to this, and the QT tree and the BT tree are in the same layer (there is no hierarchical relationship between the QT tree and the BT tree. That is, the CT tree in which the QT tree and the BT tree are combined is used. The decryption method is adopted assuming that the BT tree exists as a lower node of the QT tree, and at the same time, the QT tree also exists in the lower node of the BT tree. May be good. In this case, either QT division or BT division can be selected, that is, QT division can be performed even after BT division, but it is necessary to determine the presence or absence of the QT division flag each time.

(Limitation of TU division)
When the CU obtained by QT division or BT division is further divided into TUs, the amount of moving image coding / decoding processing increases by the TU division pattern. Therefore, in the present embodiment, in order to reduce the complexity of coding / decoding of the moving image, the TU division for the CU is limited to only one layer. At the same time, the present embodiment includes a plurality of TU division patterns. Specifically, the hierarchy of TU division for CU is limited to 0 (that is, no division) or 1 (that is, paddy character division, horizontal 4 division, and vertical 4 division are performed for 1 layer). Further, in the present embodiment, a plurality of TU division patterns (in this example, three patterns of rice field division, horizontal four division, and vertical four division) are provided.

FIG. 12 shows an example of the CU hierarchy and the shape of the CU corresponding to the hierarchy. As shown in FIG. 12, the shape of the CU is different for each layer. In the present embodiment, the TU division hierarchy for the CU is limited to 0 or 1, so that the TU division pattern is limited to that shown in FIG. 13 according to the CU hierarchy and shape. FIG. 13 is a list of TU division pattern examples corresponding to each shape of the CU. That is, when the shape of the target CU is 64 × 64, the TU division for the target CU is limited to the TU division into the TU shape shown in FIG. 13 according to the TU division mode.

In this case, in the image coding device 11, when encoding each CU, the TU division flag (division information) indicating whether or not to divide the CU into TUs and the TU division indicating the TU division mode of TU division are indicated. The mode (division information) is included in the TT information for one layer and encoded. When the TU division flag is 0, it means that the TU division is not performed, and when the TU division flag is 1, it means that the TU division is performed. In addition, the TU split mode tuSplitMode is no split (tuSplitMode = 0), a paddle character split (that is, a 2x2 grid pattern), and a horizontal split into four horizontally. It is either 4 divisions (that is, 4 divisions by horizontal boundaries) (tuSplitMode = 2) or 4 vertical divisions (that is, 4 divisions by vertical boundaries) (tuSplitMode = 3). .. The method of assigning a value to the TU split mode tuSplitMode may be another method regardless of the above. For example, the TU division mode may be a paddy character division (tuSplitMode = 0), a horizontal division into four (tuSplitMode = 1), and a vertical division into four (tuSplitMode = 2).

Further, in the above, the information regarding the TU division (division information) is expressed using two values of the TU division flag indicating the presence / absence of division and the TU division mode indicating the division method (division pattern), but one value ( A configuration using only the TU division mode) may be used. In this case, it is assumed that no division is included as one of the values of the TU division mode.

In the present embodiment, since the TU division for the CU is limited to only one layer, the complexity of coding / decoding the moving image can be reduced. Further, in the present embodiment, instead of reducing the hierarchy of TU division, the type of TU division is increased from one type (rice field division) to three types (rice field division, horizontal four division, and vertical four division). Therefore, it becomes possible to correspond to various patterns.

The plurality of division patterns are not limited to the rice field division, the horizontal four division, and the vertical four division. As will be described later, 8 × 8 division and 16 × 16 division may be added.

(Processing of TT information decoding)
The operation of TT information decoding by the TT information decoding unit 13 when the TU division layer is limited to only one layer will be described in detail with reference to FIG. FIG. 14 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

The TT information decoding unit 13 decodes the TT information from the encoded data and decodes the conversion tree TT.

(S1511) First, the TT information decoding unit 13 determines whether or not the target TU size (target CU size, target TU size) is equal to or less than the maximum TU size and larger than the minimum TU size.

(S1521) The TT information decoding unit 13 derives the TU division flag when the above conditions are not satisfied, that is, when the TU division flag does not appear in the coded data. The derivation of this TU division flag will be described later.

(S1522) On the other hand, the TT information decoding unit 13 decodes the TU division flag, which is a syntax element, when the above conditions are satisfied.

(S1531) Subsequently, the TT information decoding unit 13 refers to the TU division flag and decodes the TU division mode when it indicates that the TU division flag is divided. The TT information decoding unit 13 decodes the TU division mode from the coded data, and performs TU division in the TU division mode specified by the decoded TU division mode. The derivation of this TU division mode will be described later.

(S1541) The TT information decoding unit 13 decodes the TU residual flag indicating whether or not the target TU contains a residual.

(S1551) Next, the TT information decoding unit 13 decodes the AMT flag amt_flag.

(S1561) The TU decoding unit 22 of the TT information decoding unit 13 decodes the TU based on the TU residual flag and the AMT flag amt_flag.

(S1571) The TT information decoding unit 13 performs the same processing in the other TUs to continue the TT information decoding.

Since the process from decoding the TU residual flag to decoding the TU is a well-known technique, detailed description thereof will be omitted here.

(TU division flag derivation process)
Hereinafter, the derivation of the TU division flag (S1521) by the TT information decoding unit 13 will be described with reference to FIG. FIG. 15 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

(S1611) First, the TT information decoding unit 13 determines whether or not the target TU size is larger than the maximum TU size.

(S1621) When the TT information decoding unit 13 determines that the target TU size is larger than the maximum TU size, the target TU needs to be divided into TUs, so the TT information decoding unit 13 derives the TU division flag as 1. In this case, the TU division flag does not appear in the coded data, and the TU division flag is not coded by the image coding device 11 or decoded by the image decoding device 31.

(S1631) On the other hand, when the target TU size is the maximum TU size or less, the TT information decoding unit 13 determines whether or not the target TU size is the minimum TU size or less.

(S1641) When the TT information decoding unit 13 determines that the target TU size is equal to or less than the minimum TU size, the target TU does not need to be divided into TUs, so the TT information decoding unit 13 derives the TU division flag as 0. In this case, the TU division flag does not appear in the coded data, and the TU division flag is not coded by the image coding device 11 or decoded by the image decoding device 31.

In cases other than the above, the TU division flag appears in the coded data, and the TU division flag is coded by the coding device and decoded by the decoding device (S1522).

(Derivation of TU division mode)
Subsequently, the derivation of the TU division mode (S1531) by the TT information decoding unit 13 will be described with reference to FIG. FIG. 16 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

(S1711) First, the TT information decoding unit 13 determines whether or not the TU division flag is 1.

(S1721) The TT information decoding unit 13 does not divide the target TU when the TU division flag is other than 1 (= 0). Specifically, the TT information decoding unit 13 sets the number of loops N (number of divisions N) to 1, sftx = 0 and sfty = 0.

(S1731) When the TU division flag is 1, the TT information decoding unit 13 decodes the division mode from the coded data.

(S1741) Subsequently, the TT information decoding unit 13 determines whether or not the division mode indicates the character division of the rice field.

(S1751) When the division mode indicates the character division of the rice field, the TT information decoding unit 13 divides the target TU into the character of the rice field. Specifically, the TT information decoding unit 13 sets the number of loops N to 4, sftx = 1 and sfty = 1. The number of loops N is the number of times the processing from (S1511) to (S1531) shown in FIG. 14 is performed. sftx is the shift amount for calculating the CU width (CUwidth), and sfty is the shift amount for calculating the CU height (CUheight).

(S1761) On the other hand, when the division mode does not indicate the character division of the rice field, the TT information decoding unit 13 determines whether or not the division mode is horizontal four division.

(S1771) The TT information decoding unit 13 divides the target TU into four horizontally when the division mode indicates four horizontal divisions. Specifically, the TT information decoding unit 13 sets the number of loops N to 4, sftx = 0 and sfty = 2.

(S1781) On the other hand, when the division mode does not indicate horizontal 4-division, the TT information decoding unit 13 determines whether or not the division mode is vertical 4-division.

(S1791) The TT information decoding unit 13 divides the target TU into four vertical divisions when the division mode indicates vertical four divisions. Specifically, the TT information decoding unit 13 sets the number of loops N to 4, sftx = 2 and sfty = 0.

(S1800) The TT information decoding unit 13 determines the CU shape (CUwidth, CUheight) that is bit-shifted by sftx and sfty for which the CU shape (CUwidth, CUheight) is set as the TU shape (TUwidth, TUheight) one level below.

TUwidth = CUwidth >> sftx
TUheight = CUheight >> sfty
If the shape of CU and the shape of TU are represented by a logarithm of 2 (log2CUwidth, log2CUheight) and (log2TUwidth, log2TUheight), it is determined as follows.

log2TUwidth = log2CUwidth --sftx
log2TUheight = log2CUheight --sfty
note that,
CUwidth = 1 << log2CUwidth, CUheight = 1 << log2CUheight, TUwidth = 1 << log2TUwidth, TUheight = 1 << log2TUheight.

The shape (CUwidth, CUheight) of each CU is given to the TT information decoding unit 13 together with the depth Depth (cqtDepth, cbtDepth) of each CU.

The TT information decoding unit 13 repeats the processes from (S1511) to (S1531) shown in FIG. 14 by the number of loops N. For example, when the TU division mode is no division, the above processing is performed once, and when the TU division mode is a paddy-shaped division, horizontal 4-division or vertical 4-division, the above processing is performed four times.

By this series of processing, the CU is divided into TUs. In the present embodiment, when the CU is divided into TUs for one layer by the above series of processes, no further TU division is performed.

(Modification example)
If the CU size is large, it may not be possible to perform appropriate division according to the characteristics of the image if only one layer of TU division is used. Therefore, as the TU division mode of the CU, a division into 8 × 8 size (hereinafter referred to as 8 × 8 division) may be used in addition to the rice field division, the horizontal 4 division, or the vertical 4 division.

FIG. 17 shows a list of TU division pattern examples corresponding to each shape of the CU. As shown in FIG. 17, the TU division mode includes 8 × 8 division up to the layer having a large CU size (for example, the layer that does not include the division pattern into 8 × 8 size (D3 in this figure)). ing. In this case, the TT information decoding unit 13 has a TU division mode of no division (tuSplitMode = 0), a paddy character division (tuSplitMode = 1), a horizontal division of 4 (tuSplitMode = 2), and a vertical division of 4 (tuSplitMode = 3). And 8 × 8 division (tuSplitMode = 4) is determined. Again, the TU split mode assignment is not based on the above, but is divided into rice fields (tuSplitMode = 0), horizontal 4 splits (tuSplitMode = 1), vertical 4 splits (tuSplitMode = 2), and 8x8 splits (tuSplitMode = 3). ) Etc. may be used.

Derivation of the TU division mode by the TT information decoding unit 13 in this case will be described with reference to FIG. FIG. 18 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

Since the processes from S1711 to S1791 are the same as the processes shown in FIG. 16, the description thereof will be omitted here.

(S1792) When the division mode does not indicate vertical 4 division, the TT information decoding unit 13 determines whether or not the division mode is 8 × 8 division.

(S1793) The TT information decoding unit 13 divides the target TU into 8 × 8 when the division mode indicates 8 × 8 division. Specifically, the TT information decoding unit 13 sets the number of loops N (number of divisions N) to CUwidth * CUheight >> 6, sftx = log2CUwidth − 3 and sfty = log2CUheight − 3.

(S1800) The TT information decoding unit 13 determines the CU shape (CUwidth, CUheight) that is bit-shifted by sftx and sfty for which the CU shape (CUwidth, CUheight) is set as the TU shape (TUwidth, TUheight) one level below.

The division mode is not limited to 8 × 8 division. For example, division into 16 × 16 sizes (hereinafter referred to as 16 × 16 division) may be used. FIG. 19 shows a list of TU division pattern examples corresponding to each shape of the CU. As shown in FIG. 19, the TU division mode includes 16 × 16 division up to a layer having a large CU size (for example, a layer that does not include a division pattern into 16 × 16 size (D1 in this figure)). ing.

Alternatively, both 8 × 8 division and 16 × 16 division may be used as the division mode. FIG. 20 shows a list of TU division pattern examples corresponding to each shape of the CU in this case. As shown in FIG. 20, the TU division mode includes 16 × 16 division up to a predetermined layer (for example, a layer that does not include a division pattern into 16 × 16 size (D1 in this figure)). In the subsequent layers (for example, the layers not including the division pattern into 8 × 8 size (D2 and D3 in this figure), the TU division mode includes 8 × 8 division.

(Second embodiment)
There are as many TU division patterns as the number of combinations of BT division and TU division when the CU obtained by BT division is further divided into TUs, and the TU division pattern when the CU obtained by QT division is further divided into TUs. There are as many combinations of QT division patterns and TU division patterns. Since the pattern of BT division is larger than the pattern of QT division, the division pattern of TU of CU obtained by BT division is larger than the division pattern of TU of CU obtained by QT division. The amount of coding / decoding processing is large. Therefore, in the present embodiment, the TU division for the CU obtained by the QT division is limited to only one layer, and the TU division for the CU obtained by the BT division is not performed. Specifically, the TU division hierarchy is set to 0 (that is, no division) by the BT division, and the TU division hierarchy for the CU obtained by the QT division is 0 (that is, no division) or 1 (that is, the character of the rice field). It is limited to 1 layer of division, horizontal 4 division, and vertical 4 division).

In the present embodiment, the TU division hierarchy for the CU obtained by the QT division is limited to 0 or 1, so that the TU division pattern is limited to that shown in FIG. 21 according to the CU hierarchy and shape. is doing. FIG. 21 is a list of TU division pattern examples corresponding to each shape of the CU. That is, when the shape of the target CU is a square such as 64 × 64, the TU division for the target CU is limited to the TU division into the TU shape shown in FIG. 21 according to the TU division mode.

In the present embodiment, the TU division for the CU obtained by the QT division is limited to only one layer, and the TU division for the CU obtained by the BT division is not performed. Therefore, the complexity of coding / decoding of the moving image is increased. Coding efficiency can be maintained while reducing.

(Processing of TT information decoding)
The operation of TT information decoding by the TT information decoding unit 13 when the layer of TU division for the CU obtained by QT division is limited to only one layer will be described in detail with reference to FIG. 22. FIG. 22 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

The TT information decoding unit 13 decodes the TT information from the encoded data and decodes the conversion unit TU.

(S1512) First, the TT information decoding unit 13 determines whether or not the target TU size is equal to or less than the maximum TU size, larger than the minimum TU size, and only QT division is performed. The depth Depth (cqtDepth, cbtDepth) of each CU is given to the TT information decoding unit 13 together with the shape (CUwidth, CUheight) of each CU. When cbtDepth = 0 in the target TU, the TT information decoding unit 13 can determine that there is no BT division and only the QT division.

(S1521) The TT information decoding unit 13 derives (infer) the TU division flag when the above conditions are not satisfied, that is, when the TU division flag does not appear in the coded data. The derivation of this TU division flag will be described later.

Since the processes from S1522 to S1571 are the same as the processes shown in FIG. 14, the description thereof will be omitted here.

(TU division flag derivation process)
Hereinafter, the derivation of the TU division flag by the TT information decoding unit 13 will be described with reference to FIG. 23. FIG. 23 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

Since the processes from S1611 to S1641 are the same as the processes shown in FIG. 15, the description thereof will be omitted here.

(S1651) The TT information decoding unit 13 stabilizes whether the target CU is BT-divided. That is, it is determined whether or not the target CU is a CU without BT division or a CU obtained by BT division (presence or absence of BT division in the target CU). Any of the following can be used as the determination method.
1) If the CU is not square (CUwidth! = CUHeight), it is determined that there is a BT division. If the CU is square (CUwidth == CUHeight), it is determined that there is no BT division.
2) When the BT depth is other than 0, it is determined that there is a BT division, and when the BT depth is 0, it is determined that there is no BT division.

(S1661) When the TT information decoding unit 13 determines that the target CU is not BT-divided, the TU division flag of the target CU can be TU-divided, so that the TU division flag is decoded (S1661).

(S1671) On the other hand, when the TT information decoding unit 13 determines that the target CU is a BT-divided CU, the TU division of the target CU is not performed, so the TT information decoding unit 13 derives the TU division flag as 0. That is, the TT information decoding unit 13 determines the shape of the CU (CUwidth, CUheight) as the shape of the TU one level below (TUwidth, TUheight).

(Modification-1)
The TU division mode for the CU obtained by QT division is not limited to no division, paddy division, horizontal 4 division and vertical 4 division. For example, the TU division mode for the CU obtained by QT division may be limited to no division and paddy division. Specifically, the hierarchy of TU division for CU is limited to 0 (that is, no division) or 1 (that is, the character division of rice field is performed for one layer).

In the present embodiment, the TU division pattern for the CU obtained by the QT division is shown in FIG. 24 according to the hierarchy and shape of the CU by limiting the TU division to no division or the paddy character division. Is limited to. FIG. 24 is a list of TU division pattern examples corresponding to each shape of the CU. That is, when the shape of the target CU is a square such as 64 × 64, the TU division for the target CU is limited to the TU division into the TU shape shown in FIG. 24, depending on the TU division mode.

In this case, in the process of deriving the TU division mode by the TT information decoding unit 13, S1761 to S1793 in the process shown in FIG. 16 may be omitted.

(Modification example-2)
As described above, if the CU size is large, it may not be possible to perform appropriate division according to the characteristics of the image if the TU division is only one layer. Therefore, as the TU division mode of the CU, division into 8 × 8 size (8 × 8 division) may be used in addition to the rice field division, horizontal 4 division, or vertical 4 division. In this case, the division mode for the CU obtained by QT division is divided into no division, the paddy character division, the horizontal 4 division, the vertical 4 division and the 8x8 division, and the division mode for the CU obtained by the BT division is no division and Limited to 8x8 division.

FIG. 25 shows a list of TU division pattern examples corresponding to each shape of the CU. As shown in FIG. 25, the TU division mode includes 8 × 8 division up to the layer having a large CU size (D3 in this figure). In this case, the TT information decoding unit 13 decodes the TT information according to the processes shown in FIGS. 14 and 15, but derives the TU division mode according to the processes shown in FIG.

That is, when the TT information decoding unit 13 derives the TU division mode according to the process shown in FIG. 18, in the case of the CU obtained by the QT division, the TU division mode is no division, paddy character division, and horizontal 4. All steps may be taken to determine whether it is a split, a vertical 4 split, or an 8x8 split. On the other hand, in the case of the CU obtained by BT division, the TU division mode is limited to no division or 8 × 8 division, and the TU division mode is to determine whether it is no division or 8 × 8 division. , S1751, S1771, and S1791.

The division mode is not limited to 8 × 8 division. For example, division into 16 × 16 sizes (16 × 16 division) may be used. FIG. 26 shows a list of TU division pattern examples corresponding to each shape of the CU. As shown in FIG. 26, the TU division mode includes 16 × 16 division up to the layer having a large CU size (D1 in this figure).

Alternatively, both 8 × 8 division and 16 × 16 division may be used as the division mode. FIG. 27 shows a list of TU division pattern examples corresponding to each shape of the CU in this case. As shown in FIG. 27, the TU division mode includes 16 × 16 divisions up to a predetermined layer (D1 in this figure), and TU divisions in the subsequent layers (D2 and D3 in this figure). The mode includes 8x8 divisions.

(Third embodiment)
In intra-prediction, the appearance rate of small-sized CUs is high, so TUs tend to be small-sized. It is not rational to provide. Therefore, in the present embodiment, the TU division shown in the first embodiment is performed only when the prediction mode is inter-prediction, and the TU division is not performed when the prediction mode is intra-prediction. Specifically, when the prediction mode is inter-prediction, the hierarchy of TU division for CU is 0 (that is, no division) or 1 (that is, paddy character division, horizontal 4 division, and vertical 4 division are performed for 1 layer. ).

In the present embodiment, when the prediction mode is inter-prediction, the TU division pattern for the CU is limited to 0 or 1, so that the TU division pattern is shown in FIG. 13 according to the CU hierarchy and shape. I am limited to what I have.

In the present embodiment, when the prediction mode is inter-prediction, the TU division for the CU is limited to only one layer, and when the prediction mode is intra-prediction, the TU division for the CU is not performed. Coding efficiency can be maintained while reducing the complexity of conversion / decoding.

(Processing of TT information decoding)
When the prediction mode is inter-prediction, the operation of TT information decoding by the TT information decoding unit 13 when the layer of TU division for the CU is limited to only one layer will be described in detail with reference to FIG. 28. FIG. 28 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

The TT information decoding unit 13 decodes the TT information from the encoded data and decodes the conversion unit TU.

(S1513) First, the TT information decoding unit 13 determines whether or not the target TU size is equal to or less than the maximum TU size, larger than the minimum TU size, and the prediction mode is inter-prediction. The prediction mode (predMode) of each CU is given to the TT information decoding unit 13 together with the shape (CUwidth, CUheight) of each CU and the depth Depth (cqtDepth, cbtDepth) of each CU. The TT information decoding unit 13 determines that the prediction mode is intra-prediction when predMode = INTRA, and determines that the prediction mode is inter-prediction when predMode = INTER.

(S1521) The TT information decoding unit 13 derives the TU division flag when the above conditions are not satisfied, that is, when the TU division flag does not appear in the coded data. The derivation of this TU division flag will be described later.

Since the processes from S1522 to S1571 are the same as the processes shown in FIG. 14, the description thereof will be omitted here.

(TU division flag derivation process)
Hereinafter, the derivation of the TU division flag by the TT information decoding unit 13 will be described with reference to FIG. 29. FIG. 29 is a flowchart illustrating a schematic operation of the TT information decoding unit 13 according to the embodiment of the present invention.

Since the processes from S1611 to S1641 are the same as the processes shown in FIG. 15, the description thereof will be omitted here.

(S1681) The TT information decoding unit 13 determines whether or not the prediction mode of the target TU is inter-prediction.

(S1691) If the prediction mode of the target TU is determined to be inter-prediction, the TT information decoding unit 13 may perform TU division of the target TU, and therefore decodes the TU division flag from the coded data. (S1691).

(S1700) On the other hand, when the TT information decoding unit 13 determines that the prediction mode of the target TU is not inter-prediction (that is, intra-prediction), the TU division of the target TU is not performed, so that the TU division flag is flagged. Is derived as 0. That is, the TT information decoding unit 13 determines the shape of the CU (CUwidth, CUheight) as the shape of the TU one level below (TUwidth, TUheight).

The TU division performed when the prediction mode is inter-prediction is not limited to the above. For example, when the prediction mode is inter-prediction, the TU division shown in the modified example (FIG. 17, FIG. 19, FIG. 20) of the first embodiment may be performed, or the second embodiment (FIG. 21) may be performed. , The TU division shown in the modified example-1 (FIG. 24) and the modified example-2 (FIGS. 25 to 27) of the second embodiment may be performed.

(Fourth Embodiment)
Transformation (quantization, inverse transformation, inverse quantization) is performed on the transformation unit TU. This conversion is performed by the inverse quantization / inverse DCT unit 311.

FIG. 30 is a functional block diagram showing a configuration example of the inverse quantization / inverse DCT unit 311. As shown in FIG. 30, the inverse quantization / inverse DCT unit 311 includes an inverse quantization unit 311-1 and an inverse DCT unit 311-2. The dequantization unit 311-1 dequantizes the quantization conversion coefficient qd [] [] decoded by the TU decoding unit 22 in the TT information decoding unit 13 to derive the conversion coefficient d [] []. The inverse quantization unit 311-1 transmits the derived conversion coefficient d [] [] to the inverse DCT unit 311-2.

The inverse DCT unit 311-2 reversely transforms the received conversion coefficient d [] [] to restore the predicted residual r [] [] (also expressed as D in FIG. 10). The restored predicted residual r [] [] is sent from the inverse DCT unit 311-2 to the adder 312.

The inverse DCT section 311-2 has an AMT section 31-11 and a secondary transform section 311-2, and the inverse DCT section 311-2 has an adaptive multiple core transform (AMT) and a secondary transform. Can be used.

Specifically, the secondary conversion unit 311-2 receives information indicating that the secondary conversion is performed from the TT information decoding unit 13, and the conversion coefficient d [] [] received from the inverse quantization unit 311-1 is used as the secondary. The modified conversion coefficient d [] [] is restored by conversion. The restored modified conversion coefficient d [] [] is transmitted from the secondary conversion unit 31-12 to the AMT unit 31-11. If the secondary conversion is not performed, the conversion coefficient d [] [] received by the secondary conversion unit 31-12 is transmitted from the secondary conversion unit 311-2 to the AMT unit 31-11 as it is.

It should be noted that there is no difference between the transformation and the inverse transformation in the processing other than the value of the coefficient that is the basis of the transformation. In the following description, the term "conversion" will be used instead of "reverse conversion" for the conversion process in the inverse DCT unit 311-2.

(AMT)
In the AMT unit 31-11, the conversion basis can be adaptively switched, and in the present specification, the conversion that can be switched by an explicit flag, index, prediction mode, or the like is referred to as AMT. The conversion used in AMT is a separate conversion consisting of a vertical conversion and a horizontal conversion. The conversion basis of this conversion is DCT2, DST7, DCT8, DST1, and DCT5.

Each transformation basis Tij (i, j = 0,1, ..., N-1) can be obtained as follows.
DCT2
Tij = w0 * (2 / N) ^1/2 * cos (π * i * (2j + 1) / 2N)
w0 = (2 / N) ^1/2 (i = 0)
1 (i ≠ 0)
DCT5
Tij = w0 * w1 * (2 / (2N-1)) ^1/2 * cos (2π * i * j / (2N-1))
w0 = (2 / N) ^1/2 (i = 0)
1 (i ≠ 0)
w1 = (2 / N) ^1/2 (j = 0)
1 (j ≠ 0)
DCT8
Tij = (4 / (2N + 1)) ^1/2 * cos (π * (2i + 1) * (2j + 1) / (4N + 2))
DST1
Tij = (2 / (N + 1)) ^1/2 * sin (π * (i + 1) * (j + 1) / (N + 1))
DST7
Tij = (4 / (2N + 1)) ^1/2 * sin (π * (2i + 1) * (j + 1) / (2N + 1))
In AMT, the AMT unit 31-11 switches the conversion basis independently for the vertical conversion and the horizontal conversion. The conversion that can be selected is not limited to the above, and another conversion (conversion basis) can be used.

In AMT, the AMT unit 31-11 refers to the AMT flag (amt_flag) decoded from the coded data, and switches between a fixed conversion that does not switch by index and a selective conversion that switches by index. When amt_flag = 0, the fixed conversion is used, and when amt_flag = 1, the selective conversion is used. In the fixed conversion (amt_flag = 0), the AMT unit 31-11 does not refer to the AMT index amt_idx and performs a fixed conversion. In the selective conversion (amt_flag = 1), the AMT unit 31-11 further decodes the AMT index amt_idx used for selection from the coded data, and switches the conversion according to the prediction mode predMode and the AMT index amt_idx. In this embodiment, an example of decoding the AMT flag in units of CU or TU and decoding the AMT index amt_idx in units of TU will be described, but the unit of decoding (switching) is not limited to this. The fixed conversion is DCT2 or DST7 for both the conventional horizontal and vertical conversions.

(Intra prediction)
In AMT, the AMT unit 31-11 selects from the plurality of conversion bases described above using the prediction mode predMode and the AMT index amt_idx. FIG. 31 (a) is a table showing transformation sets, and FIG. 31 (b) is a diagram showing basis functions of each transformation basis.

In the case of predMode = INTRA, that is, intra prediction, the AMT unit 31-11 selects the conversion set to be used from the three conversion sets. Each transmutation set has two transmutation bases. For example, conversion set 0 has two conversion bases, DST7 and DCT8, as a set.

In the AMT unit 31-11, the AMT index amt_idx (amt_idx_hor and amt_idx_ver) is used to specify which of these two conversion bases is to be used. amt_idx = amt_idx_ver << 1 + amt_idx_hor, and the conversion basis is selected based on this index value. For example, in conversion set 0, DST7 is selected if amt_idx = 0, and DCT8 is selected if amt_idx = 1.

The AMT unit 31-11 determines which conversion set to use among the three conversion sets (0, 1, 2) of the intra prediction with reference to the intra prediction mode. The reference table at this time is shown in FIG. FIG. 32 shows which conversion set is selected for the horizontal and vertical transformations of each of the 67 intra-prediction modes. Further, FIG. 33 is a schematic diagram showing 67 types of intra prediction modes. As shown in FIG. 33, there are 67 types (0 to 66) of intra prediction modes.

0, 1, and 2 in FIG. 32 represent conversion sets 0, 1, and 2, respectively. For example, when the intra prediction mode is 10, the horizontal conversion (Horizontal) value in the figure is 0, so that it can be used for horizontal conversion. Since the conversion set 0 is used and the value of the vertical conversion (Vertical) in the figure is also 0, the conversion set 0 is also used for the vertical conversion.

The AMT described above will be specifically described with reference to FIG. 34. The AMT unit 31-11 in FIG. 34 is an example of the AMT unit 31-11 in FIG. 30. The AMT unit 31-11 in FIG. 34 includes a conversion set derivation unit 3111 for deriving the index coreTrSetIdx of the conversion set to be used, a conversion basis selection unit 3112 for deriving the conversion basis using coreTrSetIdx, and a conversion coefficient d [] []. It is composed of a vertical conversion unit 3113 that performs vertical conversion and a horizontal conversion unit 3114 that performs horizontal conversion.

When predMode = INTRA, the conversion set derivation unit 3111 refers to the reference table and derives the index coreTrSetIdx of the conversion set to be used next by the intra prediction mode IntraPredMode. The conversion set derivation unit 3111 is given a prediction mode (predMode, IntraPredMode) of each TU.

The conversion basis selection unit 3112 selects the conversion basis to be used from the conversion set specified by coreTrSetIdx by using the AMT index amt_idx. The conversion basis selection unit 3112 derives the transformation matrix transMatrix [] [] indicated by the selected conversion basis. The transformation base selection unit 3112 notifies the vertical conversion unit 3113 and the horizontal conversion unit 3114 of the transformation matrix transMatrix [] [] derived for the vertical / horizontal conversion, respectively, and determines the conversion coefficient d [] [] as the predicted residual r. Convert to [] [].

The vertical conversion unit 3113 converts the conversion coefficient d [] [] into the intermediate value e [] [] by one-dimensional conversion in the vertical direction, and sends the intermediate value e [] [] to the intermediate value clip unit 3115. The median clip unit 3115 derives the median value g [] [] by clipping the median value e [] [] and sends it to the horizontal conversion unit 3114. The horizontal conversion unit 3114 converts the intermediate value g [] [] into the predicted residual r [] []. The predicted residual r [] [] is sent from the horizontal conversion unit 3114 to the adder 312.

(Inter prediction)
In the present embodiment, even in the case of predMode = INTER, that is, inter-prediction, the AMT unit 31-11 selects the conversion set to be used from the three conversion sets (0, 1, 2) as shown in FIG. do. Each transmutation set has two transmutation bases.

Also in the inter-prediction, the AMT unit 31-11 uses the AMT index amt_idx (amt_idx_hor and amt_idx_ver) to specify which of these two conversion bases to use. amt_idx = amt_idx_ver << 1 + amt_idx_hor, and the conversion basis is selected based on this index value.

In the present embodiment, the AMT unit 31-11 determines which conversion set to use among the three conversion sets (0, 1, 2) of the inter-prediction with reference to the shape of the target TU. The reference table at this time is shown in FIG. FIG. 35 shows which conversion set is selected for the horizontal transformation and the vertical transformation of each of the TU shapes shown in 1) to 5). 0, 1, and 2 in the figure represent conversion sets 0, 1, and 2, respectively. For example, when the shape of the TU is 3), the horizontal transformation (Horizontal) value in the figure is 2, so the conversion set 2 is used for the horizontal transformation, and the vertical transformation (Vertical) value in the figure is also 2. The conversion set 2 is also used for vertical conversion. In FIG. 35, a conversion set in which at least one of horizontal conversion or vertical conversion is different is used for the horizontally long shape 1), 2), the square shape 3), and the vertically long shape 4), 5). Be done. Further, as shown in FIG. 35, the horizontal / vertical transformation of the horizontally long shape and the vertically long shape may be symmetrical with each other. That is, the horizontal conversion of the horizontally long shape and the vertical conversion of the vertically long shape, and the vertical conversion of the horizontally long shape and the horizontal conversion of the vertically long shape may be equal.

Here, the shapes of the five types (shapes 1) to 5)) of the TU are shown in FIG. 2) landscape 2 represents a shape that is 3) horizontally longer than a square, 1) landscape 1 represents a shape that is 2) horizontally longer than 2). Further, 5) vertically long 1 represents a shape that is vertically longer than 3) a square, and 4) vertically long 2 represents a shape that is vertically longer than 5) vertically long 1.

The shapes 1), 2), 4) and 5) can be classified according to, for example, the aspect ratio. For example, if the aspect ratio (horizontal: vertical) is 2: 1, it is 2) horizontally long, if the aspect ratio is 4: 1 or more, it is 1) horizontally long, and if the aspect ratio is 1: 2, it is 5) vertically long. If the aspect ratio is 1: 4 or more, it is set to 2 and 4) vertically long. For example, the shape can be derived by the following equation.

if (TUwidth> 2 * TH height) TU shape = shape 1) Horizontal 1
else if (TUwidth> TH height) TU shape = shape 2) Horizontal 2
if (TUwidth == TH height) TU shape = shape 3) Square
if (2 * TUwidth <TH height) TU shape = shape 4) Vertical 1
else if (TUwidth <TH height) TU shape = shape 5) Vertical 2
In the inter-prediction, when predMode = INTER, the conversion set derivation unit 3111 refers to the reference table and derives the index coreTrSetIdx of the conversion set to be used next by the shape of TU. The conversion set derivation unit 3111 is given a prediction mode (predMode) and a shape (TUwidth, TUheight) of each TU.

The processing after deriving the index coreTrSetIdx of the conversion set is the same as the intra prediction.

As described above, in the present embodiment, the AMT unit 31-11 selects not only the AMT index but also the AMT index in the inter-prediction, as the conversion basis is selected by referring to the intra-prediction mode as well as the AMT index in the intra-prediction. Since the conversion basis is selected by referring to the shape of the TU, it becomes easier to select an appropriate conversion basis depending on the target TU. For example, if the TU has vertical edges, the vertical shape is likely to be selected, and if the TU has horizontal edges, the horizontal shape is likely to be selected. By selecting, it becomes easier to select a conversion basis suitable for the prediction error.

It should be noted that which conversion set is selected for the horizontal conversion and the vertical conversion of each of the five types of TU shapes is not limited to the example shown in FIG. 35. For example, the conversion set derivation unit 3111 may select a conversion set to be used for horizontal conversion and vertical conversion of each of the five types of TU shapes with reference to the reference table shown in FIG. 37. As shown in FIG. 37, different conversion sets are used for a part of the horizontally long shape 1) and a part of the vertically long shape 4) (Horizontal, Vertical) = (2,0) and (0). , 2), the same conversion set (Horizontal, Vertical) = (1, 1) may be used for another part of the horizontally elongated shape 2) and another part of the vertically elongated shape 5). ..

Further, the shape of the TU referred to by the conversion set derivation unit 3111 is not limited to five types. For example, the conversion set derivation unit 3111 may select a conversion set to be used for horizontal conversion and vertical conversion of each of the three types of TU shapes with reference to the reference table shown in FIG. 38. The three types of shapes are horizontally long, square, and vertically long. In this case, 1) horizontally long 1 and 2) horizontally long 2, that is, TUwidth> TUheight corresponds to horizontally long, and 3) the square becomes a square. 4) Vertical 1 and 5) Vertical 2, that is, TUwidth <TUheight corresponds to vertical.

Further, the conversion set derivation unit 3111 may select a conversion set to be used for the horizontal conversion and the vertical conversion of each of the shapes of the TU with reference to the reference table shown in FIG. In this figure, in the case of 3) square, the conversion set used for horizontal conversion and vertical conversion is the same conversion set as inter-prediction (conversion set 0). This is because the square shape inter-prediction does not have the large tilt characteristics seen in the vertical and horizontal shapes in the residual bias, so it is preferable to use the same conversion set, especially the symmetric conversion set 0. Because it is. It is clear from the uppermost figure of FIG. 31 (b) that DST7 and DCT8, which are the bases of the pair constituting the conversion set 0 ((a) of FIG. 31), have symmetry with each other. ..

In FIG. 39, as in FIG. 35, the horizontally long shape 1), 2), the square shape 3), and the vertically long shape 4), 5) are at least horizontal conversion or vertical conversion. One different conversion set is used.

(Modification-1)
In the inter-prediction, the conversion set derivation unit 3111 refers not only to the shape of the TU and the AMT index. For example, the conversion set derivation unit 3111 may derive the index of the conversion set by referring to the shape of the TU, the block size of the TU, and the AMT index. Here, the block size is TUwidth + TUheight. The reference table at this time is shown in FIG. FIG. 40 shows which conversion set is selected for the horizontal and vertical transformations of each combination of the five TU shapes and the four TU block sizes. 0, 1, and 2 in the figure represent conversion sets 0, 1, and 2, respectively. For example, if the shape of the TU is 2) and the block size of the TU is 8), the horizontal conversion (Horizontal) value in the figure is 1, so the conversion set 1 is used for the horizontal conversion, and the conversion set 1 in the figure is used. Since the value of Vertical is 0, conversion set 0 is also used for vertical conversion.

Here, FIG. 41 shows the relationship between the block size of TU shown in 6) to 9) and the shape of TU shown in 1) to 5). The shapes of the five types (1) to 5)) of TU are as shown in FIG. 36.

6) indicates that the block size of TU is 96 or more, 7) indicates that the block size of TU is 48 or more, and 8) indicates that the block size of TU is 24 or more. , 9) indicate that the block size of TU is less than 24. Size thresholds and methods are not limited to this.

A logarithmic size such as log2TUwidth + log2TUheight may be used to determine the block size. Further, the TU may be classified by using the area of the TU (the product of the width TUwidth of the TU and the height TUheight) instead of the sum of the width TUwidth and the height TUheight of the TU as the block size. If it is logarithmic, log2TUwidth + log2TUheight may be considered to correspond to the area.

In this way, in the inter-prediction, since the conversion basis is selected by referring not only to the AMT index and the shape of the TU but also to the block size of the TU, it becomes easy to select an appropriate conversion basis depending on the target TU.

It should be noted that which conversion set is selected for the horizontal conversion and the vertical conversion of each combination of the five types of TU shapes and the four types of TU block sizes is not limited to the example shown in FIG. 40. For example, the conversion set derivation unit 3111 refers to the reference table shown in FIG. 42 and selects a conversion set to be used for horizontal conversion and vertical conversion of each combination of five types of TU shapes and four types of TU block sizes. You may.

(Modification example-2)
In the inter-prediction, the conversion set derivation unit 3111 refers not only to the shape of the TU, the block size of the TU, and the AMT index. For example, the conversion set derivation unit 3111 may derive the index of the conversion set by referring to the shape of the TU, the long side of the TU, and the AMT index. The reference table at this time is shown in FIG. 43. FIG. 43 shows which transformation set is selected for the horizontal and vertical transformations of each combination of the five TU shapes and the long sides of the three TUs. 0, 1, and 2 in the figure represent conversion sets 0, 1, and 2, respectively. For example, if the shape of the TU is 1) and the long side of the TU is 7), the horizontal conversion (Horizontal) value in the figure is 2, so the conversion set 1 is used for the horizontal conversion, and the conversion set 1 in the figure is used. Since the value of Vertical is 0, conversion set 0 is also used for vertical conversion.

Here, FIG. 44 shows the relationship between the long sides of the three types (6) to 8)) of the TU and the shapes of the five types (1) to 5)) of the TU. The shapes of the five types (1) to 5)) of TU are as shown in FIG. 36.

6) indicates that the length of the long side of the TU is 64 or more, 7) indicates that the length of the long side of the TU is 32 or more, and 8) indicates the length of the long side of the TU. Indicates that is less than 32.

In this way, in the inter-prediction, since the conversion basis is selected by referring not only to the AMT index and the shape of the TU but also to the long side of the TU, it becomes easy to select an appropriate conversion basis depending on the target TU.

It should be noted that which conversion set is selected for the horizontal conversion and the vertical conversion of each combination of the shapes of the five types of TUs and the long sides of the three types of TUs is not limited to the example shown in FIG. 45. For example, the conversion set derivation unit 3111 refers to the reference table shown in FIG. 45 and selects a conversion set to be used for horizontal conversion and vertical conversion of each combination of the shapes of the five types of TUs and the long sides of the three types of TUs. You may.

In the inter-prediction, the conversion set derivation unit 3111 refers not only to the shape of the target TU but also to the shape of the target CU. In this case, in the modified example-1, the block size (CUwidth + CUheight) of the target CU may be referred to together, and in the modified example 2, the long side of the target CU may be referred to together.

(Fifth Embodiment)
As shown in FIG. 33, there are 65 types of direction prediction in the intra prediction. In intra-prediction, it is desirable to select a conversion basis suitable for the intra-prediction mode, but it does not need to be as accurate as directional prediction. Therefore, in the present embodiment, in the intra prediction mode, one of the horizontal transformation and the vertical transformation is performed, the other transformation is skipped, and only scaling (shift or multiplication) is performed (IDentity transform:). ID).

Specifically, in the present embodiment, the conversion set derivation unit 3111 refers to the intra prediction mode when predMode = INTRA, and uses an ID instead of the vertical conversion in the horizontal direction (intra prediction mode is 18). , In the vertical direction (intra prediction mode is 50), ID is used instead of horizontal conversion.

The operation of the conversion set derivation unit 3111 in this case will be described. As described above, in the case of predMode = INTRA, that is, intra prediction, the conversion set derivation unit 3111 selects the conversion set to be used from the three conversion sets. In the present embodiment, the conversion set derivation unit 3111 determines which conversion set to use among the three conversion sets (0, 1, 2) with reference to the intra prediction mode. The reference table at this time is shown in FIG.

FIG. 46 shows which conversion set is selected for the horizontal and vertical transformations of each of the 67 intra-prediction modes. 0, 1, and 2 in the figure represent conversion sets 0, 1, and 2, respectively.

As shown in this figure (the part surrounded by the solid line), when the intra prediction mode is 18, the conversion set derivation unit 3111 selects the conversion set 2 for the horizontal conversion and the ID for the vertical conversion. When the intra prediction mode is 50, the conversion set derivation unit 3111 selects the ID for the horizontal conversion and the conversion set 2 for the vertical conversion.

The conversion set derivation unit 3111 refers to the reference table and derives the index of the conversion set to be used next by the intra prediction mode. The processing after deriving the index of the conversion set is as shown in the fourth embodiment.

In the present embodiment, one of the horizontal conversion and the vertical conversion is performed based on the intra prediction mode, the other conversion is skipped, and only scaling is performed. Therefore, it is possible to maintain the coding efficiency while reducing the amount of calculation. can.

The intra prediction mode using the ID is not limited to the horizontal direction (intra prediction mode 18) and the vertical direction (intra prediction mode 50). For example, the conversion set derivation unit 3111 performs vertical conversion in the near direction in the horizontal direction (intra prediction mode 18) (for example, the intra prediction mode from the horizontal direction to ± 3 (= intra prediction mode 15 to 17, 19 to 21)). ID is used instead of horizontal conversion in the vicinity of the vertical direction (intra prediction mode 50) (for example, the intra prediction mode from the vertical direction to ± 3 (= intra prediction mode 47 to 49, 51 to 53)). ID may be used for.

As shown in FIG. 46 (the part surrounded by the dotted line), when the intra prediction mode is 15 to 17, 19 to 21, the conversion set derivation unit 3111 selects the conversion set 2 for the horizontal conversion and performs the vertical conversion. Selects an ID. When the intra prediction mode is 47 to 49, 51 to 53, the conversion set derivation unit 3111 selects the ID for the horizontal conversion and the conversion set 2 for the vertical conversion.

The intra prediction mode using the ID is not particularly limited. For example, the ID may be used instead of the vertical conversion in the intra prediction modes 15 to 21, and the ID may be used instead of the horizontal conversion in the intra prediction modes 47 to 53. You may use it. Further, the near direction in the horizontal direction is not limited to the intra prediction mode from the horizontal direction to ± 3, and the near direction in the vertical direction is not limited to the intra prediction mode from the vertical direction to ± 3. Further, the ID may be used instead of the vertical conversion only in the horizontal direction or the near direction in the horizontal direction, or the ID may be used instead of the horizontal conversion only in the vertical direction or the near direction in the vertical direction.

(Modification-1)
In intra prediction, the conversion set derivation unit 3111 performs horizontal conversion and horizontal conversion for some intra prediction modes only when the size of the target TU is smaller than a predetermined size (for example, 8 × 8 or less or 16 × 16 or less). One of the vertical conversions may be performed and the ID may be used for the other conversion. The operation of the conversion set derivation unit 3111 in this case will be described. FIG. 47 is a flowchart illustrating a schematic operation of the conversion set derivation unit 3111 according to the embodiment of the present invention.

(S1811) First, the conversion set derivation unit 3111 determines whether or not the size of the target TU is smaller than a predetermined size.

(S1821) When the conversion set derivation unit 3111 determines that the size of the target TU is smaller than a predetermined size, the conversion set derivation unit 3111 selects the conversion set using the reference table of FIG.

(S1822) On the other hand, when the conversion set derivation unit 3111 determines that the size of the target TU is equal to or larger than a predetermined size, the conversion set derivation unit 3111 selects the conversion set using the reference table of FIG.

(Modification example-2)
Among the intra prediction modes, the intra prediction modes estimated to be applied in the target TU are included in the MPM candidate (candModeList). On the other hand, the remaining modes excluding the intra prediction mode corresponding to the MPM candidate from the entire intra prediction mode are called non-MPM (RemIntraPredMode). The non-MPM is further divided into a first Gr (rem_selected_mode) and a second Gr (rem_non_selected_mode).

The conversion set derivation unit 3111 may perform one of the horizontal conversion and the vertical conversion for the intra prediction mode included in the second Gr (rem_non_selected_mode) of the non-MPM, and may use the ID for the other conversion. Specifically, in the present embodiment, when predMode = INTRA, the conversion set derivation unit 3111 refers to the intra prediction mode and the intra prediction parameters, and refers to the intra prediction mode included in the second Gr of the non-MPM. Perform one of the horizontal and vertical conversions, and use the ID for the other. The intra prediction parameters include candModeList, RemIntraPredMode, rem_selected_mode, rem_non_selected_mode, and the like.

The operation of the conversion set derivation unit 3111 in this case will be described. As described above, in the case of predMode = INTRA, that is, intra prediction, the conversion set derivation unit 3111 selects the conversion set to be used from the three conversion sets. In the present embodiment, the conversion set derivation unit 3111 determines which conversion set to use among the three conversion sets (0, 1, 2) with reference to the intra prediction mode and the intra prediction parameters. The reference table at this time is shown in FIG.

FIG. 48 shows which transformation set the horizontal and vertical transformations of each intra-prediction mode contained in the MPM candidate or non-MPM 1st Gr or 2nd Gr select. In the figure, 0, 1, and 2 represent conversion sets 0, 1, and 2, respectively, the numbers on the left side represent the conversion sets for horizontal conversion, and the numbers on the right side represent the conversion set for vertical conversion.

As shown in this figure (the part surrounded by the solid line), for the intra prediction mode included in the second Gr of the non-MPM, the conversion set derivation unit 3111 has the conversion set 0 for one of the horizontal conversion and the vertical conversion. Select either 1 or 2 and select the ID for the other conversion.

The conversion set derivation unit 3111 refers to the reference table and derives the index of the conversion set to be used next by the intra prediction mode. The processing after deriving the index of the conversion set is as shown in the fourth embodiment.

The intra prediction mode using the ID is not limited to all the intra prediction modes included in the second Gr of the non-MPM. For example, the conversion set derivation unit 3111 may perform one of the horizontal conversion and the vertical conversion for a part of the intra prediction mode included in the second Gr of the non-MPM, and may use the ID for the other conversion.

As shown in FIG. 48 (the part surrounded by the dotted line), only two of the three second Gr intra prediction modes included between the two adjacent first Gr intra prediction modes are converted. The set derivation unit 3111 performs horizontal conversion and vertical conversion, and uses an ID for the other conversion.

The intra prediction mode using the ID is not limited to all or two intra prediction modes included in the second Gr of the non-MPM. For example, for one intra prediction mode included in the second Gr of non-MPM, one of the horizontal conversion and the vertical conversion may be performed, and the ID may be used for the other conversion.

Also in this modification, the conversion set derivation unit 3111 is included in the non-MPM second Gr only when the size of the target TU is smaller than the predetermined size (for example, 8 × 8 or less or 16 × 16 or less). For the prediction mode, one of the horizontal conversion and the vertical conversion may be performed, and the ID may be used for the other conversion. In this case as well, the conversion set derivation unit 3111 operates according to the flowchart shown in FIG. 47, but in S1821, the conversion set is selected using the reference table of FIG. 48.

Since it is determined whether or not to skip the one-way conversion by the intra prediction mode, it is not necessary to encode / decode the flag as to whether or not to use the ID. Further, as shown in FIGS. 46 and 48, it is determined whether the ID is used in the horizontal conversion or the vertical conversion for each intra prediction mode. Therefore, when the ID is applied to the horizontal conversion, the AMT index amt_idx Only amt_idx_ver needs to be specified in (amt_idx_ver = amt_idx), and amt_idx_ver encoding / decoding is not required. On the other hand, when applying the ID to the vertical conversion, only amt_idx_hor needs to be specified by the AMT index amt_idx (amt_idx_hor = amt_idx), and the encoding / decoding of amt_idx_hor is unnecessary.

(Sixth Embodiment)
With a rectangular TU, if you select the conversion in the appropriate direction from the horizontal conversion and the vertical conversion, the energy is concentrated on the low frequency term even with the first separation type conversion, so the second separation type conversion is unnecessary. Become. Therefore, in the present embodiment, one of the horizontal conversion and the vertical conversion is performed based on the shape of the target TU, the other conversion is skipped, and only scaling (shift or multiplication) is performed (ID).

Specifically, in the present embodiment, the conversion set derivation unit 3111 refers to the shape of the TU, and if it is a horizontally long rectangular TU, uses an ID instead of the vertical conversion, and the vertically long rectangular TU is used. If, use ID instead of horizontal conversion.

The operation of the conversion set derivation unit 3111 in this case will be described. As described above, the conversion set derivation unit 3111 selects the conversion set to be used from the three conversion sets. In the present embodiment, the conversion set derivation unit 3111 determines which conversion set to use among the three conversion sets (0, 1, 2) with reference to the shape of the TU. The reference table at this time is shown in FIG.

FIG. 49 shows which conversion set is selected for the horizontal and vertical transformations of each of the five TU shapes. 0, 1, and 2 in the figure represent conversion sets 0, 1, and 2, respectively. The shapes of the five types (1) to 5)) of TU are as shown in FIG. 35.

As shown in this figure, when the shape of the TU is 1) horizontally long 1 or 2) horizontally long 2, the conversion set derivation unit 3111 selects conversion set 2 for horizontal conversion and ID for vertical conversion. When the shape of the TU is 4) vertically long 1 or 5) vertically long 2, the conversion set derivation unit 3111 selects an ID for horizontal conversion and a conversion set 2 for vertical conversion.

The conversion set derivation unit 3111 refers to the reference table and derives the index of the conversion set to be used next by the shape of the TU. The processing after deriving the index of the conversion set is as shown in the fourth embodiment.

In the present embodiment, one of the horizontal conversion and the vertical conversion is performed based on the shape of the target TU, the other conversion is skipped, and only scaling is performed. Therefore, the coding efficiency is maintained while reducing the amount of calculation. Can be done.

Further, since it is determined whether or not to skip the one-way conversion depending on the shape of the TU, it is not necessary to encode / decode the flag as to whether or not to use the ID. Furthermore, as shown in FIG. 49, it is determined whether the ID is used for horizontal conversion or vertical conversion for each shape of the target TU, so when applying the ID to horizontal conversion, it is specified by the AMT index amt_idx. It only needs to do amt_idx_ver (amt_idx_ver = amt_idx) and does not need to encode / decode amt_idx_ver. On the other hand, when applying the ID to the vertical conversion, only amt_idx_hor needs to be specified by the AMT index amt_idx (amt_idx_hor = amt_idx), and the encoding / decoding of amt_idx_hor is unnecessary.

It should be noted that which conversion set is selected for the horizontal conversion and the vertical conversion of each of the five types of TU shapes is not limited to the example shown in FIG. 49. For example, the conversion set derivation unit 3111 may select a conversion set to be used for horizontal conversion and vertical conversion of each of the five types of TU shapes with reference to the reference table shown in FIG. In this figure, the conversion set derivation unit 3111 selects the conversion set 2 for the horizontal conversion and the ID for the vertical conversion only when the shape of the TU is 1) horizontally long 1. Further, only when the shape of the TU is 4) vertically long 1, the conversion set derivation unit 3111 selects the ID for the horizontal conversion and the conversion set 2 for the vertical conversion.

(Modification-1)
For a rectangular TU, one of the horizontal conversion and the vertical conversion may be performed, and the ID may be used for the other conversion when the block size of the TU is equal to or less than the threshold value. Specifically, the conversion set derivation unit 3111 refers to the shape of the TU and the block size of the TU, and if the block size is equal to or less than the threshold value, if the TU is a horizontally long rectangular shape, the ID is used instead of the vertical conversion. And if it is a vertically long rectangular TU, use ID instead of horizontal conversion.

The reference table at this time is shown in FIG. FIG. 51 shows which conversion set is selected for the horizontal and vertical transformations of each combination of the five TU shapes and the four TU block sizes. The shapes of the five types (1) to 5)) of TU are as shown in FIG. 36, and the block sizes of the four types (6) to 9)) of TU are as shown in FIG. 41. Is.

As shown in this figure, if the block size of TU is a threshold value (9) less than 24 in this figure), if the shape of TU is 1) horizontally long 1, the conversion set derivation unit 3111 converts to horizontal conversion. Select set 2 and select ID for vertical conversion. If the block size of the TU is a threshold value (9) less than 24 in this figure), and if the shape of the TU is 4) vertically long 1, the conversion set derivation unit 3111 selects an ID for horizontal conversion and is vertical. Select conversion set 2 for conversion.

(Modification example-2)
For a rectangular TU, one of the horizontal conversion and the vertical conversion may be performed, and the ID may be used for the other conversion when the length of the long side of the TU is equal to or less than the threshold value. Specifically, the conversion set derivation unit 3111 refers to the shape of the TU and the long side of the TU, and if the length of the long side is equal to or less than the threshold value, the conversion set derivation unit 3111 performs vertical conversion in the case of a horizontally long rectangular TU. Use ID instead, and if it is a vertically long rectangular TU, use ID instead of horizontal conversion.

The reference table at this time is shown in FIG. FIG. 52 shows which conversion set is selected for the horizontal and vertical conversions of each combination of the shapes of the five TUs and the long sides of the three types of TUs. The shapes of the five types of TUs (1) to 5)) are as shown in FIG. 36, and the long sides of the four types of TUs (6) to 8)) are as shown in FIG. Is.

As shown in this figure, if the long side of the TU is a threshold value (8) less than 32 in this figure), and if the shape of the TU is 1) horizontally long 1 or 2) horizontally long 2, the conversion set derivation unit 3111 Select conversion set 2 for horizontal conversion and select ID for vertical conversion. If the block size of the TU is a threshold value (8) less than 32 in this figure) and the shape of the TU is 4) vertically long 1 or 5) vertically long 2, the conversion set derivation unit 3111 has an ID for horizontal conversion. And select conversion set 2 for vertical conversion.

(Modification 3)
One of the horizontal conversion and the vertical conversion shown above may be performed, and the correspondence using the ID for the other conversion may be performed in the case of inter-prediction. Specifically, the conversion set derivation unit 3111 refers to the shape of the TU when predMode = INTER, and uses an ID instead of the vertical conversion when it is a horizontally long rectangular TU, and has a vertically long rectangular shape. If it is TU, use ID instead of horizontal conversion. Alternatively, when predMode = INTER, the conversion set derivation unit 3111 refers to the shape of the TU and the long side of the TU, and if the length of the long side is equal to or less than the threshold value, the conversion set derivation unit 3111 is a horizontally long rectangular TU. Uses ID instead of vertical conversion, and uses ID instead of horizontal conversion in the case of a vertically long rectangular TU.

The operation of the conversion set derivation unit 3111 in this case will be described. FIG. 53 is a flowchart illustrating a schematic operation of the conversion set derivation unit 3111 according to the embodiment of the present invention.

(S1831) First, the conversion set derivation unit 3111 determines whether or not the prediction mode is inter-prediction for the target TU.

(S1841) When the conversion set derivation unit 3111 determines that the prediction mode is inter-prediction, the conversion set derivation unit 3111 selects the conversion set using the reference table of FIG.

(S1842) On the other hand, when the conversion set derivation unit 3111 determines that the prediction mode is not inter-prediction, that is, intra-prediction, the conversion set is selected using the reference table of FIG.

In S1841, the conversion set may be selected by using any of the reference tables of FIGS. 50 to 52 instead of FIG. 49. Further, in S1842, the conversion set may be selected by using the reference table of FIG. 48 instead of FIG. 46.

(Modification example-4)
When the shape of the target TU is 1) horizontally long 1 or 4) vertically long 1 shown in FIG. 36, an ID may be used for both horizontal conversion and vertical conversion. That is, when the shape of the target TU is 1) horizontally long 1 or 4) vertically long 1 shown in FIG. 36, AMT is not performed and only secondary conversion is performed in the inverse DCT unit 311-2.

The operation of the conversion set derivation unit 3111 in this case will be described. FIG. 54 is a flowchart illustrating the schematic operation of the conversion set derivation unit 3111 according to the embodiment of the present invention.

(S1851) First, the conversion set derivation unit 3111 determines whether or not the ratio (width / height) of the sizes of the target TUs is equal to or greater than a predetermined value (for example, TUwidth / TUheight> = 4).

(S1861) When the conversion set derivation unit 3111 determines that the ratio (width / height) of the size of the target TU is less than the predetermined value, the ratio (height / width) of the size of the target TU is equal to or more than the predetermined value. (For example, TUheight / TUwidth> = 4) is determined.

(S1871) When the conversion set derivation unit 3111 determines that the size (height / width) of the target TU is less than a predetermined value, the conversion set is selected using the reference table of FIG. 35.

(S1872) On the other hand, in the conversion set derivation unit 3111, the size (width / height) of the target TU is equal to or larger than a predetermined value (that is, 1) horizontally 1 64x16, 64x8, 64x4, 32x8, 32x4, 16x4). Or, if it is determined that the size (height / width) of the target TU is equal to or larger than the predetermined value (that is, 4) 16x64, 8x64, 4x64, 8x32, 4x32, 4x16) which is vertically long 1, horizontal conversion and vertical conversion are performed. ID is used for both conversions.

In S1871, the conversion set may be selected by using the reference table of FIG. 37, FIG. 40, FIG. 43, and FIGS. 49 to 52 instead of FIG. 35.

(Example of realization by software)
In addition, a part of the image coding device 11 and the image decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the prediction parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, the inverse quantization / inverse DCT. Unit 311, addition unit 312, prediction image generation unit 101, subtraction unit 102, DCT / quantization unit 103, entropy coding unit 104, dequantization / inverse DCT unit 105, loop filter 107, coding parameter determination unit 110, The prediction parameter coding unit 111 may be realized by a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The "computer system" referred to here is a computer system built in either the image coding device 11 or the image decoding device 31, and includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In that case, a program may be held for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Further, a part or all of the image coding device 11 and the image decoding device 31 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image coding device 11 and the image decoding device 31 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

(Application example)
The image coding device 11 and the image decoding device 31 described above can be mounted on and used in various devices for transmitting, receiving, recording, and reproducing moving images. The moving image may be a natural moving image captured by a camera or the like, or an artificial moving image (including CG and GUI) generated by a computer or the like.

First, it will be described with reference to FIG. 8 that the above-mentioned image coding device 11 and image decoding device 31 can be used for transmission and reception of moving images.

FIG. 8A is a block diagram showing a configuration of a transmission device PROD_A equipped with an image coding device 11. As shown in FIG. 8A, the transmission device PROD_A modulates the carrier wave with the coding unit PROD_A1 that obtains the coded data by coding the moving image and the coded data obtained by the coding unit PROD_A1. It includes a modulation unit PROD_A2 for obtaining a modulation signal, and a transmission unit PROD_A3 for transmitting the modulation signal obtained by the modulation unit PROD_A2. The image coding device 11 described above is used as the coding unit PROD_A1.

The transmitter PROD_A has a camera PROD_A4 for capturing a moving image, a recording medium PROD_A5 for recording a moving image, an input terminal PROD_A6 for inputting a moving image from the outside, and a moving image as a source of the moving image to be input to the coding unit PROD_A1. , An image processing unit A7 for generating or processing an image may be further provided. In FIG. 8A, all of these are illustrated by the transmitter PROD_A, but some of them may be omitted.

The recording medium PROD_A5 may be a recording of an unencoded moving image, or a moving image encoded by a recording coding method different from the transmission coding method. It may be a thing. In the latter case, a decoding unit (not shown) that decodes the coded data read from the recording medium PROD_A5 according to the coding method for recording may be interposed between the recording medium PROD_A5 and the coding unit PROD_A1.

FIG. 8B is a block diagram showing a configuration of a receiving device PROD_B equipped with an image decoding device 31. As shown in FIG. 8B, the receiving device PROD_B has a receiving unit PROD_B1 that receives a modulated signal, a demodulating unit PROD_B2 that obtains coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and demodulation unit PROD_B2. It includes a decoding unit PROD_B3 that obtains a moving image by decoding the coded data obtained by the unit PROD_B2. The image decoding device 31 described above is used as the decoding unit PROD_B3.

The receiving device PROD_B is a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording a moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. It may also have PROD_B6. In FIG. 8B, a configuration in which the receiving device PROD_B includes all of them is illustrated, but a part thereof may be omitted.

The recording medium PROD_B5 may be used for recording an unencoded moving image, or may be encoded by a recording coding method different from the transmission coding method. You may. In the latter case, it is advisable to interpose a coding unit (not shown) that encodes the moving image acquired from the decoding unit PROD_B3 according to the coding method for recording between the decoding unit PROD_B3 and the recording medium PROD_B5.

The transmission medium for transmitting the modulated signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the destination is not specified in advance) or communication (here, transmission in which the destination is specified in advance). It may refer to an aspect). That is, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of terrestrial digital broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by radio broadcasting. Further, a broadcasting station (broadcasting equipment, etc.) / receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

In addition, servers (workstations, etc.) / clients (television receivers, personal computers, smartphones, etc.) such as VOD (Video On Demand) services and video sharing services using the Internet are transmitters that send and receive modulated signals via communication. This is an example of PROD_A / receiver PROD_B (usually, in LAN, either wireless or wired is used as a transmission medium, and in WAN, wired is used as a transmission medium). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multifunctional mobile phone terminals.

The client of the video sharing service has a function of decoding the encoded data downloaded from the server and displaying it on the display, as well as a function of encoding the moving image captured by the camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, it will be described with reference to FIG. 9 that the above-mentioned image coding device 11 and image decoding device 31 can be used for recording and reproducing moving images.

FIG. 9A is a block diagram showing a configuration of a recording device PROD_C equipped with the above-mentioned image coding device 11. As shown in FIG. 9A, the recording device PROD_C uses the coding unit PROD_C1 to obtain the coded data by coding the moving image and the coded data obtained by the coding unit PROD_C1 to the recording medium PROD_M. It has a writing unit PROD_C2 for writing. The image coding device 11 described above is used as the coding unit PROD_C1.

The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of a type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration). It may be loaded into a drive device (not shown) built in the recording device PROD_C, such as (trademark).

Further, the recording device PROD_C has a camera PROD_C3 that captures a moving image, an input terminal PROD_C4 for inputting a moving image from the outside, and a reception for receiving the moving image as a source of the moving image to be input to the coding unit PROD_C1. The unit PROD_C5 and the image processing unit PROD_C6 for generating or processing an image may be further provided. In FIG. 9A, all of these are illustrated by the recording device PROD_C, but some of them may be omitted.

The receiving unit PROD_C5 may receive an unencoded moving image, or receives coded data encoded by a coding method for transmission different from the coding method for recording. It may be something to do. In the latter case, a transmission decoding unit (not shown) that decodes the coded data encoded by the transmission coding method may be interposed between the receiving unit PROD_C5 and the coding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, and the like (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of moving images). .. In addition, a camcorder (in this case, the camera PROD_C3 is the main source of the moving image), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is the main source of the moving image), a smartphone (this). In this case, the camera PROD_C3 or the receiver PROD_C5 is the main source of moving images) is also an example of such a recording device PROD_C.

FIG. 9B is a block showing the configuration of the reproduction device PROD_D equipped with the above-mentioned image decoding device 31. As shown in FIG. 9B, the reproduction device PROD_D obtains a moving image by decoding the coded data read by the reading unit PROD_D1 and the reading unit PROD_D1 that reads the coded data written in the recording medium PROD_M. It has a decoding unit PROD_D2 to obtain. The image decoding device 31 described above is used as the decoding unit PROD_D2.

The recording medium PROD_M may be of a type built in the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory. It may be of a type connected to the playback device PROD_D, or may be loaded into a drive device (not shown) built in the playback device PROD_D, such as (3) DVD or BD. good.

Further, the playback device PROD_D has a display PROD_D3 for displaying a moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. It may also have PROD_D5. In FIG. 9B, all of these are illustrated by the reproduction device PROD_D, but some of them may be omitted.

The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits coded data encoded by a transmission coding method different from the recording coding method. It may be something to do. In the latter case, it is preferable to interpose a coding unit (not shown) that encodes the moving image by a coding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

Examples of such a playback device PROD_D include a DVD player, a BD player, an HDD player, and the like (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is the main supply destination of the moving image). .. In addition, a television receiver (in this case, display PROD_D3 is the main supply destination of moving images) and digital signage (also called electronic signage or electronic bulletin board, etc., and display PROD_D3 or transmitter PROD_D5 is the main supply destination of moving images. (Before), desktop PC (in this case, output terminal PROD_D4 or transmitter PROD_D5 is the main supplier of moving images), laptop or tablet PC (in this case, display PROD_D3 or transmitter PROD_D5 is video). An example of such a playback device PROD_D is a smartphone (in this case, the display PROD_D3 or the transmitter PROD_D5 is the main supply destination of the moving image), which is the main supply destination of the image.

(Hardware realization and software realization)
Further, each block of the image decoding device 31 and the image coding device 11 described above may be realized by hardware by a logic circuit formed on an integrated circuit (IC chip), or may be realized by a CPU (Central Processing Unit). It may be realized by software using.

In the latter case, each of the above devices is a CPU that executes an instruction of a program that realizes each function, a ROM (Read Only Memory) that stores the above program, a RAM (RandomAccess Memory) that expands the above program, the above program, and various data. It is equipped with a storage device (recording medium) such as a memory for storing the data. Then, an object of the embodiment of the present invention is a recording in which the program code (execution format program, intermediate code program, source program) of the control program of each of the above-mentioned devices, which is software for realizing the above-mentioned function, is readablely recorded by a computer. It can also be achieved by supplying the medium to each of the above devices and having the computer (or CPU or MPU) read and execute the program code recorded on the recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic discs such as floppy (registered trademark) discs / hard disks, and CD-ROMs (Compact Disc Read-Only Memory) / MO discs (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc) / CD-R (CD Recordable) / Blu-ray Disc (registered trademark) and other optical discs, IC cards (including memory cards) / Cards such as optical cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory: registered trademark) / Semiconductor memories such as flash ROM, or PLD (Programmable logic device) ) And logic circuits such as FPGA (Field Programmable Gate Array) can be used.

Further, each of the above devices may be configured to be connectable to a communication network, and the above program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit the program code. For example, Internet, Intranet, Extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television / Cable Television) communication network, Virtual Private network (Virtual Private) Network), telephone line network, mobile communication network, satellite communication network, etc. can be used. Further, the transmission medium constituting this communication network may be any medium as long as it can transmit the program code, and is not limited to a specific configuration or type. For example, even wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared ray such as IrDA (Infrared Data Association) and remote control. , BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It can also be used wirelessly. The embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave, in which the program code is embodied by electronic transmission.

The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the claims. That is, an embodiment obtained by combining technical means appropriately modified within the scope of the claims is also included in the technical scope of the present invention.
(Mutual reference of related applications)
This application claims the benefit of priority to the Japanese patent application filed on August 26, 2016: Japanese Patent Application No. 2016-166318, and by reference to it, all of its contents Included in this book.

The embodiment of the present invention can be suitably applied to an image decoding device that decodes coded data in which image data is encoded, and an image coding device that generates coded data in which image data is encoded. can. Further, it can be suitably applied to the data structure of the coded data generated by the image coding device and referenced by the image decoding device.

11 Image coding device 13 TT information decoding unit 22 TU decoding unit 31 Image decoding device 41 Image display device

Claims (1)

In an image decoder that decodes a coding unit (CU)
A CU decoding unit that decodes the parameters and conversion coefficients used for prediction,
An inverse quantization / inverse conversion unit that generates a prediction error by inversely quantizing and inversely converting the conversion coefficient for each conversion unit (TU).
A predictive image generator that generates a predictive image from predictive parameters,
It is equipped with an adder that adds the prediction error and the predicted image to generate a decoded image.
The CU decoding unit determines whether the target TU size is equal to or less than the maximum TU size, is larger than the minimum TU size, and the prediction mode is inter-prediction, and based on the above determination, the TU Decrypt the flag indicating whether to convert in small units,
When the flag indicates conversion in units smaller than TU, it is characterized by decoding the mode indicating the division direction, dividing the TU in the direction indicated by the mode, and converting an area having a size of 1/4 of TU. Image decoding device.

Images