JP2014013975A - Image decoding device, data structure of encoded data, and image encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a processing amount of processing for estimating a division pattern of a target CU from a reference layer in a hierarchical encoding system.SOLUTION: A hierarchical moving image decoding device 1 comprises: a reference CU setting unit 1481 that sets a reference CU; and a PU division estimation unit 1482 that determines a PU division type for a target CU in accordance with a degree of overlapping of a reference area and the reference CU.

Description

  The present invention relates to an image decoding device that decodes hierarchically encoded data in which an image is hierarchically encoded, an image encoding device that generates hierarchically encoded data by hierarchically encoding an image, and a hierarchical code Relates to the data structure of the data.

  One of information transmitted in the communication system or information recorded in the storage device is an image or a moving image. 2. Description of the Related Art Conventionally, a technique for encoding an image for transmitting and storing these images (hereinafter including moving images) is known.

  As a moving image encoding method, H.264 is used. H.264 / MPEG-4. AVC and HEVC (High-Efficiency Video Coding) which is a successor codec are known (Non-Patent Document 1).

  In such a moving image coding system, an image (picture) constituting a moving image is a slice obtained by dividing the image, a coding tree block obtained by dividing the slice (hereinafter referred to as CTB (Coding)). Tree Block))), which is managed and encoded / decoded by a hierarchical structure composed of coding units (sometimes called coding units) obtained by recursively dividing a CBT into quadtrees. The

  The coding unit (hereinafter referred to as CU) is further divided into a transform unit for managing transform coefficient processing and a prediction unit for managing predicted image processing as appropriate.

  In these moving image encoding methods, a prediction image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and the prediction image is subtracted from the input image (original image). The prediction residual obtained in this way (sometimes referred to as “difference image” or “residual image”) is orthogonally transformed and quantized and encoded.

  Examples of the predicted image generation method include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).

  For inter prediction, a predicted image is generated by motion compensation between frames. On the other hand, in intra prediction, based on locally decoded images in the same frame, predicted images in the frame are sequentially generated.

  In HEVC, a prediction unit size for motion compensation in inter prediction is expressed by a combination of division by CU and division type signaled for each CU.

  In recent years, a hierarchical encoding technique for hierarchically encoding an image according to a necessary data rate has been proposed.

  Hierarchical coding methods include ISO / IEC and ITU-T standards as H.264 standards. H.264 / AVC Annex G Scalable Video Coding (SVC).

  SVC supports spatial scalability, temporal scalability, and SNR scalability. For example, in the case of spatial scalability, first, an image obtained by down-sampling an original image to a desired resolution is used as a lower layer. It is encoded with H.264 / AVC. Note that a lower layer referred to in a target layer to be decoded is also referred to as a reference layer. Next, in the target layer, inter-layer prediction is performed in order to remove redundancy between layers.

  As the inter-layer prediction, there is motion information prediction in which information related to motion prediction is predicted from information in a reference layer at the same time, or texture prediction in which prediction is performed from an image obtained by up-sampling a decoded image of a reference layer at the same time (non-patent document 2). In the motion information prediction, motion information is encoded using motion information (motion vector or the like) of a reference layer as an estimated value.

  In the SVC, when the resolution is changed between layers, the size of the target block that is the target of the prediction process in the target layer is estimated according to whether the motion vectors of the reference layer are the same.

  In the SVC inter-layer prediction unit estimation process, a motion vector mv, a reference index refIdx, a macroblock type mbType, and a sub macroblock type subMbType are derived by estimation.

  In addition, the flow of the process of estimating these parameters will be described with reference to FIG.

  First, the partition position on the reference layer corresponding to the target macroblock (MB) is derived. Here, the partition is a sub macroblock (subMB) obtained by dividing the MB.

  Subsequently, with reference to mv and refIdx in the reference layer, estimated values of mv and refIdx are derived for each processing unit in the target layer. That is, the estimated values of mv and refIdx are derived for each minimum processing unit included in the target MB shown in FIG. Thereby, as shown in FIG. 47B, estimated values of mv and refIdx are derived in each processing unit. A 4 × 4 rectangle shown in FIG. 47B indicates a processing unit, and an arrow indicates a motion vector.

  Based on the estimated values of mv and refIdx in the target MB, partition division (subMB division) for dividing the MB into subMBs is derived. That is, partition division is determined so that mv and refIdx estimation values in the same partition are the same. In the partition R101 shown in FIG. 47B, mv (refIdx) is the same in each processing unit, and mv (refIdx) is the same in each processing unit in the partition R102. Therefore, as shown in FIG. 47 (c), the macroblock MB100 is divided into a submacroblock subMB101 corresponding to the partition R101 and a submacroblock subMB102 corresponding to the partition R102.

  Further, for each subMB in the target MB, subMB partition division for further dividing subMB is derived based on the estimated values of mv and refIdx. That is, as in the case of partitioning, sub partitioning is determined so that the mv and refIdx estimation values in the same partition are the same. The flow of sub partitioning is the same as that of the macro block MB100 shown in FIG. 47, and the macro block MB100 shown in FIG. 47 may be read as a sub macro block.

"High efficiency video coding (HEVC) text specification draft 7 (JCTVC-I1003_d1)", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 9th Meeting: Geneva, CH, 27 April-7 May 2012 (released in April 2012) ITU-T H.264 “Advanced video coding for generic audiovisual services” (published in November 2007)

  However, in the inter-layer prediction unit estimation process as described above, the estimated amount of partition division is derived after deriving the estimated motion vector in all the processing units included in the target MB, and thus the processing amount is large.

  In SVC, MB is composed of 16 × 16 pixels, but in HEVC, CTB corresponding to MB may be 64 × 64 pixels at the maximum. Thus, when the CTB has a size larger than that of the conventional MB, it is necessary to compare a large number of mvs in the partitioning method as described above.

  That is, when inter-layer prediction unit estimation processing is performed in HEVC, if a method similar to SVC is used, it is necessary to determine whether many motion vectors match, and the prediction unit of the target layer is estimated. Become more.

  The present invention has been made in view of the above problems, and an object of the present invention is to perform image decoding that can reduce the amount of processing for estimating a division pattern of a target CU from a reference layer in a hierarchical coding scheme. It is to realize an apparatus or the like.

  In order to solve the above problems, an image decoding apparatus according to the present invention decodes hierarchically encoded data in which image information relating to images of different quality for each layer is hierarchically encoded, and is a target to be decoded. An image decoding apparatus that restores an image in a layer, wherein a target coding unit that is a coding target to be decoded in the target layer is a region corresponding to the target coding unit in a reference layer that is a decoded layer. A reference coding unit setting means for setting a reference coding unit to determine a degree of overlap with the reference area based on a coding unit having an overlapping part with a certain reference area, the reference area and the reference coding A division pattern decision that determines a division pattern into prediction units that are prediction image processing units for the target coding unit according to the degree of overlap with the unit. Characterized in that it comprises a means.

  The image decoding device is a so-called hierarchical image decoding device that decodes hierarchically encoded data in which image information relating to images of different quality for each layer is hierarchically encoded, and restores the image.

  Such a hierarchical coding scheme is sometimes referred to as (SVC; Scalable Video Coding), and is standardized in, for example, H.264 / AVC Annex G SVC. In addition, the quality of a moving image here means a factor that affects the appearance of a subjective and objective moving image. The quality of the moving image includes, for example, “resolution”, “frame rate”, and “image quality”. Hereinafter, a layer corresponding to higher quality is referred to as a higher layer, and a layer corresponding to lower quality is referred to as a lower layer.

  In the above configuration, the target layer refers to a layer that is a decoding target.

  According to the above configuration, the reference layer that is the decoded layer is referred to in the target prediction unit that is the processing target of the predicted image in the target layer.

  In the hierarchical coding method, a layer in which image information has been decoded may be referred to when decoding a target layer. As described above, a layer referred to when decoding a target layer is referred to as a reference layer. The reference layer is generally a lower layer of the target layer. However, it is not necessary to refer to all layers lower than the target layer, and at least a lower layer in which information necessary for decoding in the target layer is decoded may be referred to.

  Further, in the moving picture coding scheme such as HEVC as described above, generally, a coding unit that is a processing unit of coding is further divided into prediction units that are processing units of a predicted image. The division pattern here includes “no division”, that is, the case where the encoding unit and the prediction unit have the same size.

  That the target coding unit corresponds to the region in the reference layer is that the spatial position of the target coding unit in the target image (frame) on the target layer is the spatial position of the reference region in the reference image (frame) on the reference layer. It corresponds to that.

  In the above configuration, the reference coding unit for which the degree of overlap with the reference region should be determined is preferably a coding unit having a property that represents the coding unit around the reference region.

  Also, such a reference coding unit is set based on, for example, a coding unit having an overlapping portion with a reference region that is a region corresponding to the target coding unit.

  For example, a coding unit selected according to a predetermined criterion from among coding units having an overlapping portion with a reference region that is a region corresponding to the target coding unit can be set as the reference coding unit. In addition, it is not limited to selecting a single coding unit, a representative property is extracted from coding units around the reference region, and a virtual single coding unit is generated based on the extracted property. Thus, a reference coding unit may be used. The virtual single coding unit is generated so as to be an area overlapping the reference area.

  According to the above configuration, a division pattern into prediction units, which are prediction image processing units, is determined for the target coding unit in accordance with the degree of overlap between the reference region and the reference coding unit.

  Therefore, it is possible to estimate the division pattern of the target coding unit into the prediction unit with a relatively small amount of processing without strictly determining the boundary of the coding unit in the reference layer.

  Specifically, when partition partitioning is determined based on the prediction unit boundary position of the reference layer, the determination is complicated because there are a plurality of HEVC CU sizes. Can be estimated.

  As a result, it is possible to reduce the amount of processing for deriving the prediction unit division pattern.

  In the image decoding device according to the present invention, the reference coding unit is preferably a coding unit in the reference layer including a predetermined pixel in the reference region.

  According to the above configuration, the reference coding unit is a coding unit in the reference layer including a predetermined pixel in the reference region. The predetermined pixel may be a pixel in the center of the reference area or an upper left pixel in the reference area.

  According to the above configuration, the reference coding unit including a predetermined pixel in the reference area can be specifically identified specifically.

  In the image decoding device according to the present invention, it is preferable that the predetermined pixel is an upper left pixel of the reference region.

  According to the above configuration, the predetermined pixel is the upper left pixel of the reference area. In image processing, the upper left corner of an image or a processing unit that divides the image is often the origin or the reference point. Therefore, such a reference point can be used in the division pattern determination process. Thereby, it is possible to simplify the specific processing contents of the division pattern determination processing.

  In the image decoding apparatus according to the present invention, it is preferable that the reference coding unit is a coding unit having the largest area among coding units having an overlapping portion with the reference region.

  Of the coding units having overlapping portions with the reference region, the coding unit having the largest area is likely to be the same PU partition type as the target CU.

  According to the above configuration, since the coding unit having the maximum area can be set as the reference coding unit, it is possible to improve the estimation accuracy of the division pattern.

  In the image decoding apparatus according to the present invention, the degree of overlap between the reference region and the reference coding unit indicates a magnitude relationship regarding the width and height of the overlapping portion between the reference region and the reference coding unit. In addition, it is preferable that the division pattern determination unit determines the division pattern according to the magnitude relationship.

  According to the above configuration, the division pattern can be specifically determined with a relatively small amount of processing according to the size of the overlapping region between the reference region and the reference coding unit.

  In the image decoding device according to the present invention, the degree of overlap between the reference region and the reference coding unit indicates an inclusion relationship between the reference region and the reference coding unit, and the division pattern determination is performed. The means preferably determines the division pattern according to the inclusion relation.

  According to the above configuration, the division pattern can be specifically determined with a relatively small amount of processing in accordance with the inclusion relationship between the reference region and the reference coding unit.

  More specifically, when the reference area is included in the reference coding unit, a division pattern that does not perform division can be selected.

  In the image decoding device according to the present invention, the degree of overlap between the reference region and the reference coding unit indicates a positional relationship of the lower right pixel of the reference coding unit with respect to the reference region, and the division pattern The determining means preferably determines the division pattern according to the positional relationship.

  According to the above configuration, the division pattern can be specifically determined with a relatively small amount of processing in accordance with the positional relationship between the reference region and the reference coding unit.

  In the image decoding apparatus according to the present invention, it is preferable that the division pattern determination means determines a symmetric partition as the division pattern.

  According to the above configuration, a symmetric partition is determined as the division pattern. Conversely, according to the above configuration, an asymmetric partition is not derived. For this reason, since it is not necessary to determine exactly where the boundary is in the vertical position or where the boundary is in the horizontal position, the processing can be simplified.

  In the image decoding apparatus according to the present invention, it is preferable that the division pattern determination unit restricts or prohibits division when the target coding unit is a predetermined size or less.

  According to the above configuration, division is restricted or prohibited when the target coding unit is a predetermined size or less. For example, in an 8 × 8 CU, 2N × 2N can always be determined as a division pattern. That is, it is possible to prohibit the division of the prediction unit in the target coding unit.

  Further, for example, selection of N × N as a division pattern or selection of an asymmetric partition may be limited.

  The occurrence of smaller partitions can be suppressed. As a result, it is possible to suppress the generation of a small-sized prediction unit that becomes a bottleneck of processing, and to reduce the average processing amount.

  In the image decoding device according to the present invention, the division pattern determining means, when the reference region is included in the reference coding unit, based on the division pattern of the reference coding unit, in the target coding unit. It is preferable to derive the division pattern.

  According to the above configuration, when the reference region is included in the reference coding unit, the divided pattern can be derived with high accuracy while suppressing the processing amount based on the division pattern of the reference coding unit.

  In the image decoding device according to the present invention, when the ratio of the resolution of the target layer to the resolution of the reference layer is smaller than a predetermined value, the division pattern determination unit determines the size of the reference coding unit and the target coding unit. It is preferable to determine the division pattern according to the size relationship with the size.

  According to the above configuration, when the ratio of the resolution of the target layer to the resolution of the reference layer is smaller than a predetermined value, the size depends on the size relationship between the size of the reference coding unit and the size of the target coding unit. Determine the division pattern. Therefore, since the division pattern determination process only needs to be compared in magnitude relation, the determination process can be simplified.

  The image decoding apparatus according to the present invention further comprises flag decoding means for decoding a reference layer estimation flag, which is a flag indicating that estimation processing from the reference layer is performed, from the hierarchically encoded data, and the target coding unit When the reference layer estimation flag decoded in step 1 indicates that estimation processing from a reference layer is to be performed, the setting of the reference coding unit by the reference coding unit setting unit and the division by the division pattern determination unit Preferably, pattern determination is performed.

  According to the above configuration, the division pattern can be estimated from the reference layer in the coding unit for performing the estimation process from the reference layer.

  The data structure of the hierarchically encoded data according to the present invention is the data structure of the hierarchically encoded data decoded by the image decoding device according to claim 12, wherein the reference layer estimation flag is in a skip mode. The reference layer estimation flag that is encoded after the skip flag that indicates whether or not, and before the prediction mode flag that is a flag indicating the type of prediction mode, and decoded in the target encoding unit, When indicating that the estimation process from the reference layer is performed, it is preferable that the encoding of the prediction mode flag is omitted.

  According to the said structure, when a reference layer estimation flag (base_mode_flag) shows performing the estimation process from a reference layer, encoding of a prediction mode flag (pred_mode_flag) is abbreviate | omitted. Therefore, the coding efficiency can be improved by the amount that the prediction mode flag is omitted.

  Note that when the reference layer estimation flag indicates that the estimation process from the reference layer is performed, the flag indicating the division type (part_mode) may be omitted. As a result, the encoding efficiency can be further improved.

  In order to solve the above-described problem, an image encoding device according to the present invention hierarchically encodes image information related to images of different quality for each layer, and includes a code including information related to an image in a target layer to be encoded. An image encoding device for generating encoded data, wherein a target encoding unit that is an encoding target encoding unit in the target layer corresponds to the target encoding unit in a reference layer that is an encoded layer A reference coding unit setting means for setting a reference coding unit to determine a degree of overlap with the reference region based on a coding unit having an overlapping portion with a reference region that is a region; the reference region and the reference Division that determines a division pattern into prediction units that are prediction image processing units for the target coding unit according to the degree of overlap with the coding unit Characterized in that it comprises a turn determining means.

  The image coding apparatus configured as described above is also within the scope of the present invention, and in this case as well, the same operations and effects as those of the image decoding apparatus can be obtained.

  Further, the data structure of hierarchically encoded data generated in the image encoding device and decoded in the image decoding device is also within the scope of the present invention.

  An image decoding apparatus according to the present invention decodes hierarchically encoded data in which image information relating to images of different quality for each layer is hierarchically encoded, and restores an image in a target layer to be decoded The target coding unit that is a coding target to be decoded in the target layer has an overlapping part with a reference region that is a region corresponding to the target coding unit in a reference layer that is a decoded layer Based on the coding unit, reference coding unit setting means for setting a reference coding unit for determining the degree of overlap with the reference region, and according to the degree of overlap between the reference region and the reference coding unit Thus, the target coding unit includes a division pattern determination unit that determines a division pattern into prediction units that are processing units of the prediction image.

  An image encoding apparatus according to the present invention hierarchically encodes image information related to images of different quality for each layer, and generates image encoded data including information related to images in a target layer to be encoded The apparatus overlaps a reference region that is a region corresponding to the target coding unit in a reference layer that is a coded layer, with respect to a target coding unit that is a coding unit to be coded in the target layer A reference coding unit setting means for setting a reference coding unit for determining a degree of overlap with the reference region based on a coding unit having a degree of overlap between the reference region and the reference coding unit And a division pattern determination unit that determines a division pattern into prediction units that are prediction image processing units for the target coding unit. .

  Therefore, it is possible to reduce the amount of processing for deriving the prediction unit division pattern.

It is a functional block diagram which illustrates about the structure of the PU division | segmentation type derivation | leading-out part contained in the hierarchy moving image decoding apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the layer structure of the hierarchy coding data which concerns on embodiment of this invention, Comprising: (a) has shown about the hierarchy moving image encoder side, (b) is a hierarchy moving image decoding. The device side is shown. It is a figure for demonstrating the structure of the hierarchy coding data which concerns on embodiment of this invention, Comprising: (a) has shown the sequence layer which prescribes | regulates sequence SEQ, (b) has prescribed | regulated picture PICT. (C) shows a slice layer that defines a slice S, (d) shows a tree block layer that defines a tree block TBLK, and (e) Indicates a CU layer that defines a coding unit (CU) included in the tree block TBLK. It is a figure which shows the pattern of PU division | segmentation type | mold, (a)-(h) is PU division type 2NxN, 2NxnU, 2NxnD, 2NxN, 2NxnU, and 2N, respectively. The partition shape in the case of xnD is shown. It is a functional block diagram which shows the schematic structure of the said hierarchy moving image decoding apparatus. It is a functional block diagram which shows the schematic structure of the prediction parameter decompression | restoration part with which the said hierarchy moving image decoding apparatus is provided. It is a figure shown about the direction of the intra prediction which can be utilized in the said hierarchy moving image decoding apparatus. It is a figure which shows intra prediction mode and the name matched with the said intra prediction mode. It is a functional block diagram which shows the schematic structure of the texture restoration part with which the said hierarchy moving image decoding apparatus is provided. It is a functional block diagram which shows the schematic structure of the base decoding part with which the said hierarchy moving image decoding apparatus is provided. It is a figure for demonstrating operation | movement of the spatial merge candidate derivation | leading-out part with which the said merge candidate derivation | leading-out part is provided. It is a figure which shows operation | movement of the merge candidate deriving part between layers with which the said merge candidate deriving part is provided. It is a figure explaining operation | movement of the time merge candidate derivation | leading-out part with which the said merge candidate derivation | leading-out part is provided. It is a figure which shows the example of a merge candidate combination list. It is a figure which shows operation | movement of the zero merge candidate derivation | leading-out part with which the said merge candidate derivation | leading-out part is provided. It is a flowchart which shows an example of the flow of operation | movement of the said merge candidate derivation | leading-out part. It is a figure which illustrates about another operation | movement of the said merge candidate derivation | leading-out part. (A)-(c) shows the example which derives the inter-layer merge candidate from two or more positions. (D) shows an example of prohibiting the derivation of inter-layer merge candidates. It is a flowchart which shows another example of the flow of operation | movement of the said merge candidate derivation | leading-out part. It is a functional block diagram illustrated about the structure which concerns on the modification of the said merge candidate derivation | leading-out part. It is a figure for demonstrating the area | region which has not been decoded when a scan order is a raster scan order. It is a figure for demonstrating the area | region which has not been decoded when a scanning order is a Z scanning order. It is a functional block diagram which shows schematic structure of the hierarchy moving image encoder which concerns on one Embodiment of this invention. It is a functional block diagram which shows the schematic structure of the prediction parameter decompression | restoration part with which the said hierarchy moving image encoder is provided. It is a figure shown about the direction of the intra prediction which can be utilized in the said hierarchy moving image encoder. It is the figure shown about the structure of the transmitter which mounts the said hierarchy moving image encoder, and the receiver which mounts the said hierarchy moving image decoder. (A) shows a transmission device equipped with a hierarchical video encoding device, and (b) shows a reception device equipped with a hierarchical video decoding device. It is the figure shown about the structure of the recording device carrying the said hierarchy moving image encoder, and the reproducing | regenerating apparatus carrying the said hierarchy moving image decoding apparatus. (A) shows a recording device equipped with a hierarchical video encoding device, and (b) shows a playback device equipped with a hierarchical video decoding device. It is a functional block diagram which illustrates about the structure of the merge candidate derivation | leading-out part contained in the hierarchy moving image decoding apparatus which concerns on one Embodiment of this invention. It is a figure explaining the example in which the reference CU setting part with which the said PU division | segmentation type derivation part is provided sets a reference CU with respect to the object CU contained in the object frame on an object layer. It is the figure which showed more concretely about the setting of the reference CU of FIG. It is the figure which showed about the method which sets a PU division | segmentation type using the result of having determined the position in the reference area of the position of the lower right pixel of reference CU. It is the figure which showed about the method which sets a PU division | segmentation type using the result of having determined the position in the reference area of the position of the lower right pixel of reference CU. It is the figure which showed about the method which sets a PU division | segmentation type using the result of having determined the position in the reference area of the position of the lower right pixel of reference CU. It is the figure which showed about the method which sets a PU division | segmentation type using the result of having determined the position in the reference area of the position of the lower right pixel of reference CU. It is a figure which demonstrates more concretely about the method of determining the position of a lower right pixel. It is a figure which demonstrates more concretely about the method of determining the position of a lower right pixel. It is the figure which showed about the method of determining PU partition type using the result of having determined the overlapping degree of the overlapping area of a reference CU and a reference area. It is the figure which showed about the method of determining PU partition type using the result of having determined the overlapping degree of the overlapping area of a reference CU and a reference area. It is the figure which showed about the method of determining PU partition type using the result of having determined the overlapping degree of the overlapping area of a reference CU and a reference area. It is a figure for demonstrating more concretely about determination of the duplication degree of an duplication area | region. It is a figure for demonstrating more concretely about determination of the duplication degree of an duplication area | region. It is a table which shows the relationship between a syntax element value and CU type. This is a syntax table for coding base_mode_flag with coding_unit. It is a syntax table when determining base_mode_flag by prediction_unit. It is a flowchart shown about an example of the flow of the base skip CU decoding process in a hierarchy moving image decoding apparatus. It is a flowchart shown about an example of the flow of the skip CU decoding process in a hierarchy moving image decoding apparatus. It is a flowchart shown about an example of the flow of the inter CU decoding process in a hierarchy moving image decoding apparatus. It is a figure explaining the prediction unit estimation process between layers of SVC.

The hierarchical moving picture decoding apparatus 1 and the hierarchical moving picture encoding apparatus 2 according to an embodiment of the present invention will be described based on FIGS. 1 to 46 as follows.
〔Overview〕
A hierarchical video decoding device (image decoding device) 1 according to the present embodiment receives encoded data that has been subjected to scalable video coding (SVC) by a hierarchical video encoding device (image encoding device) 2. Decrypt. Scalable video coding is a coding method that hierarchically encodes moving images from low quality to high quality. Scalable video coding is, for example, H.264. H.264 / AVC Annex G SVC. Note that the quality of a moving image here widely means an element that affects the appearance of a subjective and objective moving image. The quality of the moving image includes, for example, “resolution”, “frame rate”, “image quality”, and “pixel representation accuracy”. Therefore, hereinafter, if the quality of the moving image is different, it means that, for example, “resolution” is different, but it is not limited thereto. For example, in the case of moving images quantized in different quantization steps (that is, moving images encoded with different encoding noises), it can be said that the quality of moving images is different from each other.

  In addition, the SVC is sometimes classified into (1) spatial scalability, (2) temporal scalability, and (3) SNR (Signal to Noise Ratio) scalability from the viewpoint of the type of information to be hierarchized. Spatial scalability is a technique for hierarchizing resolution and image size. Time scalability is a technique for layering at a frame rate (the number of frames per unit time). Also, SNR scalability is a technique for hierarchizing in coding noise.

Prior to detailed description of the hierarchical video encoding device 2 and the hierarchical video decoding device 1 according to the present embodiment, first, (1) the hierarchical video encoding device 2 generates and the hierarchical video decoding device 1 performs decoding. The layer structure of the hierarchically encoded data to be performed will be described, and then (2) a specific example of the data structure that can be adopted in each layer will be described.
[Layer structure of hierarchically encoded data]
Here, encoding and decoding of hierarchically encoded data will be described with reference to FIG. FIG. 2 is a diagram schematically illustrating a case where a moving image is hierarchically encoded / decoded by three layers of a lower layer L3, a middle layer L2, and an upper layer L1. That is, in the example shown in FIGS. 2A and 2B, of the three layers, the upper layer L1 is the highest layer and the lower layer L3 is the lowest layer.

  In the following, a decoded image corresponding to a specific quality that can be decoded from hierarchically encoded data is referred to as a decoded image of a specific hierarchy (or a decoded image corresponding to a specific hierarchy) (for example, in the upper hierarchy L1). Decoded image POUT # A).

  FIG. 2A shows a hierarchical video encoding device 2 # A-2 # C that generates encoded data DATA # A-DATA # C by hierarchically encoding input images PIN # A-PIN # C, respectively. Is shown. FIG. 2B illustrates a hierarchical video decoding device 1 # A that generates decoded images POUT # A to POUT # C by decoding the hierarchically encoded data DATA # A to DATA # C, respectively. 1 # C is shown.

  First, the encoding device side will be described with reference to FIG. The input images PIN # A, PIN # B, and PIN # C that are input on the encoding device side have the same original image but different image quality (resolution, frame rate, image quality, and the like). The image quality decreases in the order of the input images PIN # A, PIN # B, and PIN # C.

  The hierarchical video encoding apparatus 2 # C of the lower hierarchy L3 encodes the input image PIN # C of the lower hierarchy L3 to generate encoded data DATA # C of the lower hierarchy L3. Basic information necessary for decoding the decoded image POUT # C of the lower layer L3 is included (indicated by “C” in FIG. 2). Since the lower layer L3 is the lowest layer, the encoded data DATA # C of the lower layer L3 is also referred to as basic encoded data.

  Further, the hierarchical video encoding apparatus 2 # B of the middle hierarchy L2 encodes the input image PIN # B of the middle hierarchy L2 with reference to the encoded data DATA # C of the lower hierarchy, and performs the middle hierarchy L2 Encoded data DATA # B is generated. In addition to the basic information “C” included in the encoded data DATA # C, additional data necessary for decoding the decoded image POUT # B of the intermediate hierarchy is added to the encoded data DATA # B of the intermediate hierarchy L2. Information (indicated by “B” in FIG. 2) is included.

  Further, the hierarchical video encoding apparatus 2 # A of the upper hierarchy L1 encodes the input image PIN # A of the upper hierarchy L1 with reference to the encoded data DATA # B of the intermediate hierarchy L2 to Encoded data DATA # A is generated. The encoded data DATA # A of the upper layer L1 is used to decode the basic information “C” necessary for decoding the decoded image POUT # C of the lower layer L3 and the decoded image POUT # B of the middle layer L2. In addition to the necessary additional information “B”, additional information (indicated by “A” in FIG. 2) necessary for decoding the decoded image POUT # A of the upper layer is included.

  As described above, the encoded data DATA # A of the upper layer L1 includes information related to decoded images having a plurality of different qualities.

  Next, the decoding device side will be described with reference to FIG. On the decoding device side, the decoding devices 1 # A, 1 # B, and 1 # C corresponding to the layers of the upper layer L1, the middle layer L2, and the lower layer L3 are encoded data DATA # A and DATA # B, respectively. , And DATA # C are decoded to output decoded images POUT # A, POUT # B, and POUT # C.

  It is also possible to reproduce a moving image having a specific quality by extracting a part of the information of the upper layer encoded data and decoding the extracted information in a lower specific decoding device.

  For example, the hierarchical moving picture decoding apparatus 1 # B in the middle hierarchy L2 receives information necessary for decoding the decoded picture POUT # B from the hierarchical encoded data DATA # A in the upper hierarchy L1 (that is, the hierarchical encoded data). The decoded image POUT # B may be decoded by extracting “B” and “C”) included in DATA # A. In other words, on the decoding device side, the decoded images POUT # A, POUT # B, and POUT # C can be decoded based on information included in the hierarchically encoded data DATA # A of the upper hierarchy L1.

  The hierarchical encoded data is not limited to the above three-layer hierarchical encoded data, and the hierarchical encoded data may be hierarchically encoded with two layers or may be hierarchically encoded with a number of layers larger than three. Good.

  Also, a part or all of the encoded data related to the decoded image of a specific hierarchy is encoded independently of the other hierarchy, and it is not necessary to refer to information of the other hierarchy when decoding the specific hierarchy. Hierarchically encoded data may be configured as described above. For example, in the example described above with reference to FIGS. 2A and 2B, it has been described that “C” and “B” are referred to for decoding the decoded image POUT # B, but the present invention is not limited thereto. It is also possible to configure the hierarchically encoded data so that the decoded image POUT # B can be decoded using only “B”.

  When SNR scalability is realized, the same original image is used as the input images PIN # A, PIN # B, and PIN # C, and the decoded images POUT # A, POUT # B, and POUT # C have different image quality. Hierarchically encoded data can also be generated so that In that case, the lower layer hierarchical video encoding device generates hierarchical encoded data by quantizing the prediction residual using a larger quantization width than the upper layer hierarchical video encoding device. To do.

  In this document, the following terms are defined for convenience of explanation. The following terms are used to indicate the following technical matters unless otherwise specified.

  Upper layer: A layer positioned higher than a certain layer is referred to as an upper layer. For example, in FIG. 2, the upper layers of the lower layer L3 are the middle layer L2 and the upper layer L1. The decoded image of the upper layer means a decoded image with higher quality (for example, high resolution, high frame rate, high image quality, etc.).

  Lower layer: A layer located lower than a certain layer is referred to as a lower layer. For example, in FIG. 2, the lower layers of the upper layer L1 are the middle layer L2 and the lower layer L3. Further, the decoded image of the lower layer refers to a decoded image with lower quality.

  Target layer: A layer that is the target of decoding or encoding.

  Reference layer: A specific lower layer referred to for decoding a decoded image corresponding to a target layer is referred to as a reference layer.

  In the example shown in FIGS. 2A and 2B, the reference layers of the upper hierarchy L1 are the middle hierarchy L2 and the lower hierarchy L3. However, the present invention is not limited to this, and the hierarchically encoded data can be configured so that it is not necessary to refer to all of the lower layers in decoding of the specific layer. For example, the hierarchical encoded data can be configured such that the reference layer of the upper hierarchy L1 is either the middle hierarchy L2 or the lower hierarchy L3.

  Base layer: A layer located at the lowest layer is referred to as a base layer. The decoded image of the base layer is the lowest quality decoded image that can be decoded from the encoded data, and is referred to as a basic decoded image. In other words, the basic decoded image is a decoded image corresponding to the lowest layer. The partially encoded data of the hierarchically encoded data necessary for decoding the basic decoded image is referred to as basic encoded data. For example, the basic information “C” included in the hierarchically encoded data DATA # A of the upper hierarchy L1 is the basic encoded data.

  Enhancement layer: The upper layer of the base layer is referred to as an enhancement layer.

  Layer identifier: The layer identifier is for identifying a hierarchy, and corresponds to the hierarchy one-to-one. The hierarchically encoded data includes a hierarchical identifier used for selecting partial encoded data necessary for decoding a decoded image of a specific hierarchy. A subset of hierarchically encoded data associated with a layer identifier corresponding to a specific layer is also referred to as a layer representation.

  In general, for decoding a decoded image of a specific hierarchy, a layer expression of the hierarchy and / or a layer expression corresponding to a lower layer of the hierarchy is used. That is, in decoding the decoded image of the target layer, layer representation of the target layer and / or layer representation of one or more layers included in a lower layer of the target layer are used.

  Inter-layer prediction: Inter-layer prediction is based on a syntax element value included in a layer expression of a layer (reference layer) different from the layer expression of the target layer, a value derived from the syntax element value, and a decoded image. It is to predict the syntax element value of the target layer, the encoding parameter used for decoding of the target layer, and the like. Inter-layer prediction in which information related to motion prediction is predicted from reference layer information (at the same time) may be referred to as motion information prediction. Further, inter-layer prediction that predicts a decoded image of a lower layer (at the same time) from an up-sampled image may be referred to as texture prediction (or inter-layer intra prediction). Note that the hierarchy used for inter-layer prediction is, for example, a lower layer of the target layer. In addition, performing prediction within a target layer without using a reference layer may be referred to as intra-layer prediction.

Note that the above terms are for convenience of explanation until they are tired, and the above technical matters may be expressed by other terms.
[Data structure of hierarchically encoded data]
Hereinafter, the case where HEVC and its extended system are used as an encoding system for generating encoded data of each layer will be exemplified. However, the present invention is not limited to this. It may be generated by an encoding method such as H.264 / AVC.

  Further, the lower layer and the upper layer may be encoded by different encoding methods. The encoded data of each layer may be supplied to the hierarchical video decoding device 1 via different transmission paths, or supplied to the hierarchical video decoding device 1 via the same transmission path. It may be done.

  For example, when transmitting ultra-high-definition video (moving image, 4K video data) with a base layer and one extended layer in a scalable encoding, the base layer downscales 4K video data, and interlaced video data. MPEG-2 or H.264 The enhancement layer may be encoded by H.264 / AVC and transmitted over a television broadcast network, and the enhancement layer may encode 4K video (progressive) with HEVC and transmit over the Internet.

(Basic layer)
FIG. 3 is a diagram illustrating a data structure of encoded data (hierarchically encoded data DATA # C in the example of FIG. 2) that can be employed in the base layer. Hierarchically encoded data DATA # C illustratively includes a sequence and a plurality of pictures constituting the sequence.

  FIG. 3 shows a hierarchical structure of data in the hierarchically encoded data DATA # C. 3A to 3E respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, and a tree block that defines a tree block TBLK. It is a figure which shows the CU layer which prescribes | regulates the coding unit (Coding Unit; CU) contained in a layer and tree block TBLK.

(Sequence layer)
In the sequence layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined. As shown in FIG. 3A, the sequence SEQ includes a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), an adaptive parameter set APS (Adaptation Parameter Set), and pictures PICT _{1 to} PICT. _{It includes NP} (NP is the total number of pictures included in the sequence SEQ) and supplemental enhancement information (SEI).

  In the sequence parameter set SPS, a set of encoding parameters referred to by the hierarchical video decoding device 1 for decoding the target sequence is defined.

  In the picture parameter set PPS, a set of encoding parameters referred to by the hierarchical video decoding device 1 for decoding each picture in the target sequence is defined. A plurality of PPS may exist. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.

  The adaptive parameter set APS defines a set of encoding parameters that the hierarchical video decoding device 1 refers to in order to decode each slice in the target sequence. There may be a plurality of APSs. In that case, one of a plurality of APSs is selected from each slice in the target sequence.

(Picture layer)
In the picture layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 3B, the picture PICT includes a picture header PH and slices S _{1 to} S _NS (NS is the total number of slices included in the picture PICT).

Note that, hereinafter, when it is not necessary to distinguish each of the slices S _{1 to} S _NS , the reference numerals may be omitted. The same applies to data included in the hierarchically encoded data DATA # C described below and to which subscripts are added.

  The picture header PH includes a coding parameter group referred to by the hierarchical video decoding device 1 in order to determine a decoding method of the target picture. Note that the encoding parameter group is not necessarily included directly in the picture header PH, and may be included indirectly, for example, by including a reference to the picture parameter set PPS.

(Slice layer)
In the slice layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a slice S (also referred to as a target slice) to be processed is defined. As shown in FIG. 3C, the slice S includes a slice header SH and a sequence of tree blocks TBLK _{1 to} TBLK _NC (NC is the total number of tree blocks included in the slice S).

  The slice header SH includes a group of coding parameters that the hierarchical video decoding device 1 refers to in order to determine a method for decoding the target slice. Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.

  As slice types that can be specified by the slice type specification information, (1) I slice that uses only intra prediction at the time of encoding, (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

  Note that the slice header SH may include a reference to the picture parameter set PPS (pic_parameter_set_id) and a reference to the adaptive parameter set APS (aps_id) included in the sequence layer.

The slice header SH includes a filter parameter FP that is referred to by an adaptive filter provided in the hierarchical video decoding device 1. The filter parameter FP includes a filter coefficient group. The filter coefficient group includes (1) tap number designation information for designating the number of taps of the filter, (2) filter coefficients a _{0 to} a _NT-1 (NT is the total number of filter coefficients included in the filter coefficient group), and , (3) offset is included.

(Tree block layer)
In the tree block layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a processing target tree block TBLK (hereinafter also referred to as a target tree block) is defined. Note that the tree block may be referred to as a coding tree block (CTB) or a maximum coding unit (LCU).

The tree block TBLK includes a tree block header TBLKH and coding unit information CU _{1 to} CU _NL (NL is the total number of coding unit information included in the tree block TBLK). Here, first, a relationship between the tree block TBLK and the coding unit information CU will be described as follows.

  The tree block TBLK is divided into partitions for specifying a block size for each process of intra prediction or inter prediction and conversion.

  The partition of the tree block TBLK is divided by recursive quadtree partitioning. The tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree.

  Hereinafter, a partition corresponding to a leaf that is a node at the end of the coding tree is referred to as a coding node. In addition, since the encoding node is a basic unit of the encoding process, hereinafter, the encoding node is also referred to as an encoding unit (CU). Note that the coding node may be referred to as a coding block (CB).

That is, coding unit information (hereinafter referred to as CU information) CU _{1 to} CU _NL is information corresponding to each coding node (coding unit) obtained by recursively dividing the tree block TBLK into quadtrees. is there.

  Also, the root of the coding tree is associated with the tree block TBLK. In other words, the tree block TBLK is associated with the highest node of the tree structure of the quadtree partition that recursively includes a plurality of encoding nodes.

  Note that the size of each coding node is half the size of the coding node to which the coding node directly belongs (that is, the partition of the node one layer higher than the coding node).

  Further, the size of the tree block TBLK and the size that each coding node can take are the size specification information of the minimum coding node and the maximum coding node included in the sequence parameter set SPS of the hierarchical coding data DATA # C. And the minimum coding node hierarchy depth difference. For example, when the size of the minimum coding node is 8 × 8 pixels and the difference in the layer depth between the maximum coding node and the minimum coding node is 3, the size of the tree block TBLK is 64 × 64 pixels. The size of the encoding node can take any of four sizes, namely, 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels.

(Tree block header)
The tree block header TBLKH includes an encoding parameter referred to by the hierarchical video decoding device 1 in order to determine a decoding method of the target tree block. Specifically, as shown in FIG. 3D, tree block division information SP_TBLK that specifies a division pattern of the target tree block into each CU, and a quantization parameter difference that specifies the size of the quantization step Δqp (qp_delta) is included.

  The tree block division information SP_TBLK is information representing a coding tree for dividing the tree block. Specifically, the shape and size of each CU included in the target tree block, and the position in the target tree block Is information to specify.

  Note that the tree block division information SP_TBLK may not explicitly include the shape or size of the CU. For example, the tree block division information SP_TBLK may be a set of flags indicating whether the entire target tree block or a partial region of the tree block is to be divided into four. In that case, the shape and size of each CU can be specified by using the shape and size of the tree block together.

  The quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target tree block and the quantization parameter qp ′ in the tree block encoded immediately before the target tree block.

(CU layer)
In the CU layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a CU to be processed (hereinafter also referred to as a target CU) is defined.

  Here, before describing specific contents of data included in the CU information CU, a tree structure of data included in the CU will be described. The encoding node is a node at the root of a prediction tree (PT) and a transform tree (TT). The prediction tree and the conversion tree are described as follows.

  In the prediction tree, the encoding node is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined. In other words, the prediction block is one or a plurality of non-overlapping areas constituting the encoding node. The prediction tree includes one or a plurality of prediction blocks obtained by the above division.

  Prediction processing is performed for each prediction block. Hereinafter, a prediction block that is a unit of prediction is also referred to as a prediction unit (PU).

  Broadly speaking, there are two types of partitioning in the prediction tree (hereinafter abbreviated as PU partitioning): intra prediction and inter prediction.

  In the case of intra prediction, there are 2N × 2N (the same size as the encoding node) and N × N division methods.

  In the case of inter prediction, the division method is 2N × 2N (the same size as the encoding node), 2N × N, 2N × nU, 2N × nD, N × 2N, nL × 2N, nR × 2N, and N XN etc. The types of PU division will be described later with reference to the drawings.

  In the transform tree, the encoding node is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined. In other words, the transform block is one or a plurality of non-overlapping areas constituting the encoding node. The conversion tree includes one or a plurality of conversion blocks obtained by the above division.

  There are two types of division in the transformation tree: one in which an area having the same size as that of a coding node is assigned as a transformation block, and the other in division by recursive quadtree division as in the above-described division of a tree block.

  The conversion process is performed for each conversion block. Hereinafter, a transform block that is a unit of transform is also referred to as a transform unit (TU).

(Data structure of CU information)
Next, specific contents of data included in the CU information CU will be described with reference to FIG. As shown in FIG. 3E, the CU information CU specifically includes a skip flag SKIP, prediction tree information (hereinafter abbreviated as PT information) PTI, and conversion tree information (hereinafter abbreviated as TT information). Include TTI).

  The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target PU. When the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, A part of the PT information PTI and the TT information TTI in the CU information CU are omitted. Note that the skip flag SKIP is omitted for the I slice.

[PT information]
The PT information PTI is information related to a prediction tree (hereinafter abbreviated as PT) included in the CU. In other words, the PT information PTI is a set of information related to each of one or a plurality of PUs included in the PT, and is referred to when a predicted image is generated by the hierarchical video decoding device 1. As shown in FIG. 3E, the PT information PTI includes prediction type information PType and prediction information PInfo.

  The prediction type information PType is information that specifies whether intra prediction or inter prediction is used as a predicted image generation method for the target PU.

  The prediction information PInfo includes intra prediction information PP_Intra or inter prediction information PP_Inter depending on which prediction method the prediction type information PType specifies. Hereinafter, a PU to which intra prediction is applied is also referred to as an intra PU, and a PU to which inter prediction is applied is also referred to as an inter PU.

  The inter prediction information PP_Inter includes a coding parameter that is referred to when the hierarchical video decoding device 1 generates an inter prediction image by inter prediction. More specifically, the inter prediction information PP_Inter includes inter PU division information that specifies a division pattern of the target CU into each inter PU, and inter prediction parameters for each inter PU.

  The intra prediction information PP_Intra includes a coding parameter that is referred to when the hierarchical video decoding device 1 generates an intra predicted image by intra prediction. More specifically, the intra prediction information PP_Intra includes intra PU division information that specifies a division pattern of the target CU into each intra PU, and intra prediction parameters for each intra PU. The intra prediction parameter is a parameter for designating an intra prediction method (prediction mode) for each intra PU.

  Further, the PU division information may include information specifying the shape, size, and position of the target PU. Details of the PU partition information will be described later.

[TT information]
The TT information TTI is information regarding a conversion tree (hereinafter abbreviated as TT) included in the CU. In other words, the TT information TTI is a set of information regarding each of one or a plurality of TUs included in the TT, and is referred to when the hierarchical video decoding device 1 decodes residual data. Hereinafter, a TU may be referred to as a block.

As shown in FIG. 3 (e), the TT information TTI includes TT division information SP_TT that specifies a division pattern of the target CU into each transform block, and quantized prediction residuals QD _{1 to} QD _NT (NT is the target The total number of blocks included in the CU).

  Specifically, the TT division information SP_TT is information for determining the shape and size of each TU included in the target CU and the position within the target CU. For example, the TT division information SP_TT can be realized from information (split_transform_unit_flag) indicating whether or not the target node is divided and information (trafoDepth) indicating the division depth.

  For example, when the size of the CU is 64 × 64, each TU obtained by the division can take a size from 32 × 32 pixels to 4 × 4 pixels.

  Each quantized prediction residual QD is encoded data generated by the hierarchical video encoding device 2 performing the following processes 1 to 3 on a target block that is a processing target block.

Process 1: The prediction residual obtained by subtracting the prediction image from the encoding target image is subjected to frequency conversion (for example, DCT conversion (Discrete Cosine Transform) and DST conversion (Discrete Sine Transform));
Process 2: Quantize the transform coefficient obtained in Process 1;
Process 3: Variable length coding is performed on the transform coefficient quantized in Process 2;
Note that the quantization parameter qp described above represents the size of the quantization step QP used when the hierarchical moving image encoding apparatus 2 quantizes the transform coefficient (QP = 2 ^{qp / 6} ).

(Prediction parameter)
Details of prediction parameters in inter prediction and intra prediction will be described. As described above, the prediction information PInfo includes an inter prediction parameter or an intra prediction parameter.

  The inter prediction parameters include, for example, a merge flag (merge_flag), a merge index (merge_idx), an estimated motion vector index (mvp_idx), a reference image index (ref_idx), an inter prediction flag (inter_pred_flag), and a motion vector residual (mvd). Is mentioned.

  On the other hand, examples of the intra prediction parameters include an estimated prediction mode flag, an estimated prediction mode index, and a residual prediction mode index.

(PU partition information)
The PU partition type specified by the PU partition information includes the following eight patterns in total, assuming that the size of the target CU is 2N × 2N pixels. That is, 4 symmetric splittings of 2N × 2N pixels, 2N × N pixels, N × 2N pixels, and N × N pixels, and 2N × nU pixels, 2N × nD pixels, nL × 2N pixels, And four asymmetric splittings of nR × 2N pixels. N = 2 ^m (m is an arbitrary integer of 1 or more). Hereinafter, an area obtained by dividing the target CU is also referred to as a partition.

  4A to 4H specifically illustrate the positions of the boundaries of PU division in the CU for each division type.

  FIG. 4A shows a 2N × 2N PU partition type that does not perform CU partitioning. FIGS. 4B, 4C, and 4D show the partition shapes when the PU partition types are 2N × N, 2N × nU, and 2N × nD, respectively. 4 (e), (f), and (g) show the shapes of the partitions when the PU partition types are N × 2N, nL × 2N, and nR × 2N, respectively. FIG. 4H shows the shape of the partition when the PU partition type is N × N.

  The PU partition types shown in FIGS. 4A and 4H are also referred to as square partitions based on the partition shape. Moreover, the PU division type of FIGS. 4B to 4G is also referred to as non-square division.

  Also, in FIGS. 4A to 4H, the numbers given to the respective regions indicate the region identification numbers, and the regions are processed in the order of the identification numbers. That is, the identification number represents the scan order of the area.

[Partition type for inter prediction]
In the inter PU, seven types other than N × N (FIG. 4 (h)) are defined among the above eight division types. The six asymmetric partitions are sometimes called AMP (Asymmetric Motion Partition).

  A specific value of N is defined by the size of the CU to which the PU belongs, and specific values of nU, nD, nL, and nR are determined according to the value of N. For example, a 128 × 128 pixel inter-CU includes 128 × 128 pixels, 128 × 64 pixels, 64 × 128 pixels, 64 × 64 pixels, 128 × 32 pixels, 128 × 96 pixels, 32 × 128 pixels, and 96 × It is possible to divide into 128-pixel inter PUs.

[Partition type for intra prediction]
In the intra PU, the following two types of division patterns are defined. A division pattern 2N × 2N in which the target CU is not divided, that is, the target CU itself is handled as one PU, and a pattern N × N in which the target CU is divided into four PUs symmetrically.

  Therefore, in the intra PU, the division patterns (a) and (h) can be taken in the example shown in FIG.

  For example, an 128 × 128 pixel intra CU can be divided into 128 × 128 pixel and 64 × 64 pixel intra PUs.

(Enhancement layer)
For the enhancement layer encoded data, for example, a data structure substantially similar to the data structure shown in FIG. 3 can be adopted. However, in the encoded data of the enhancement layer, additional information can be added or parameters can be omitted as follows.

  Information indicating hierarchical encoding may be encoded in the SPS.

  In the slice layer, spatial scalability, temporal scalability, and SNR scalability hierarchy identification information (dependency_id, temporal_id, and quality_id, respectively) may be encoded. Filter information and filter on / off information (described later) can be encoded by a PPS, a slice header, a macroblock header, or the like.

  In the CU information CU, a skip flag (skip_flag), a base mode flag (base_mode_flag), and a prediction mode flag (pred_mode_flag) may be encoded.

  In addition, these flags may specify whether the CU type of the target CU is an intra CU, an inter CU, a skip CU, or a base skip CU.

  Intra CUs and skip CUs can be defined in the same manner as in the HEVC scheme described above. For example, in the skip CU, “1” is set in the skip flag. If it is not a skip CU, “0” is set in the skip flag. In the intra CU, “0” is set in the prediction mode flag.

  In addition, the inter CU may be defined as a CU to which non-skip and motion compensation (MC) is applied. In the inter CU, for example, “0” is set in the skip flag and “1” is set in the prediction mode flag.

  The base skip CU is a CU type that estimates CU or PU information from a reference layer. In the base skip CU, for example, “1” is set in the skip flag and “1” is set in the base mode flag.

  Also, in the PT information PTI, it may be specified whether the PU type of the target PU is an intra PU, an inter PU, a merge PU, or a base merge PU.

  Intra PU, inter PU, and merge PU can be defined similarly to the case of the above-mentioned HEVC system.

  The base merge PU is a PU type that estimates PU information from a reference layer. Further, for example, in the PT information PTI, a merge flag and a base mode flag may be encoded, and using these flags, it may be determined whether or not the target PU is a PU that performs base merge. That is, in the base merge PU, “1” is set to the merge flag and “1” is set to the base mode flag.

  Of the motion vector information included in the enhancement layer, motion vector information that can be derived from the motion vector information included in the lower layer can be omitted from the enhancement layer. With such a configuration, the code amount of the enhancement layer can be reduced, so that the coding efficiency is improved.

  Further, as described above, the encoded data of the enhancement layer may be generated by an encoding method different from the encoding method of the lower layer. That is, the encoding / decoding process of the enhancement layer does not depend on the type of the lower layer codec.

  The lower layer is, for example, MPEG-2 or H.264. It may be encoded by the H.264 / AVC format.

  If the target layer and the reference layer are encoded using different encoding methods, the reference layer parameters are converted to the corresponding parameters of the target layer or similar parameters, so that corresponding compatibility between the layers is achieved. Can keep. For example, MPEG-2, H.264, etc. A macroblock in the H.264 / AVC format can be interpreted as a CTB in HEVC.

Note that the parameters described above may be encoded independently, or a plurality of parameters may be encoded in combination. When a plurality of parameters are encoded in combination, an index is assigned to the combination of parameter values, and the assigned index is encoded. Also, if a parameter can be derived from another parameter or decoded information, the encoding of the parameter can be omitted.
[Hierarchical video decoding device]
Below, the structure of the hierarchy moving image decoding apparatus 1 which concerns on this embodiment is demonstrated with reference to FIGS.

(Configuration of Hierarchical Video Decoding Device)
The schematic configuration of the hierarchical video decoding device 1 will be described with reference to FIG. FIG. 5 is a functional block diagram illustrating a schematic configuration of the hierarchical video decoding device 1. The hierarchical video decoding device 1 decodes the hierarchical encoded data DATA supplied from the hierarchical video encoding device 2 by the HEVC method, and generates a decoded image POUT # T of the target layer.

  As illustrated in FIG. 5, the hierarchical video decoding device 1 includes a NAL demultiplexing unit 11, a variable length decoding unit 12, a prediction parameter restoration unit 14, a texture restoration unit 15, and a base decoding unit 16.

  The NAL demultiplexing unit 11 demultiplexes hierarchically encoded data DATA transmitted in units of NAL units in NAL (Network Abstraction Layer).

  The NAL is a layer provided to abstract communication between a VCL (Video Coding Layer) and a lower system that transmits and stores encoded data.

  VCL is a layer that performs moving image encoding processing, and encoding is performed in VCL. On the other hand, the lower system here is H.264. H.264 / AVC and HEVC file formats and the MPEG-2 system. In the example shown below, the lower system corresponds to the decoding process in the target layer and the reference layer.

  In NAL, a bit stream generated by VCL is divided into units called NAL units and transmitted to a destination lower system. The NAL unit includes encoded data encoded by the VCL and a header for appropriately delivering the encoded data to the destination lower system. Also, the encoded data in each layer is stored in the NAL unit, is NAL multiplexed, and is transmitted to the hierarchical moving image decoding apparatus 1.

  The NAL demultiplexing unit 11 demultiplexes the hierarchical encoded data DATA, and extracts the target layer encoded data DATA # T and the reference layer encoded data DATA # R. Further, the NAL demultiplexing unit 11 supplies the target layer encoded data DATA # T to the variable length decoding unit 12, and also supplies the reference layer encoded data DATA # R to the base decoding unit 16.

  The variable length decoding unit 12 performs a decoding process of information for decoding various syntax values from the binary included in the target layer encoded data DATA # T.

  Specifically, the variable length decoding unit 12 decodes prediction information, encoded information, and transform coefficient information from the encoded data DATA # T as follows.

  That is, the variable length decoding unit 12 decodes the prediction information regarding each CU or PU from the encoded data DATA # T. The prediction information includes, for example, designation of a CU type or a PU type.

  When the CU is an inter CU, the variable length decoding unit 12 decodes the PU partition information from the encoded DATA # T. In addition, in each PU, the variable length decoding unit 12 further converts motion information such as a reference image index RI, an estimated motion vector index PMVI, and a motion vector residual MVD, and mode information as encoded data DATA as prediction information. Decrypt from #T.

  On the other hand, when the CU is an intra CU, the variable length decoding unit 12 further includes (1) size designation information for designating the size of the prediction unit and (2) prediction index designation for designating the prediction index as the prediction information. The intra prediction information including information is decoded from the encoded data DATA # T.

  The variable length decoding unit 12 decodes the encoded information from the encoded data DATA # T. The encoded information includes information for specifying the shape, size, and position of the CU. More specifically, the encoding information includes tree block division information that specifies a division pattern of the target tree block into each CU, that is, the shape, size, and target tree block of each CU included in the target tree block. Contains information that specifies the position within.

  The variable length decoding unit 12 supplies the decoded prediction information and encoded information to the prediction parameter restoration unit 14.

  Further, the variable length decoding unit 12 decodes the quantization prediction residual QD for each block and the quantization parameter difference Δqp for the tree block including the block from the encoded data DATA # T. The variable length decoding unit 12 supplies the decoded quantization prediction residual QD and the quantization parameter difference Δqp to the texture restoration unit 15 as transform coefficient information.

  The base decoding unit 16 decodes base decoding information, which is information regarding a reference layer referred to when decoding a decoded image corresponding to the target layer, from the reference layer encoded data DATA # R. The base decoding information includes a base prediction parameter, a base transform coefficient, and a base decoded image. The base decoding unit 16 supplies the decoded base decoding information to the prediction parameter restoration unit 14 and the texture restoration unit 15.

  The prediction parameter restoration unit 14 restores the prediction parameter using the prediction information and the base decoding information. The prediction parameter restoration unit 14 supplies the restored prediction parameter to the texture restoration unit 15. Note that the prediction parameter restoration unit 14 can refer to motion information stored in a frame memory 155 (described later) included in the texture restoration unit 15 when restoring the prediction parameter.

  The texture restoration unit 15 generates a decoded image POUT # T using the transform coefficient information, the base decoding information, and the prediction parameter, and outputs the decoded image POUT # T to the outside. The texture restoration unit 15 stores information on the restored decoded image in a frame memory 155 (described later) provided therein.

  Details of the base decoding unit 16, the prediction parameter restoring unit 14, and the texture restoring unit 15 will be described below.

(Prediction parameter restoration unit)
The detailed configuration of the prediction parameter restoration unit 14 will be described with reference to FIG. FIG. 6 is a functional block diagram illustrating the configuration of the prediction parameter restoration unit 14.

  As shown in FIG. 6, the prediction parameter restoration unit 14 includes a prediction type selection unit 141, a switch 142, an intra prediction mode restoration unit 143, a motion vector candidate derivation unit 144, a motion information restoration unit 145, a merge candidate derivation unit 146, a merge An information restoration unit 147 and a PU partition type derivation unit 148 are provided.

  The prediction type selection unit 141 sends a switching instruction to the switch 142 according to the CU type or the PU type, and controls the prediction parameter derivation process. Specifically, it is as follows.

  When the intra CU or the intra PU is designated, the prediction type selection unit 141 controls the switch 142 so that the prediction parameter can be derived using the intra prediction mode restoration unit 143.

  When either inter CU (no merging) or inter PU (no merging) is specified, the prediction type selection unit 141 uses the motion information restoration unit 145 to control the switch 142 so that a prediction parameter can be derived.

  When any one of the base skip CU, the base merge PU, the skip CU, and the merge PU is designated, the prediction type selection unit 141 uses the merge information restoration unit 147 to control the switch 142 so that the prediction parameter can be derived.

  The switch 142 supplies the prediction information to any of the intra prediction mode restoration unit 143, the motion information restoration unit 145, and the merge information restoration unit 147 in accordance with an instruction from the prediction type selection unit 141. A prediction parameter is derived at a supply destination of the prediction information.

  The intra prediction mode restoration unit 143 derives a prediction mode from the prediction information. That is, the intra prediction mode restoration unit 143 restores the prediction parameter in the prediction mode.

  Here, the definition of the prediction mode will be described with reference to FIG. FIG. 7 shows the definition of the prediction mode. As shown in the figure, 36 types of prediction modes are defined, and each prediction mode is specified by a number (intra prediction mode index) from “0” to “35”. Moreover, as shown in FIG. 8, the following names are assigned to each prediction mode. That is, “0” is “Intra_Planar (planar prediction mode, plane prediction mode)”, “1” is “Intra DC (intra DC prediction mode)”, and “2” to “34” are “ "Intra Angular (direction prediction)" and "35" is "Intra From Luma". “35” is unique to the color difference prediction mode, and is a mode for performing color difference prediction based on luminance prediction. In other words, the color difference prediction mode “35” is a prediction mode using the correlation between the luminance pixel value and the color difference pixel value. The color difference prediction mode “35” is also referred to as an LM mode. The number of prediction modes (intraPredModeNum) is “35” regardless of the size of the target block.

  The motion vector candidate derivation unit 144 uses the base decoding information to derive an estimated motion vector candidate by intra-layer motion estimation processing or inter-layer motion estimation processing. The motion vector candidate derivation unit 144 supplies the derived motion vector candidates to the motion information restoration unit 145.

  The motion information restoration unit 145 restores motion information regarding each inter PU that is not merged. That is, the motion information restoring unit 145 restores motion information as a prediction parameter.

  When the target CU (PU) is an inter CU (inter PU), the motion information restoration unit 145 restores motion information from the prediction information. More specifically, the motion information restoration unit 145 acquires a motion vector residual (mvd), an estimated motion vector index (mvp_idx), an inter prediction flag (inter_pred_flag), and a reference image index (refIdx). Then, based on the value of the inter prediction flag, a reference image list use flag is determined for each of the reference image list L0 and the reference image list L1. Subsequently, when the corresponding reference image list use flag indicates that the reference image is used, the motion information restoration unit 145 derives an estimated motion vector based on the value of the estimated motion vector index, A motion vector is derived based on the motion vector residual and the estimated motion vector. The motion information restoration unit 145 outputs the motion vector (motion compensation parameter) together with the derived motion vector, the reference image list use flag, and the reference image index.

  The merge candidate derivation unit 146 derives various merge candidates by using decoded motion information supplied from a frame memory 155 described later and / or base decoding information supplied from the base decoding unit 16 and the like. The merge candidate derivation unit 146 supplies the derived merge candidates to the merge information restoration unit 147.

  The merge information restoration unit 147 restores motion information regarding each PU that is merged within or between layers. That is, the motion information restoring unit 145 restores motion information as a prediction parameter.

  Specifically, when the target CU (PU) is a skip CU (merge PU) that performs merging within a layer, the merge information restoration unit 147 uses the merge candidate list derived by the merge candidate derivation unit 146 by intra-layer merging. Then, the motion information is restored by deriving a motion compensation parameter corresponding to the merge index (merge_idx) included in the prediction information.

  In addition, when the merge information restoration unit 147 is a base skip CU that performs merging between layers, the merge information merging unit 146 derives a merge index (merge_idx) included in the prediction information from the merge candidate list derived by inter-layer merging. The motion information is restored by deriving the corresponding motion compensation parameter.

  The PU partition type deriving unit 148 estimates the PU partition type to the PU of the target CU in the target layer using the encoded information and the base decoding information. The PU partition type deriving unit 148 supplies the estimated PU partition type to the merge candidate deriving unit 146 and the merge information restoring unit 147.

  Details of the merge candidate derivation unit 146 and the PU partition type derivation unit 148 will be described later.

(Texture restoration part)
The detailed configuration of the texture restoration unit 15 will be described with reference to FIG. FIG. 9 is a functional block diagram illustrating the configuration of the texture restoration unit 15.

  As shown in FIG. 9, the texture restoration unit 15 includes an inverse orthogonal transform / inverse quantization unit 151, a texture prediction unit 152, an adder 153, a loop filter unit 154, and a frame memory 155.

The inverse orthogonal transform / inverse quantization unit 151 (1) inversely quantizes the quantized prediction residual QD included in the transform coefficient information supplied from the variable length decoding unit 12, and (2) obtained by inverse quantization. The DCT coefficient is subjected to inverse orthogonal transform (for example, DCT (Discrete Cosine Transform) transform), and (3) the prediction residual D obtained by the inverse orthogonal transform is supplied to the adder 153. When the quantization prediction residual QD is inversely quantized, the inverse orthogonal transform / inverse quantization unit 151 derives a quantization step QP from the quantization parameter difference Δqp included in the transform coefficient information. The quantization parameter qp can be derived by adding the quantization parameter difference Δqp to the quantization parameter qp ′ related to the tree block that has been inversely quantized / inversely orthogonally transformed immediately before, and the quantization step QP is performed from the quantization parameter qp to QP. = 2 It can be derived by ^{qp / 6} . Further, the generation of the prediction residual D by the inverse orthogonal transform / inverse quantization unit 151 is performed in units of blocks (transform units).

  The texture prediction unit 152 refers to the base decoded image included in the base decoding information or the decoded decoded image stored in the frame memory according to the prediction parameter, and generates a predicted image.

  More specifically, the texture prediction unit 152 includes an inter prediction unit 152A, an intra-layer intra prediction unit 152B, and an inter-layer intra prediction unit 152C.

  The inter prediction unit 152A generates a prediction image related to each inter prediction partition by inter prediction. Specifically, the inter prediction unit 152A generates a prediction image from the reference image using the motion information supplied as a prediction parameter from the motion information restoration unit 145 or the merge information restoration unit 147.

  The intra-layer intra prediction unit 152B generates a prediction image related to each intra-prediction partition by intra-layer intra prediction. Specifically, the intra-layer intra prediction unit 152B generates a prediction image from the decoded image that has been decoded in the target partition, using the prediction mode supplied from the intra prediction mode restoration unit 143 as a prediction parameter.

  The intra-layer intra prediction unit 152C generates a prediction image related to each intra prediction partition by inter-layer intra prediction. Specifically, the intra-layer intra prediction unit 152C generates a prediction image based on the base decoded image included in the base decoding information, using the prediction mode supplied from the intra prediction mode restoration unit 143 as a prediction parameter. The base decoded image may be appropriately upsampled according to the resolution of the target layer.

  The texture prediction unit 152 supplies the prediction image generated by the inter prediction unit 152A, the intra-layer intra prediction unit 152B, or the inter-layer intra prediction unit 152C to the adder 153.

  The adder 153 adds the texture prediction unit 153 prediction image and the prediction residual D supplied from the inverse orthogonal transform / inverse quantization unit 151 to generate a decoded image.

  The loop filter unit 154 subjects the decoded image supplied from the adder 153 to deblocking processing and filtering processing using adaptive filter parameters.

  The frame memory 155 stores the decoded image that has been filtered by the loop filter unit 154.

(Base decoding unit)
The detailed configuration of the base decoding unit 16 will be described with reference to FIG. FIG. 10 is a functional block diagram illustrating the configuration of the base decoding unit 16.

  As illustrated in FIG. 10, the base decoding unit 16 includes a variable length decoding unit 161, a base prediction parameter restoration unit 162, a base transform coefficient restoration unit 163, and a base texture restoration unit 164.

  The variable length decoding unit 161 performs a decoding process of information for decoding various syntax values from the binary included in the reference layer encoded data DATA # R.

  Specifically, the variable length decoding unit 161 decodes prediction information and transform coefficient information from the encoded data DATA # R. The syntax of the prediction information and transform coefficients decoded by the variable length decoding unit 161 is the same as that of the variable length decoding unit 12, and therefore detailed description thereof is omitted here.

  The variable length decoding unit 161 supplies the decoded prediction information to the base prediction parameter restoring unit 162 and also supplies the decoded transform coefficient information to the base transform coefficient restoring unit 163.

  The base prediction parameter restoration unit 162 restores the base prediction parameter based on the prediction information supplied from the variable length decoding unit 161. The method by which the base prediction parameter restoration unit 162 restores the base prediction parameter is the same as that of the prediction parameter restoration unit 14, and thus detailed description thereof is omitted here. The base prediction parameter restoration unit 162 supplies the restored base prediction parameter to the base texture restoration unit 164 and outputs it to the outside.

  The base transform coefficient restoration unit 163 restores transform coefficients based on the transform coefficient information supplied from the variable length decoding unit 161. The method by which the base transform coefficient restoration unit 163 restores the transform coefficients is the same as that of the inverse orthogonal transform / inverse quantization unit 151, and thus detailed description thereof is omitted here. The base conversion coefficient restoration unit 163 supplies the restored base conversion coefficient to the base texture restoration unit 164 and outputs it to the outside.

  The base texture restoration unit 164 uses the base prediction parameter supplied from the base prediction parameter restoration unit 162 and the base transform coefficient supplied from the base transform coefficient restoration unit 163 to generate a decoded image. Specifically, the base texture restoration unit 164 performs the same texture prediction as the texture prediction unit 152 based on the base prediction parameter, and generates a predicted image. Also, the base texture restoration unit 164 generates a prediction residual based on the base conversion coefficient, and generates a base decoded image by adding the generated prediction residual and the predicted image generated by texture prediction.

  Note that the base texture restoration unit 164 may perform the same filter processing as the loop filter unit 154 on the base decoded image. Further, the base texture restoration unit 164 may include a frame memory for storing the decoded base decoded image, or may refer to the decoded base decoded image stored in the frame memory in texture prediction. Good.

<< Details of merge candidate derivation unit >>
Next, the detailed configuration of the merge candidate derivation unit 146 will be described with reference to FIG. FIG. 27 is a functional block diagram illustrating the configuration of the merge candidate derivation unit 146.

  As illustrated in FIG. 27, the merge candidate derivation unit 146 includes a merge candidate derivation control unit (determination unit) 1461, a merge candidate storage unit 1462, a slice type determination unit 1463, and an individual merge candidate derivation unit 1464.

  The merge candidate derivation control unit 1461 controls the individual merge candidate derivation unit 1464 to derive a predetermined number (merge candidate derivation number) of merge candidates and store them in the merge candidate storage unit 1462. As the merge candidate derivation number, for example, the value of merge_idx + 1 is normally used. Note that an arbitrary integer greater than or equal to the value of merge_idx + 1 may be used as the merge candidate derivation number. For example, a value obtained by adding 1 to the maximum value of merge_idx may be used as MRG_MAX_NUM_CANDS as the merge candidate derivation number.

  The merge candidate storage unit 1462 stores a plurality of merge candidates. Merge candidates are recorded as an ordered list (merge candidate list).

  The slice type determination unit 1463 determines the slice type of the slice including the target PU in response to the request, and outputs the result.

  The individual merge candidate derivation unit 1464 derives and outputs merge candidates by a designated derivation method. The detailed operation of each merge candidate derivation unit selected based on the designated derivation method will be described later. The derived merge candidate is illustratively composed of a reference image list use flag (predFlagLX), a reference image index (refIdxLX), and a motion vector (mvLX) for the reference image list LX. Here, LX is L0 or L1.

(Details of individual merge candidate derivation unit)
More specifically, the individual merge candidate derivation unit 1464 includes a spatial merge candidate derivation unit (target layer candidate derivation unit, spatial motion information candidate derivation unit) 1464A, an inter-layer merge candidate derivation unit (interlayer candidate derivation unit) 1464B, and time merge. A candidate deriving unit (target layer candidate deriving unit, temporal motion information candidate deriving unit) 1464C, a merge merge candidate deriving unit (target layer candidate deriving unit) 1464D, and a zero merge candidate deriving unit (target layer candidate deriving unit) 1464E ing. Although not shown in FIG. 27, the spatial merge candidate derivation unit 1464A and the temporal merge candidate derivation unit 1464C include the decoded CU and PU encoding parameters stored in the frame memory 155, in particular, motion in units of PUs. Compensation (motion compensation parameters) is provided.

  In the following, when referring to the spatial merge candidate derivation unit 1464A, the inter-layer merge candidate derivation unit 1464B, the temporal merge candidate derivation unit 1464C, the combined merge candidate derivation unit 1464D, and the zero merge candidate derivation unit 1464E, These are collectively referred to as “merge candidate derivation unit”. Each merge candidate derivation unit derives merge candidates according to a predetermined priority order. The merge candidate derivation order is controlled by the merge candidate derivation control unit 1461.

  Further, when performing intra-layer merging, the merge candidate derivation control unit 1461 can cause each merge candidate derivation unit to derive arbitrary merge candidates excluding inter-layer merge candidates. Further, when performing inter-layer merging, the merge candidate derivation control unit 1461 can cause each merge candidate derivation unit to derive arbitrary merge candidates including inter-layer merge candidates.

(Spatial merge candidate derivation unit)
FIG. 11 is a diagram for explaining the operation of the spatial merge candidate derivation unit 1464A. FIG. 11 shows the positional relationship between the target PU and adjacent blocks A0, A1, B0, B1, and B2. In general, the spatial merge candidate derivation unit 1464A outputs the motion compensation parameters in the adjacent blocks as merge candidates. Assume that the derivation order is A1, B1, B0, A0, and B2, for example. The derived merge candidates are stored in the merge candidate storage unit 1462. More precisely, the merge candidates are added to the end of the merge candidate list stored in the merge candidate storage unit 1462 in the derived order. The position of each adjacent block can be expressed as follows, where the upper left coordinates of the PU are (xP, yP) and the PU sizes are nPSW and nPSH.
A0: (xP-1, yP + nPSH)
A1: (xP-1, yP + nPSH-1)
B0: (xP + nPSW, yP-1)
B1: (xP + nPSW-1, yP-1)
B2: (xP-1, yP-1)
Note that when any of the following conditions is satisfied, a merge candidate corresponding to the position N (N is any one of A0, A1, B0, B1, or B2) is not derived.
The block at position N is not available (not available).
-The block at position N is intra-coded.
When N is B2 and all merge candidates corresponding to positions A0, A1, B0, and B1 are derived.
When the PU partition type is 2N × N or N × 2N, and the PU index is 1, and the PU of block N and index 0 has the same motion compensation parameter.
When N is B0 and block N and block B1 have the same motion compensation parameter.
When N is A0 and block N and block A1 have the same motion compensation parameter.
If N is B2 and block N has the same motion compensation parameters as either block A1 or block B1.

  Here, a case where a certain block is not available is a case where the block is outside the screen, outside the slice, or undecoded. In addition, two blocks having the same motion compensation parameter means that the reference image list use flag, the reference image index, and the motion vector are all equal for both the reference image lists L0 and L1. The determination of the identity of motion compensation parameters (match determination) will be described in detail later.

(Inter-layer merge candidate derivation unit)
FIG. 12 is a diagram illustrating the operation of the inter-layer merge candidate derivation unit 1464B. FIG. 12 illustrates an area on the reference layer that is referred to when an inter-layer merge candidate is derived. In the inter-layer merge candidate derivation unit 1464B, for example, the motion compensation parameter at the lower right position C0 in the region on the reference layer (hereinafter, abbreviated as the corresponding reference region) corresponding to the target PU is output as the merge candidate. Is done. The derived merge candidates are stored in the merge candidate storage unit 1462. More precisely, the merge candidates are added to the end of the merge candidate list stored in the merge candidate storage unit 1462 in the derived order.

  Specifically, the inter-layer merge candidate derivation unit 1464B derives merge candidates by referring to the base decoding information as follows.

  The base decoding information referred to by the inter-layer merge candidate derivation unit 1464B includes the motion compensation parameters (mxLX_RL, refIdxLX_RL, predFlagLX_RL) of the reference layer.

  Further, the merge candidate (merge candidate C0) output by the inter-layer merge candidate derivation unit 1464B is configured by motion compensation parameters (mvLX_C0, refIdxLX_C0, predFlagLX_C0).

here,
Upper left pixel position of the target prediction unit: (xP, yP)
Corresponding reference layer pixel position: (xPR, yPR)
Ratio of target layer resolution to reference layer: (scaleX, scaleY)
Target PU size: nPSW * nPSH
Then, the sizes of the corresponding reference areas are nPSWR = ceil (nPSW / scaleX) and nPSHR = ceil (nPSH / scaleY). The corresponding reference area is an area on the reference layer corresponding to the target PU.

At this time, if the position of C0 (xC0, yC0),
(xC0, yC0) = (xPR + nPSWR-1, yPR + nPSHR-1)
And the merge candidate C0 is
mvL_C0 [0] = mvLX_RL (xC0, yC0) [0] * scaleX
mvL_C0 [1] = mvLX_RL (xC0, yC0) [1] * scaleY
refIdxLX_C0 = refIdxLX_RL (xC0, yC0)
predFlagLX_C0 = predFlagLX_RL (xC0, yC0)
It becomes.

  As described above, the spatial merge candidate is derived based on the position information of the upper right (B1, B0), lower left (A1, A0), and upper left (B2) of the target PU, whereas the inter-layer merge candidate is It is derived based on the information of the lower right (C0) position in the corresponding reference area. As a result, the merge candidate list includes various merge candidates having different properties from the spatial merge candidates.

  As a result, there is a higher possibility that any merge candidate included in the merge candidate list matches the actual motion information, thereby reducing the amount of code of the motion information.

  Note that the present invention is not limited to the above example, and the inter-layer merge candidate derivation unit 1464B may be configured to derive a merge candidate from an arbitrary position in the corresponding reference region.

  For example, it is only necessary that motion information decoded in a region on a reference layer corresponding to an undecoded region in the target prediction unit can be derived as a merge candidate.

  Further, the inter-layer merge candidate derivation unit 1464B may derive based on information on a position adjacent to any of the lower right (C0) right, lower, and lower right.

  For example, the inter-layer merge candidate derivation unit 1464B may be able to derive a merge candidate from a position on the reference layer corresponding to a position near the region R1 of the target PU illustrated in FIG. The region R1 can be expressed as a region in a predetermined range including the lower right position in the target reference region.

  Qualitatively speaking, from the position on the reference layer corresponding to a position different from the position from which the spatial merge candidate is derived (“different position”, in other words, “position separated by a predetermined distance or more”), It is only necessary that the intermerging candidate is derived.

  In the spatial merge candidate, there is mostly motion information corresponding to the upper left area of the target PU, and less motion information corresponding to the lower right area (area R1). According to the above configuration, a merge candidate having a property different from that of the spatial merge candidate can be derived from the position in the corresponding reference region as the inter-layer merge candidate.

  A supplementary explanation of the undecrypted area is as follows. In general, as shown in FIG. 20, tree blocks are decoded in raster scan order starting from the upper left corner of the picture and moving in the lower right direction. In such a case, a tree block subsequent to the target tree block including the target prediction unit in the raster scan order is an undecoded region. In the raster scan order, the tree block located below the target tree block or the tree block whose vertical position is the same as the target tree block and located right of the target tree block has not been decoded yet. It is. The area included in the undecoded tree block is an undecoded area.

  Next, regarding the processing order of the coding units (CU) in the tree block, the CUs in the tree block are processed in the so-called Z-scan order. In the Z scan, the tree block is divided into four congruent squares, and processing is performed in the order of the CUs included in the upper left, upper right, lower left, and lower right square regions. Subsequently, processing is also performed recursively for each square area. For example, the upper left area is further divided into four congruent square areas, and CUs included in each square area are processed in the order of upper left, upper right, lower left, and lower right.

  An example of Z scanning will be described with reference to the example of FIG. In FIG. 21, the numbers assigned to the respective regions represent the scan order by Z scan. The upper left area obtained by dividing the tree block into four includes 1 to 7 CUs, the upper right area includes 8 CUs, the lower left area includes 9 CUs, and the lower right area includes 10 to 13 CUs. The upper left area including the CUs 1 to 7 is further divided into four, and the upper left area includes 1 CU, the upper right area includes 2 CUs, the lower left area includes 3 CUs, and the lower right area includes 4 to 7 CUs. A CU after the target CU in the Z-scan order is a CU that has not been decoded, and a region included in the CU is a region that has not been decoded.

  Further, the inter-layer merge candidate derivation unit 1464B may be restricted or prohibited from deriving merge candidates from a specific position.

  For example, it may be prohibited to derive a merge candidate from the upper left position C0 in the corresponding reference region as shown in FIG.

  In some cases, motion information corresponding to the lower right (C0) position in the corresponding reference area is not recorded in the memory. In such a case, information corresponding to a position near the lower right (C0) is used. Based on the above, merge candidates may be derived. For example, when the motion information of the reference layer is recorded by being thinned out at a specific interval (N pixel unit), it is changed to the position of C0, that is, the position of (xC0, yC0) ((xC0 // N) * N, (yC0 // N) * N) may be derived based on motion information corresponding to the position. Here, the operator “//” is an operator whose value of “x // y” is a quotient obtained by dividing x by y.

(Time merge candidate derivation unit)
FIG. 13 is a diagram for explaining the operation of the time merge candidate derivation unit 1464C. Explaining with reference to FIG. 13A, the temporal merge candidate derivation unit 1464C generally derives the temporal merge candidate as follows. That is, when the current picture is currPic, the temporal merge candidate deriving unit 1464C occupies the PU on the reference image specified by the reference image index refIdxL0 that occupies substantially the same spatial position as the target PU in the current picture. Alternatively, the temporal merge candidate is derived by copying the motion compensation parameter of the PU on the reference image specified by the reference image index refIdxL1. A method of deriving the reference index number refIdxL0 and the reference index number refIdxL1 will be described with reference to FIG. The reference index number refIdxLX (where X is 0, 1 or C) is obtained as follows using the reference pictures refIdxLXA, refIdxLXB, and refIdxLXC of the adjacent PU, A, B, and C blocks of the target PU.
(1) If refIdxLXA = refIdxLXB = refIdxLXC,
When refIdxLXA = -1, refIdxLX = 0
Otherwise, refIdxLX = refIdxLXA
(2) When refIdxLXA = refIdxLXB
When refIdxLXA = -1, refIdxLX = refIdxLXC
Otherwise, refIdxLX = refIdxLXA
(3) If refIdxLXB = refIdxLXC,
When refIdxLXB = -1, refIdxLX = refIdxLXA
Otherwise, refIdxLX = refIdxLXB
(4) When refIdxLXA = refIdxLXC
When refIdxLXA = -1, refIdxLX = refIdxLXB
Otherwise, refIdxLX = refIdxLXA
(5) When refIdxLXA = -1,
refIdxLX = min (refIdxLXB, refIdxLXC)
(6) When refIdxLXB = -1,
refIdxLX = min (refIdxLXA, refIdxLXC)
(7) When refIdxLXC = -1,
refIdxLX = min (refIdxLXA, refIdxLXB)
(8) In other cases,
refIdxLX = min (refIdxLXA, refIdxLXB, refIdxLXC)
Here, min is a function that takes a minimum value.
The coordinates of the blocks A and B are as follows.
A: (xP-1, yP + nPSH-1)
B: (xP + nPSW-1, yP-1)
The coordinates of the block C are any of the following C0, C1, and C2. If the PU corresponding to each position is available and other than intra, the refIdxLX of the PU at that position is set as refIdxLXC.
C0: (xP + nPSW-1, yP-1)
C1: (xP-1, yP + nPSH)
C2: (xP-1, yP-1)
When refIdxL0 and refIdxL1 are derived as described above, the motion compensation parameter of the position of the reference picture (xP + nPSW, yP + nPSH) indicated by refIdxL0 is used to determine the L0 motion vector, and the reference indicated by refIdxL1 A temporal merge candidate is derived by determining the L1 motion vector using the motion compensation parameter of the picture position (xP + nPSW, yP + nPSH). That is, the motion vectors mvLXCol [0] and mvLXCol [0] for each reference picture list LX (X = 0, X = 1 or X = C) are calculated from the reference picture indicated by the LX list and refIdxLX. Specifically, if the PU at the position of the reference picture (xP + nPSW, yP + nPSH) indicated by refIdxLX is unavailable or in the intra prediction mode, the motion vector mvLXCol of the temporal merge candidate LX [0], mvLXCol [1] is set to 0. Otherwise, that is, when PredFlagL0 of the PU is 0, the L1 motion vector MvL1 of the PU is used as the temporal merge candidate LX motion vectors mvLXCol [0] and mvLXCol [1]. In other cases, the L0 motion vector MvL0 of the PU is used as the temporal merge candidate LX motion vectors mvLXCol [0] and mvLXCol [1].

  Subsequently, the motion vector mvLXCol is scaled using the POC (Picture Order Count) of the current frame and the POC of the reference picture to obtain a final temporal merge candidate. The time merge candidate derivation unit 1464C stores the derived time merge candidate in the merge candidate storage unit 1462.

(Join merge candidate derivation unit)
In general, the merge merge candidate deriving unit 1464D derives a merge merge candidate by combining motion vectors of two different derived merge candidates already derived and stored in the merge candidate storage unit 1462.

  The merge merge candidate derivation unit 1464D increments the merge merge candidate count combCnt from 0, and merges until the number of elements in the merge candidate list matches the merge candidate derivation number, or until combCnt exceeds the maximum value (5). Candidates are derived and added to the end of the merge candidate list.

  The procedure for deriving the merge merge candidate corresponding to the specific merge merge candidate count combCnt is as follows.

First, the merge candidate combination list is referred to with combCnt as an index (combIdx), and two merge candidates used for the combination, that is, indexes l0CandIdx and L1CandIdx indicating the positions of the L0 merge candidate and the L1 merge candidate on the merge candidate list, respectively To derive. An example of the merge candidate combination list is shown in FIG. The merge merge candidate is generated by copying the motion compensation parameters of the L0 merge candidate to the reference image list L0 and the L1 merge candidate to the reference image list L1. Note that when any of the following conditions is satisfied, a merge merge candidate corresponding to combCnt is not derived.
-L0 reference image list use flag of L0 merge candidate is 0
-L1 reference image list use flag of L1 merge candidate is 0
The motion vectors or reference images of the L0 merge candidate and the L1 merge candidate match. The combined merge candidate is derived by the above procedure. In the motion compensation parameter corresponding to the merge merge candidate, both the L0 and L1 reference image list use flags are 1. That is, the merge merge candidate is a merge candidate that performs bi-prediction. Thus, in situations where bi-prediction is not applicable (eg, PUs in P slices), merge merge candidates are not included in the merge candidate list.

(Zero merge candidate derivation unit)
FIG. 15 is a diagram illustrating the operation of the zero merge candidate derivation unit 1464E. If the number of merge candidates in the merge candidate storage unit 1462 reaches the number of merge candidate derivations, the zero merge candidate derivation unit 1464E does not perform processing (zero merge candidates are not derived). On the other hand, if the number of merge candidates does not reach the number of merge candidate derivations, the zero merge candidate derivation unit 1464E generates a merge candidate having a zero vector until the number of merge candidates reaches the number of merge candidate derivations to generate a merge candidate list. Add to. That is, the index of the merge candidate to be referenced is zeroCand _m , and the L0 motion vector (mvL0zeroCand _m [0], mvL0zeroCand _m [1]) and the L1 motion vector (mvL1zeroCand _m [0], mvL1zeroCand _m [1]) are both A candidate such as 0 is derived. Here, the index zeroCand _m uses a value obtained by adding 1 to the value of the last index of the merge candidate list already derived. m is an index starting from 0, and is incremented by 1 when a zero merge candidate is added to the merge candidate list.

(Merge candidate derivation process flow)
An example of the merge candidate derivation process flow will be described with reference to FIG. FIG. 16 is a flowchart illustrating an example of the operation flow of the merge candidate derivation unit 146.

  As shown in FIG. 16, first, merge candidate S0 to merge candidate S2 are derived in spatial merge candidate derivation unit 1464A (S101).

  Subsequently, the merge candidate C0 is derived in the inter-layer merge candidate deriving unit 1464B (S102).

  Subsequently, the merge candidate T is derived in the time merge candidate deriving unit 1464C (S103).

  Subsequently, a merge candidate C is derived in the merge merge candidate deriving unit 1464D (S104). Note that S104 may be configured to be executed when the slice type determination unit 1463 determines that the slice type is a B slice.

  Finally, the merge candidate Z is derived in the zero merge candidate derivation unit 1464E (S105).

(Action / Effect)
As described above, the hierarchical video decoding device 1 decodes the hierarchically encoded data DATA in which image information relating to images of different quality for each layer is hierarchically encoded, and the target layer to be decoded Is a hierarchical video decoding device 1 that restores the decoded image POUT # T in the target layer using a predicted image generated by motion compensated prediction based on the restored motion information. In the target prediction unit that is the processing target of prediction image generation in, a reference layer that is a decoded layer is referred to, and decoding is performed in a region on the reference layer that corresponds to an undecoded region in a peripheral region that includes the target prediction unit Inter-layer merge candidate derivation unit 146 that derives motion information as a candidate for estimating motion information in the target prediction unit. It is configured to include B.

  Therefore, according to the above configuration, motion information on a reference area corresponding to an area that cannot be used because it is not decoded in the same layer can be added to the merge candidate list.

  Thereby, in the generation of the motion information candidate list in hierarchical encoding, there is an effect that the amount of motion information can be reduced by deriving various motion information as candidates.

(Modification of merge candidate derivation unit)
In the following, a preferred modification of the merge candidate derivation unit 146 will be described.

(Derive layer merge candidates from two or more positions)
As illustrated in FIGS. 17A to 17C, the inter-layer merge candidate derivation unit 1464B may derive inter-layer merge candidates from two or more positions on the reference region. Hereinafter, each of FIGS. 17A to 17C will be described in order.

  [1] As illustrated in FIG. 17A, the inter-layer merge candidate derivation unit 1464B may further target the inter-layer merge candidate derivation in addition to the position C0 in the upper left position C1 in the corresponding reference region.

  For example, the inter-layer merge candidate derivation unit 1464B uses the position C1 when the position C0 is unavailable. That is, as the priority of merge candidate derivation by the inter-layer merge candidate derivation unit 1464B, the lower right position C0 has priority over the position C1.

  According to the above modification, the inter-layer merge candidate deriving unit 1464B may be able to derive the inter-layer merge candidate even when the motion information at the position C0 cannot be referred to because the position C0 cannot be used.

  [2] As illustrated in FIG. 17B, the inter-layer merge candidate derivation unit 1464B merges the lower right adjacent blocks C0, C2, and C3 of the corresponding reference area and the upper left position C1 of the corresponding reference area. It may be used for derivation of candidates. It should be noted that the priority order for merging candidate derivation is illustratively in the order of positions C0, C1, C2, and C3.

  When the target PU or the corresponding reference area is small, there is a possibility that the pixel corresponding to the upper left position and the pixel corresponding to the lower right position in the corresponding reference area belong to the same motion information recording unit.

  By using the motion information of the lower right adjacent block, the possibility that the motion information can be derived from a recording unit different from the upper left portion in the corresponding reference area is increased.

  This increases the possibility of deriving different motion information from the upper left part of the corresponding reference area.

  [3] As illustrated in FIG. 17C, the inter-layer merge candidate derivation unit 1464B includes a lower right position C0 in the corresponding reference area, a lower right adjacent block C1 in the corresponding reference area, and an upper left position C1 in the corresponding reference area. The right adjacent block C3 and the lower adjacent block C4 of the corresponding reference region may be used for deriving inter-layer merge candidates.

  Further, the priorities of merging candidate derivation may be in the order of positions C0, C1, C2, C3, and C4.

  Further, if there is no motion information at each of the positions C0 to C4, it is not necessary to add an inter-layer merge candidate.

  In addition, when merge candidate derivation control unit 1461 performs a match determination between each of the motion compensation parameters at positions C1 to C4 and the motion compensation parameter at position C0, the motion compensation parameters do not match as a result of the determination. In addition, the inter-layer merge candidate derivation unit 1464B may derive a corresponding merge candidate.

  The operation of the function equalMotion (A, B) for determining whether the motion compensation parameters of the block A and the block B coincide with each other can be exemplarily defined as follows.

equalMotion (A, B) = (predFlagL0A == predFlagL0B) && (predFlagL1A == predFlagL1B) && mvL0A [0] == mvL0B [0] && mvL0A [1] == mvL0B [1] && mvL1A [1] = [0] && mvL1A [1] == mvL1B [1])
Here, predFlagL0A and predFlagL1A are 1 when the reference pictures L0 and L1 are used in the block A, respectively, and 0 otherwise. mvL0 [0] and mvL0 [1] are the horizontal motion vector and vertical motion vector of L0, and mvL1 [0] and mvL1 [1] are the horizontal motion vector and vertical motion vector of L1. In the case of block B, A is replaced with B.

  Further, the inter-layer merge candidate derivation unit 1464B derives merge candidates in the order of priority while performing the above-described matching determination, and adds inter-layer merge candidates until the number of unique merge candidates becomes two. Also good.

  According to the configuration according to the modified example, since the possibility that different motion information can be derived increases, the possibility that the motion information to be encoded is included in the merge candidate increases. As a result, the amount of code of motion information can be reduced.

(Match judgment)
The merge candidate derivation control unit 1461 may perform match determination of motion compensation parameters of merge candidates derived by each merge candidate derivation unit (hereinafter referred to as merge candidate match determination). Further, the merge candidate derivation control unit 1461 may store only the merge candidates determined to be unique as a result of the match determination in the merge candidate storage unit 1462. Further, the merge candidate derivation control unit 1461 may omit the determination of matching between the inter-layer merge candidate derived by the inter-layer merge candidate derivation unit 1464B and the merge candidate derived by another merge candidate derivation unit.

  For example, in the flowchart shown in FIG. 16, the temporal merge candidate derivation unit 1464C and the combined merge candidate derivation unit 1464D derive merge candidates after the inter-layer merge candidate derivation unit 1464B. Here, you may abbreviate | omit the coincidence determination with the time merge candidate and the merge merge candidate, and the merge candidate included in the merge candidate list.

  According to the configuration according to the modification, the inter-layer merge candidate derivation step (S102), the temporal merge candidate derivation step (S103), and the merged merge candidate derivation step (S104) can be executed in parallel.

  Also, the merge candidate derivation control unit 1461, when performing a match determination between the spatial merge candidate and the inter-layer merge candidate, performs a match determination only with the merge candidate S0 of the spatial merge candidate (the first merge candidate in the merge candidate list). Preferably it is done.

  Thereby, it is possible to reduce the possibility that the same merge candidate is included in the candidate list while suppressing a decrease in parallelism.

(Omit the merge candidate derivation step)
Another example of the merge candidate derivation process will be described with reference to FIG. FIG. 18 is a flowchart illustrating another example of the operation of the merge candidate derivation unit 146.

  Hereinafter, an example in which the temporal merge candidate derivation step is omitted when the inter-layer merge candidate derivation step is executed will be described with reference to FIG.

  As shown in FIG. 18, first, merge candidate S0 to merge candidate S2 are derived in spatial merge candidate derivation unit 1464A (S201).

  Next, the merge candidate derivation control unit 1461 determines whether or not the inter-layer merge candidate derivation is valid (S202). The merge candidate derivation control unit 1461 can determine that the inter-layer merge candidate derivation is effective in the following case (A) or (B), for example.

(A) When a flag included in SPS or PPS indicates that inter-layer merge prediction is valid; or
(B) A reference layer corresponding to the target PU exists, and the target PU and the reference layer are P slices or B slices.

  If the inter-layer merge candidate derivation is valid (YES in S202), the inter-layer merge candidate derivation unit 1464B derives the merge candidate C0 (S204).

  On the other hand, if the inter-layer merge candidate derivation is not valid (NO in S202), the merge candidate T is derived in the temporal merge candidate derivation unit 1464C (S203).

  That is, since the inter-layer merge candidate step (S204) and the temporal merge candidate derivation step (S203) are performed alternatively, when one of the two is executed, the other is omitted.

  Thereafter, the merge candidate C is derived in the merge merge candidate deriving unit 1464D (S205). Finally, the merge candidate Z is derived in the zero merge candidate derivation unit 1464E (S206).

  The code amount reduction width of the motion information when the temporal merge candidate and the inter-layer merge candidate are used together is not significantly different from the code amount reduction width of the motion information when either one is used.

  Therefore, when the inter-layer merge candidate is used, the code amount does not increase so much even if the temporal merge candidate is not used. Furthermore, if the derivation of the time merge candidate is omitted, the processing load related to the derivation of the time merge candidate can be reduced, and the memory resource can be saved.

  According to the above configuration, when using inter-layer merge candidates, derivation of temporal merge candidates is omitted.

  Therefore, the time merge candidate is not used when the inter-layer merge candidate is used. As a result, it is possible to reduce the amount of processing necessary for deriving the temporal merge candidate and the amount of memory for recording the motion vector of the reference image, while suppressing a significant increase in the amount of code of motion information.

  The reverse is also true. For example, when temporal merge candidates are used, the code amount does not increase so much even if inter-layer merge candidates are not used.

  According to the above configuration, when using temporal merge candidates, derivation of inter-layer merge candidates is omitted. Therefore, while not increasing the code amount of motion information so much, the processing amount required for the inter-layer merge candidate can be reduced.

  As described above, by selectively deriving one of the inter-layer merge candidate and the temporal merge candidate, it is possible to avoid a case where both the inter-layer merge candidate and the temporal merge candidate are used together. That is, it is possible to avoid a case where both merge candidates must be processed. Thereby, it is possible to reduce the worst value of the processing amount (the processing amount when the processing amount is the largest).

(Inter-layer intra candidate derivation unit)
A modification of the merge candidate derivation unit 146 will be described with reference to FIG. The individual merge candidate derivation unit 1464 may further include an inter-layer intra candidate derivation unit 1464F.

  In this modification, the concept of the merge candidate list is expanded. Specifically, the inter-layer intra candidate derivation unit 1464F signals inter-layer intra prediction (texture prediction) in the merge candidate list.

  That is, the inter-layer intra candidate derivation unit 1464F adds an inter-layer intra candidate that is a candidate for signaling inter-layer intra prediction to the merge candidate list. When the inter-layer intra candidate is selected from the merge candidate list, the inter-layer intra prediction unit 152C of the texture prediction unit 152 generates a prediction image based on the image obtained by up-sampling the decoded pixels of the reference layer.

  In this modification, the merge candidate derivation control unit 1461 may add each merge candidate to the merge candidate list in the following priority order.

1) Spatial merge candidate, 2) Temporal merge candidate, 3) Inter-layer merge candidate, 4) Join merge candidate, 5) Zero merge candidate, 6) Inter-layer intra candidate Further, the merge candidate derivation unit 146 derives the merge candidate For example, it can be performed by the following method.

  That is, as one method, merge candidates are added in the priority order of 1) to 6) until the maximum merge candidate derivation number in the merge candidate list is reached. As another method, the merge candidate derivation maximum value minus one merge candidate is added in the priority order of 1) to 5), and 6) the inter-layer merge candidate is merged after that. Add as

  According to the said structure, intra prediction between layers can be selected per PU. Thereby, compared with the case where a selection flag (base_mode_flag) is provided for every CU, side information can be reduced.

  Further, the merge candidate derivation control unit 1461 may add each merge candidate to the merge candidate list in the following priority order.

1) Spatial merge candidate, 2) Temporal merge candidate, 3) Inter-layer candidate, 4) Joined merge candidate, 5) Zero-merge candidate Here, 3) Inter-layer candidate is a merge candidate related to inter-layer prediction. Is an inter-layer merge candidate or an inter-layer intra candidate. A candidate to be added as an inter-layer candidate according to a prediction type of a CU (hereinafter referred to as a reference CU) to which a region on a reference layer corresponding to the target PU belongs is between an inter-layer merge candidate and an inter-layer intra candidate. Can be switched. This switching may be controlled by the merge candidate derivation control unit 1461.

  When the reference CU is an inter CU, the inter-layer merge candidate derivation unit 1464B derives the inter-layer merge candidate as an inter-layer candidate. When the inter-layer merge candidate is selected, the inter prediction unit 152A of the texture prediction unit 152 generates a prediction image by motion compensation prediction.

  When the reference CU is an intra CU, the inter-layer intra candidate derivation unit 1464F derives the inter-layer intra candidate as an inter-layer candidate. When the inter-layer intra candidate is selected, the inter-layer intra prediction unit 152C of the texture prediction unit 152 generates a prediction image based on an image obtained by up-sampling the decoded pixels of the reference layer.

  When the reference layer is an inter CU, the decoded image of the inter CU is not used for motion information prediction in inter-layer prediction in order to reduce the processing amount in the motion compensation process. Further, when the reference layer is an intra CU, there is no motion compensation information in the intra CU.

  According to the above configuration, candidates are generated using information available in the reference CU, depending on whether the reference CU is an inter CU or an intra CU.

  Thereby, in the merge candidate list, the number of merge candidates can be reduced compared to the case where the inter-layer merge candidate and the inter-layer intra candidate are different candidates.

  In addition, when adding merge candidates, it is possible to suppress the overhead of referring to information that is not available.

<< Details of PU partition type deriving section >>
Next, the detailed configuration of the PU partition type deriving unit 148 will be described with reference to FIG. FIG. 1 is a functional block diagram illustrating the configuration of the PU partition type deriving unit 148.

  As illustrated in FIG. 1, the PU partition type derivation unit 148 includes a reference CU setting unit 1481 and a PU partition estimation unit 1482.

  The reference CU setting unit 1481 sets an area on the reference layer corresponding to the target CU when decoding the target CU to be decoded, and refers to the set reference area at the time of PU partition estimation. Set the CU on the power reference layer. Hereinafter, the region set by the reference CU setting unit 1481 is referred to as a reference region, and the CU on the reference layer is referred to as a reference CU.

  The PU partition estimation unit 1482 refers to the reference CU set by the reference CU setting unit 1481, and estimates the partition type (estimated PU partition type) into PUs in the target CU of the target layer.

  More specifically, the PU partition estimation unit 1482 includes a lower right pixel position determination unit 1482A, an overlap region determination unit 1482B, and a partition type determination unit 1482C.

  The lower right pixel position determination unit 1482A determines the position of the position of the lower right pixel in the reference CU within the reference area.

  The overlapping area determination unit 1482B determines the degree of overlapping of an area where the reference CU and the reference area overlap (hereinafter referred to as an overlapping area). The overlapping degree of the overlapping region includes, for example, the size of the overlapping region, the size of the overlapping region, the relationship between the width and the height, and the inclusion relationship between the reference CU and the reference region.

  The division type determination unit 1482C determines the PU division type in the target CU according to the determination result of the lower right pixel position determination unit 1482A or the overlapping region determination unit 1482B.

  In the following, details of operations of the reference CU setting unit 1481 and the PU partition estimation unit 1482 will be described in order with reference to FIGS. 28 to 40.

(Operation of reference CU setting unit)
An example in which the reference CU setting unit 1481 sets a reference CU for the target CUtgtCU included in the target frame tgtFR1 on the target layer will be described with reference to FIG.

  As illustrated in FIG. 28, first, the reference CU setting unit 1481 sets a reference region refREG on the reference layer corresponding to the target CUtgtCU, which is a CU included in the reference frame refFR1 on the reference layer.

  That is, the reference frame refFR1 is illustratively a frame on the reference layer at the same time as the target frame tgtFR1.

  Further, here, the target CUtgtCU “corresponds” to the reference region refREG means that the spatial position of the target CUtgtCU in the target frame tgtFR1 on the target layer is the spatial position of the reference region refREG in the reference frame refFR1 on the reference layer. It means that it corresponds.

  Next, the reference CU setting unit 1481 sets a reference CU based on the reference area refREG.

  In the example shown in FIG. 28, the reference area refREG is included in the coding tree block refCTB1 on the reference layer. Also, the coding tree block refCTB1 includes refCU0, refCU1, refCU2, and refCU3, which are CUs having a hierarchical depth of 1. Also, refCU0 includes refCU00, refCU01, refCU02, and refCU03, which are split CUs with a hierarchical depth of 2.

  Here, there is a possibility that the reference region refREG has an overlapping portion with a plurality of CUs on the reference layer. That is, the reference area refREG may overlap with a plurality of CUs on the reference layer.

  In the example shown in FIG. 28, the reference area refREG has an overlapping part with refCU00, refCU01, refCU02, refCU03, refCU1, refCU2, and refCU3.

  Hereinafter, an example in which the reference CU setting unit 1481 mainly sets a CU on the reference layer including the pixel refCUPX corresponding to the upper left pixel tgtCUPX of the target CU as the reference CU will be described. That is, in the example shown in FIG. 28, refCU00 is the reference CU.

  The setting of the reference CU as described above will be described more specifically with reference to FIG. As shown in FIG. 29, the height and width of the target frame tgtFR1 on the target layer are hPic and wPic, respectively. Also, the height and width of the target CUtgtCU are hCu and wCu, respectively, and the coordinate representation in the target frame tgtFR1 at the position of the upper left pixel is (xCu, yCu). Further, the height and width of the reference frame refFR1 on the reference layer are set to hRefPic and wRefPic, respectively.

  When setting the reference CU corresponding to the upper left pixel, the reference CU setting unit 1481 sets the reference CU as follows. First, the reference CU setting unit 1481 derives the coordinates (xRefReg, yRefReg) of the reference region refREG by the following calculation. Note that scaleX and scaleY below are the ratios of the width and height of the target frame to the width and height of the reference frame.

scaleX = wPic / wRefPic
scaleY = hPic / hRefPic
xRefReg = xCu / scaleX
yRefReg = yCu / scaleY
Subsequently, the reference CU setting unit 1481 sets a CU including the coordinates (xRefReg, yRefReg) of the reference region refREG obtained as described above as a reference CU.

  The reference CU setting unit 1481 is not limited to the above, and may set the reference CU corresponding to the center pixel as follows. First, the reference CU setting unit 1481 derives the width and height (wRefReg, hRefReg) of the reference region refREG by the following calculation. Note that the method for deriving scaleX, scaleY, xRefReg and yRefReg shown below has already been described.

wRefReg = wCu / scaleX
hRefReg = hCu / scaleY
At this time, the coordinates (xRefC, yRefC) of the center pixel of the reference area are
xRefC = xRefReg + (hRefReg >> 1)
yRefC = yRefReg + (wRefReg >> 1)
Is obtained.

  Subsequently, the reference CU setting unit 1481 sets a CU including the coordinates (xRefC, yRefC) of the center pixel of the reference area obtained as described above as a reference CU.

  In the above example, the values of scaleX and scaleY are calculated on the assumption that the entire reference frame on the reference layer and the entire target frame on the target layer correspond spatially, but are not limited thereto. For example, a case where the partial region on the target layer spatially corresponds to the entire reference frame on the reference layer can be considered. In such a case, the values of scaleX and scaleY are calculated based on the position of the partial area on the target layer (offset from the upper left of the target frame) and the size of the partial area.

(About the operation of the PU partition estimation unit)
Next, an example in which the PU partition estimation unit 1482 estimates the PU partition type of the target CUtgtCU will be described with reference to FIGS. 30 to 40.

[When using a single reference CU (a CU including the upper left pixel of the reference area)]
First, the lower right pixel position determination unit 1482A determines the position of the lower right pixel refCUPX of the reference CUrefCU in the reference region refREG, and based on the determination result, the division type determination unit 1482C determines the PU partition type in the target CU. An example will be described. As described above, the upper left pixel of the reference area refREG is included in the reference CUrefCU.

[A] Lower right pixel is upper half of reference area As shown in FIG. 30A, the position of the lower right pixel refCUPX of the reference CUrefCU is positioned at the upper half of the reference area refREG (area above the straight line M1). When it is determined that there is a partition type determination unit 1482C, the PU partition type in the target CUtgtCU is estimated to be 2N × N (see FIG. 30B).

  When the lower right pixel of the reference CU is in the upper half position of the reference area, at least a horizontal CU boundary exists in the upper half of the reference area. In general, when a CU boundary exists, the possibility that the two regions sandwiching the boundary have different motion information is higher than when there is no CU boundary. Therefore, there is a high possibility that an object boundary exists in the upper half of the reference area. Therefore, by dividing the upper half area where the object boundary is likely to exist and the lower half area where the existence of the object boundary is unknown into different PUs, a PU with no object boundary is set. Can increase the possibility of

[B] Lower right pixel is lower left of reference area As shown in FIG. 31A, the position of the lower right pixel refCUPX of the reference CUrefCU is lower left in the reference area refREG (below the straight line M1 and from the straight line M2. When it is determined that it is in the left region), the division type determination unit 1482C estimates the PU division type in the target CUtgtCU as N × 2N (see FIG. 31B).

  When the lower right pixel of the reference CU is in the lower left position of the reference area, at least a vertical CU boundary exists in the left half of the reference area. Therefore, by using the PU division type that divides the left half and the right half of the reference area, it is possible to increase the possibility that a PU having no object boundary is set, as in the case of [A].

[C] Lower right pixel is lower right of reference area As shown in FIG. 32A, the position of the lower right pixel refCUPX of the reference CUrefCU is a lower right position in the reference area refREG (below the straight line M1 and a straight line). When it is determined that it is in the area to the right of M2, the division type determination unit 1482C estimates the PU division type in the target CUtgtCU as 2N × 2N (see FIG. 32B).

  When the lower right pixel of the reference CU is in the lower right position of the reference area, the area of the overlapping area of the reference area and the reference CU is large. Therefore, it is highly possible that most of the reference area is included in a single CU (reference CU), and there is a high possibility that uniform motion information is included. For this reason, encoding efficiency can be improved by using 2N × 2N, which is the most suitable PU partition when there is uniform motion in the reference region.

[D] Reference CU includes reference area As shown in FIG. 33A, when it is determined that the position of the lower right pixel refCUPX of the reference CUrefCU is at the lower right position outside the reference area refREG, that is, reference When the CUrefCU includes the reference region refREG, the division type determination unit 1482C estimates the PU division type in the target CUtgtCU as 2N × 2N (see FIG. 32B).

  When the reference CU includes a reference area, there is a high possibility of having uniform motion information within the reference area. For this reason, encoding efficiency can be improved by using 2N × 2N, which is the most suitable PU partition when there is uniform motion in the reference region.

  As described above, the PU division type can be derived by the position determination process of the lower right pixel without strictly deriving the position of the CU boundary in the reference region. Thereby, it is possible to derive the PU partition type with a relatively small processing amount.

  A method in which the lower right pixel position determination unit 1482A determines the position of the lower right pixel refCUPX will be described more specifically with reference to FIGS. 34 and 35. FIG.

  As shown in FIG. 34, the position of the upper left pixel of the reference region refREG is (xRefReg, yRefReg), and the size of the reference region refREG is wRefReg × hRefReg. Further, the position of the lower right pixel refCUPX of the reference CUrefCU is assumed to be (xRefCUBR, yRefCUBR).

  Each of the determination conditions [A] to [D] will be described as follows.

[A] Lower right pixel is upper half of reference area FIG. 35 is a diagram illustrating the relationship between the position of the lower right pixel refCUPX of the reference CUrefCU and the determination result.

  First, the lower right pixel position determination unit 1482A determines whether the expression (A1) is true or false.

yRefCuBR <yRefReg + (hRefReg >> 1) (A1)
In Expression (A1), it is determined whether or not the position of the lower right pixel refCUPX of the reference CUrefCU is in the upper half of the reference area refREG. That is, when the formula (A1) is true, the position of the lower right pixel refCUPX of the reference CUrefCU exists in the section A shown in FIG. Note that the labels A to D given to the sections shown in FIG. 35 correspond to the determination conditions [A] to [D]. For example, the position where the determination condition “[A] lower right pixel satisfies the upper half of the reference area” corresponds to the section A.

[B] If the lower right pixel is the lower left expression (A1) of the reference region, the lower right pixel position determination unit 1482A further determines the authenticity of the expression (A2).

xRefCuBR <xRefReg + (wRefReg >> 1) (A2)
In the expression (A2), it is determined whether or not the position of the lower right pixel refCUPX of the reference CUrefCU is at the lower left of the reference area refREG. That is, when the formula (A2) is true, the position of the lower right pixel refCUPX of the reference CUrefCU exists in the section B shown in FIG.

[C] If the lower right pixel is the lower right expression (A2) of the reference area, the lower right pixel position determination unit 1482A further determines the authenticity of the expression (A3).

xRefCuBR <xRefReg + wRefReg ||
xRefCuBR <yRefReg + hRefReg (A3)
In the equation (A3), it is determined whether or not the position of the lower right pixel refCUPX of the reference CUrefCU is at the lower right of the reference region refREG. That is, when the formula (A3) is true, the position of the lower right pixel refCUPX of the reference CUrefCU exists in the section C shown in FIG.

[D] Reference CU includes reference region When expression (A3) is false, lower right pixel position determination unit 1482A further determines the authenticity of expression (A4).

(xRefCuBR> = xRefReg + wRefReg &&
yRefCuBR> = yRefReg + hRefReg) (A4)
In Expression (A4), it is determined whether or not the reference CUrefCU includes the reference area refREG. That is, when the formula (A4) is true, the position of the lower right pixel refCUPX of the reference CUrefCU exists in the section D shown in FIG.

[Determination by Degree of Overlap of Area Overlapping Reference CU and Reference Area (Overlapping Area)]
Next, an example in which the overlapping region determination unit 1482B determines the degree of overlapping between the reference CUrefCU and the reference region refREG, and the partition type determination unit 1482C determines the PU partition type in the target CU based on the determination result. Will be described.

  In FIG. 36 as well, as in FIG. 34, the position of the upper left pixel of the reference region refREG is (xRefReg, yRefReg). Further, the position of the lower right pixel refCUPX of the reference CUrefCU is assumed to be (xRefCUBR, yRefCUBR). Further, the position of the lower right pixel of the reference area refREG is (xRefRegBR, yRefRegBR), and the position of the upper left pixel of the reference CUrefCU is (xRefCU, yRefCU).

[A ′] Reference CUrefCU includes reference region refREG As shown in FIG. 36A, when it is determined that reference CUrefCU includes reference region refREG (when reference region refREG matches the overlapping region), division The type determination unit 1482C estimates the PU partition type in the target CUtgtCU as 2N × 2N (see FIG. 36B). In the determination, for example, the position of the upper left pixel of the reference CUrefCU is compared with the position of the upper left pixel of the reference area refREG, and the width and height of the reference CUrefCU are compared with the width and height of the reference area refREG. This can be done.

  Alternatively, the width (wOver) of the overlapping area and the width (hOver) of the overlapping area may be calculated, and the determination may be performed based on these values. wOver and hOver can be calculated using the following equations.

wOver = Min (xRefRegBR, xRefCuBR)-Max (xRefReg, xRefCu)
hOver = Min (yRefRegBR, yRefCuBR)-Max (yRefReg, yRefCu)
Here, when the reference CUrefCU includes the reference area refREG, the overlapping area and the reference area refREG coincide with each other, and the following relational expression is established.

wOver == wRefReg && hOver == hRefReg
When the reference CUrefCU has a specific restriction on the positional relationship of the reference region refREG, the values of wOver and hOver can be calculated by a simplified calculation. For example, when the reference CUrefCU includes the upper left pixel of the reference area refREG, the upper left pixel of the reference CUrefCU is always located at the upper left than the upper left pixel of the reference area refREG. Therefore, in such a case, the values of wOver and hOver can be calculated by the following equations, respectively.

wOver = Min (xRefRegBR, xRefCuBR)-xRefReg
hOver = Min (yRefRegBR, yRefCuBR)-yRefReg
[B ′] When the width of the overlapping area is greater than or equal to the height thereof As shown in FIG. 37 (a) or (b), the overlapping area of the reference CUrefCU and the reference area refREG When it is determined that the width is equal to or greater than the height of the overlapping region, the division type determination unit 1482C estimates the PU division type in the target CUtgtCU as 2N × N (see FIG. 37C). In FIGS. 37A and 37B, wOver is abbreviated as “w” and hOver is abbreviated as “h”.

  Specifically, the overlapping area determination unit 1482B determines whether the expression (B1) is true or false.

wOver> = hOver (B1)
If the formula (B1) is true, it is determined that the width of the overlapping region is equal to or greater than the height.

[C ′] If the width of the overlapping region is less than its height,
In other cases (when neither [A ′] nor “B ′”), as shown in FIG. 38 (a) or (b), the width of the overlapping region of the reference CUrefCU and the reference region refREG is When it is determined that the height is less than the overlap area, the division type determination unit 1482C estimates the PU division type in the target CUtgtCU as N × 2N (see FIG. 38C).

  If the above formula (B1) is false, the overlapping area determination unit 1482B may determine that the width of the overlapping area is less than its height.

  In the above description, the position of the upper left pixel (xRefReg, yRefReg) of the reference area refREG and the position of the lower right pixel refCUPX of the reference CUrefCU (xRefCUBR, yRefCUBR) are used for comparison. However, the present invention is not limited to this, and the above determination can also be applied to the case where a reference CU corresponding to a pixel position other than the upper left pixel of the reference region refREG is used.

(Action / Effect)
As described above, the hierarchical moving image decoding apparatus 1 decodes hierarchically encoded data in which image information related to images of different quality for each layer is hierarchically encoded, in the target layer to be decoded. The hierarchical video decoding device 1 for restoring an image, wherein a target CU in the target layer overlaps with the reference region based on a CU having an overlapping portion with a reference region corresponding to the target CU in the reference layer. A reference CU setting unit 1481 for setting a reference CU to be determined, and a PU partition estimation unit 1482 for determining a PU partition type for the target CU according to the degree of overlap between the reference region and the reference CU. It is the composition provided with.

  Therefore, it is possible to reduce the amount of processing for deriving the PU partition type.

(Modification of PU partition type deriving unit)
Below, the preferable modification of PU division | segmentation type derivation | leading-out part 148 is demonstrated.

(Judgment using PU partition type of reference CU)
In accordance with the determination by the lower right pixel position determination unit 1482A, the division type determination unit 1482C may perform a determination using the PU division type of the reference CU.

[When exact calculation is omitted]
Specifically, if it is determined that the above-described case of [A] to [C] described in “When using a single reference CU” is used, it is as described above. That is, when it is determined that the lower right pixel refCUPX of the reference CUrefCU is included in the reference region refREG, the division type determination unit 1482C determines the PU partition type of the target CU without using the PU partition type of the reference CU.

On the other hand, when it determines with corresponding to said [D], division type determination part 1482C determines the PU division type of object CU as follows using the PU division type of reference CU. That is,
[D1] When the PU partition type of the reference CU is 2N × 2N:
The division type determination unit 1482C determines the PU division type of the target CU as 2N × 2N.

[D2] When the PU partition type of the reference CU is 2N × N or 2N × nU or 2N × nD:
The division type determination unit 1482C determines the PU division type of the target CU as 2N × N.

[D3] When the PU partition type of the reference CU is N × 2N or nL × 2N or nR × 2N:
The division type determination unit 1482C determines the PU division type of the target CU as N × 2N.

  According to the above configuration, when there is no partition boundary (PU boundary) in the reference CU, 2N × 2N which is a PU partition type without a partition boundary is determined as a PU partition type in the target CU (in the case of [D1]). .

  Also, when there is a possibility that a PU boundary exists in the reference CU (in the case of [D2] or [D3]), the exact calculation of the position of the PU boundary in the reference CU is omitted, and the boundary is either vertical or horizontal. Only the possibility is determined, and the PU partition type in the target CU is determined.

  Note that in the case of “determination based on the degree of overlap of the reference CU and the reference region overlapping”, the determinations of [D1] to [D3] may be performed in the case of [A ′].

[When performing exact calculations]
The configuration when the strict calculation is omitted is as described above, but the division type determination unit 1482C determines the PU partition type in the target CU by strictly calculating the position of the PU boundary in the reference CU. It may be a configuration. Hereinafter, a configuration example of such a division type determination unit 1482C will be shown.

  In the case where it is determined that it corresponds to [A] to [C] described in the above “in the case of using a single reference CU”, it is as described above.

On the other hand, when it determines with corresponding to said [D], division type determination part 1482C determines the PU division type of object CU as follows using the PU division type of reference CU. That is,
[D1] When the PU partition type of the reference CU is 2N × 2N:
The division type determination unit 1482C determines the PU division type of the target CU as 2N × 2N.

[D2] When the PU partition type of the reference CU is 2N × N or 2N × nU or 2N × nD:
As illustrated in FIGS. 39A, 39B, and 39C, when the PU partition type of the reference CU is 2N × N or 2N × nU or 2N × nD, the partition type determination unit 1482C performs the following calculation: Then, the position of the PU boundary in the reference CU (that is, the position of the horizontal PU boundary) is strictly calculated.

  Here, as shown in FIG. 40, the position of the upper left pixel of the reference CUrefCU is (xRefCu, yRefCu), and the height and width of the reference CUrefCU are hRefCu, wRefCu, respectively. In addition, the reference position of the PU boundary in the reference CUrefCu is a position (xRefCu, yPub) where the left side of the reference CU intersects the PU boundary (yPub is the position of the vertical PU boundary). Further, the position of the upper left pixel of the reference area refREG is (xRefReg, yRefReg), and the height and width of the reference area are hRefReg and wRefReg, respectively.

  [1] The division type determination unit 1482C derives the position yPub in the y direction of the horizontal PU boundary as follows.

yPub = yRefCu + (hRefCu * bPos)
bPos = 0.25 (when 2NxnU)
0.5 (2NxN)
0.75 (for 2NxnD)
[2] The partition type determination unit 1482C determines the PU partition type as follows according to the position in the reference area refREG of yPub. In the following, d = yPub−yRefReg. That is, here, the division type determination unit 1482C determines the PU division type according to the position in the reference region refREG of the horizontal PU boundary of the reference CUrefCU.

[2-1] When d ≦ 0 or d ≧ hRefReg This is a case where the reference area refREG is included in any partition of the reference CUrefCU. The division type determination unit 1482C determines the PU division type as 2N × 2N.

[2-2] 0 <d <0.25 * hRefReg This is a case where the PU boundary of the reference CUrefCU crosses the position of 1/4 or more above the reference region refREG. The division type determination unit 1482C determines the PU division type as 2N × nU.

[2-3] In the case of 0.25 * hRefReg ≦ d <0.75 * hRefReg This is a case where the PU boundary of the reference CUrefCU crosses the vicinity of the center of the reference region refREG (position of upper 1/4 to lower 1/4). . The division type determination unit 1482C determines the PU division type as 2N × N.

[2-4] In the case of 0.75 * hRefReg ≦ d <hRefReg This is a case where the PU boundary of the reference CUrefCU crosses the position of ¼ or more below the reference region refREG. The division type determination unit 1482C determines the PU division type as 2N × nD.

[D3] When the PU partition type of the reference CU is N × 2N or nL × 2N or nR × 2N:
The division type determination unit 1482C strictly calculates the position of the PU boundary (that is, the position of the vertical PU boundary) in the reference CU. As the calculation method, the calculation method shown in FIG. 40 applied to the vertical PU boundary can be used, and thus detailed description thereof is omitted.

(Judgment when resolution ratio is small)
When the ratio of the resolution of the target layer to the resolution of the reference layer is equal to or less than a predetermined value (for example, 1.5 or less), the division type determination unit 1482C increases or decreases the size of the reference CU and the size of the target CU. The PU partition type may be determined accordingly.

  For example, when the size of the reference CU is larger than the size of the target CU, the division type determination unit 1482C may determine the PU division type as 2N × 2N.

  For example, when the size of the reference CU is equal to or smaller than the size of the target CU, the division type determination unit 1482C may set the PU division type of the reference CU as the PU division type.

  According to the above configuration, the determination process can be simplified.

(Limit by block size)
It is preferable to suppress the generation of a small size PU due to PU partitioning. Therefore, the division type determination unit 1482C may be configured to always use 2N × 2N as the PU division type in a predetermined size CU (for example, 8 × 8 CU). That is, the division type determination unit 1482C may prohibit division in a CU having a predetermined size.

  According to the above configuration, it is possible to reduce the proportion of small-sized PU inter-CUs that become a bottleneck for processing, thereby reducing the average processing amount.

  Note that the partition type determination unit 1482C may derive only a symmetric partition as a PU partition type in a CU having a predetermined size. Further, the division type determination unit 1482C may limit deriving N × N as a PU division pattern.

(Asymmetric partition limitation)
When determining the PU partition type from the reference layer, the partition type determining unit 1482C may be configured not to derive the asymmetric partition as the PU partition type. For example, when estimating the PU partition type of the target CU from the PU partition type of the reference CU, the partition type determining unit 1482C may be configured to derive a symmetric partition according to the directionality of the boundary as the PU partition type.

  In other words, the determination using the reference CU PU partition type is not limited to the case of [D], and can also be applied to [A] to [C]. In addition, when the above-described determination using the PU partition type of the reference CU is also applied to [A] to [C], the partition type determination unit 1482C may derive a symmetric partition as a PU partition type.

  According to the above configuration, since it is not necessary to determine the exact position of the vertical or horizontal boundary, the processing can be simplified.

(Selection / setting of reference CU)
In the above description, the configuration in which the reference CU setting unit 1481 sets the CU including the upper left pixel of the target CU as the reference CU among the plurality of CUs having an overlapping portion with the reference region refREG has been described.

  However, the present invention is not limited to the above configuration. More generally, the reference CU setting unit 1481 may be configured to set a single reference CU based on a predetermined criterion from among a plurality of CUs having an overlapping portion with the reference region refREG.

  For example, the reference CU setting unit 1481 may set a CU on a reference layer including a pixel corresponding to a pixel at a predetermined position included in the target CU among the plurality of CUs as a reference CU. As a specific example, the reference CU setting unit 1481 may set a CU on the reference layer including a pixel corresponding to the central pixel included in the target CU as the reference CU.

  In another specific example, the reference CU setting unit 1481 may use, as a reference CU, a CU having the largest area among a plurality of CUs having overlapping portions with the reference region refREG.

  According to the above configuration, the CU on the reference layer including the pixel corresponding to the central pixel included in the target CU, or the CU having the largest area among the plurality of CUs may be the same PU partition type as the target CU. Therefore, the estimation accuracy can be improved.

  Further, the reference CU setting unit 1481 may generate a virtual single CU from a plurality of CUs having overlapping portions with the reference region refREG. Specifically, the reference CU setting unit 1481 extracts representative properties in a plurality of CUs from CUs around the reference region, generates a virtual single CU based on the extracted properties, and generates a reference code. It may be a unit. For example, the reference CU setting unit 1481 can generate the virtual single CU based on the size and shape of a plurality of CUs or by combining these pieces of information. The virtual single coding unit is generated so as to be an area overlapping the reference area.

  Further, the reference CU setting unit 1481 may use a plurality of CUs having overlapping portions with the reference region refREG as a reference CU list.

(Supplement to PU partition estimation unit)
In the description of the PU partition estimation unit 1482 in FIG. 1 described above, the PU partition estimation unit 1482 has been described as a configuration including both the lower right pixel position determination unit 1482A and the overlapping region determination unit 1482B, but one of the determination units is always used. The PU partition estimation unit 1482 may be configured to include one of the lower right pixel position determination unit 1482A and the overlap region determination unit 1482B.

(Other variations)
In the following, other modifications will be described.

(Syntax table configuration example)
A configuration example of the syntax for encoding the base skip CU will be described below. The outline of the configuration is as follows. First, a base mode flag (base_mode_flag) indicating whether to use reference layer information is encoded. When the base mode flag is “true”, estimation of the PU partition type by the PU partition type deriving unit 148 described above is used to derive the PartMode.

  Depending on the syntax configuration, estimation other than the PU partition type (for example, pred_mode) may be executed.

[Configuration to encode base_mode_flag with coding_unit]
A configuration example in which base_mode_flag is encoded with coding_unit will be described with reference to FIGS. 41 and 42.

  FIG. 42 shows a configuration example of syntax when base_mode_flag is encoded with coding_unit.

  As shown in FIG. 42, base_mode_flag may be encoded immediately after skip_flag in coding_unit (SYN11). In the following, as a technical matter related to coding of base_mode_flag, omission and derivation of prediction mode pred_mode and PU partition type part_mode will be considered together.

  More specifically, with reference to FIG. 42, in the configuration of the encoded data, the base mode flag (base_mode_flag) is arranged after the skip flag (skip_flag) and before the prediction mode flag (pred_mode_flag).

  Further, when the base mode flag (base_mode_flag) is “true”, the prediction mode flag (pred_mode_flag) and the partition type (part_mode) are not arranged (corresponding to SYN12 and SYN13, respectively).

  According to the configuration example of the syntax shown in FIG. 42, decoding in base skip CU (decoding related to base_mode_flag) is performed as follows.

  (1) The hierarchical video decoding device 1 decodes base_mode_flag.

  (2) When base_mode_flag is “true”, the hierarchical video decoding device 1 derives PredMode and PartMode by estimation (infer).

  (3) On the other hand, when base_mode_flag is “false”, the hierarchical video decoding device 1 decodes the syntax element values of pred_mode_flag and part_mode from the encoded data, and based on the decoded syntax value, PredMode And PartMode are derived.

  The above decoding process is represented in a table format as shown in FIG. FIG. 41 is a table showing the relationship between syntax element values and CU types. In the figure, “-” indicates a syntax element that does not need to be decoded in the CU type.

  According to the above syntax configuration, as shown in FIG. 41, in the case of a base skip CU, since pred_mode_flag and part_mode are not decoded, encoding efficiency is improved.

(Derivation of prediction mode (PredMode))
Below, the derivation | leading-out method of prediction mode (PredMode) is demonstrated.

[Derivation Method 1]
When the base layer is an I slice, the hierarchical video decoding device 1 may always use intra prediction (MODE_INTRA) in the target CU.

  On the other hand, when the base layer is not an I slice, the hierarchical video decoding device 1 may use the PredMode of the reference CU. For example, a CU on the reference layer including the upper left pixel of the target CU can be used as the reference CU.

[Derivation Method 2]
When the base layer is an I slice, the hierarchical video decoding device 1 may always use intra prediction (MODE_INTRA) in the target CU.

  On the other hand, when the base layer is not an I slice, the hierarchical video decoding device 1 uses inter prediction (MODE_INTER).

  In the derivation methods 1 and 2, the condition “when the base layer is not an I slice” may be the condition “when the base layer is IDR (Instantaneous Decoding Refresh)”. IDR is initialization of a reference picture, and an IDR picture is a picture whose reference has been reset. That is, in an IDR picture, pictures subsequent to that picture can be correctly decoded without referring to information prior to that picture (slice).

(Derivation of merge_flag)
A configuration example in which merge_flag is encoded according to determination of base_mode_flag in prediction_unit will be described using FIG. 43.

  FIG. 43 shows a configuration example of syntax when encoding merge_flag after determining base_mode_flag in prediction_unit.

  As shown in FIG. 43, base_mode_flag may be determined immediately before mereg_flag in prediction_unit (SYN 21). Further, when base_mode_flag is “true”, merge_flag need not be arranged.

  That is, each PU included in the base skip CU in which base_mode_flag is “true” may be configured to always use the merge mode (merge_flag = true).

  According to the above configuration, the three flags pred_mode, part_mode, and merge_flag can be omitted. Thereby, the code amount of these three flags can be reduced, and the encoding efficiency is improved.

(Flow of CU decoding process in hierarchical video decoding apparatus)
An example of the flow of the CU decoding process related to the prediction parameter in the hierarchical video decoding device 1 will be described with reference to FIGS. 44, 45, and 46. 44, 45, and 46 are flowcharts illustrating examples of the flow of the decoding process regarding the prediction parameters of the base skip CU, the skip CU, and the inter CU, respectively.

  First, an example of the flow of the base skip CU decoding process in the hierarchical video decoding device 1 will be described with reference to FIG.

  When the target CU is a base skip CU, the reference CU setting unit 1481 sets a reference CU corresponding to the target CU (S301).

  Subsequently, the PU partition estimation unit 1482 estimates the PU partition type in the target CU by inter-layer prediction (S302). Specifically, the PU partition type is estimated by the partition type determination unit 1482C based on the determination result in the lower right pixel position determination unit 1482A or the determination result in the overlap region determination unit 1482B.

  Subsequently, when a PU is divided and a target PU is set according to the estimated PU partition type (S303), the merge candidate derivation unit 146 derives a merge candidate list for the target PU (S304). For example, the merge candidate derivation unit 146 derives an inter-layer merge candidate in derivation of the merge candidate list. However, how much the inter-layer prediction is performed can be arbitrarily set. For example, in the base skip CU, only the PU partition type may be the target of inter-layer prediction. That is, it is possible to adopt a configuration in which only the intra-layer merge candidates are derived in S304. In addition, in the base skip CU, instead of the S301 “reference CU setting” and S302 “PU partition estimation” processing, S401 “PU partition setting to 2N × 2N” described later may be executed. Further, PU division may be estimated in CUs other than the base skip CU.

  Subsequently, the merge information restoration unit 147 selects a merge candidate from the derived merge candidate list (S305).

  Here, if the target PU is not the final PU in the processing order in the target CU (NO in S306), the processes of S303 to S305 are repeatedly executed.

  Thereafter, when the processing is completed up to the last PU in the processing order in the target CU (YES in S306), the base skip CU decoding processing ends.

  Next, an example of the flow of the skip CU decoding process in the hierarchical video decoding device 1 will be described with reference to FIG.

  When the target CU is a skip CU, the prediction parameter restoration unit 14 sets the PU partition type to 2N × 2N (that is, no PU partition) (S401) and sets the target PU (S402).

  Subsequently, the merge candidate list derivation unit 146 derives a merge candidate list for the target PU (S403). Further, the merge information restoration unit 147 selects a merge candidate from the derived merge candidate list (S404), and then the skip CU decoding process ends.

  Next, an example of the flow of the inter CU decoding process in the hierarchical video decoding device 1 will be described with reference to FIG.

  When the target CU is an inter CU, the PU partition type is decoded from the encoded data (S501). Further, the partition to the PU in the target CU is set according to the decoded PU partition type, and the target PU is set (S502).

  Subsequently, it is determined whether or not the target PU is a merge PU (S503). When the target PU is a merge PU (YES in S503), the merge candidate derivation unit 146 derives a merge candidate list (S504), and the merge information restoration unit 147 selects a merge candidate from the merge candidate list (S505).

  On the other hand, when the target PU is not a merge PU (NO in S503), PU information is decoded from the encoding parameters (S507).

  After the process of S505 or S507, if the target PU is not the final PU in the process order in the target CU (NO in S506), the processes of S502 to S507 are repeatedly executed.

Thereafter, when the processing is completed up to the final PU in the processing order in the target CU (YES in S506), the inter-CU decoding processing is ended.
[Hierarchical video encoding device]
Below, the structure of the hierarchy moving image encoder 2 which concerns on this embodiment is demonstrated with reference to FIGS.

(Configuration of Hierarchical Video Encoding Device)
The schematic configuration of the hierarchical video encoding device 2 will be described with reference to FIG. FIG. 22 is a functional block diagram showing a schematic configuration of the hierarchical video encoding device 2. The hierarchical video encoding device 2 encodes the input image PIN # T of the target layer with reference to the reference layer encoded data DATA # R to generate hierarchical encoded data DATA of the target layer. It is assumed that the reference layer encoded data DATA # R has been encoded in the hierarchical video encoding apparatus corresponding to the reference layer.

  As shown in FIG. 22, the hierarchical video encoding device 2 includes a prediction parameter determination unit 21, a prediction information generation unit 22, a base decoding unit 23, a texture information generation unit 24, a variable length encoding unit 25, and a NAL multiplexing unit. 26.

  The prediction parameter determination unit 21 determines a prediction parameter used for prediction of a predicted image and other encoding settings based on the input image PIN # T.

  The prediction parameter determination unit 21 performs encoding settings including prediction parameters as follows.

  First, the prediction parameter determination unit 21 generates a CU image for the target CU by sequentially dividing the input image PIN # T into slice units, tree block units, and CU units.

  In addition, the prediction parameter determination unit 21 generates encoded information (sometimes referred to as header information) based on the result of the division process. The encoding information includes (1) tree block information that is information about the size and shape of the tree block belonging to the target slice and the position in the target slice, and (2) the size, shape, and target of the CU belonging to each tree block. CU information which is information about the position in the tree block.

  Further, the prediction parameter determination unit 21 refers to the CU image, the tree block information, and the CU information, and predicts the prediction type of the target CU, the division information of the target CU into the PU, and the prediction parameter (the target CU is an intra CU). If so, the intra prediction mode, and in the case of an inter CU, a motion compensation parameter in each PU is derived.

  The prediction parameter determination unit 21 includes (1) a prediction type of the target CU, (2) a possible division pattern for each PU of the target CU, and (3) a prediction mode that can be assigned to each PU (if it is an intra CU). The cost is calculated for all combinations of the intra prediction mode and the motion compensation parameter in the case of inter CU), and the prediction type, division pattern, and prediction mode with the lowest cost are determined.

  The prediction parameter determination unit 21 supplies the encoded information and the prediction parameter to the prediction information generation unit 22 and the texture information generation unit 24. Although not shown for simplicity of explanation, the above-described encoding setting determined by the prediction parameter determination unit 21 can be referred to by each unit of the hierarchical video encoding device 2.

  The prediction information generation unit 22 generates prediction information including a syntax value related to the prediction parameter based on the prediction parameter supplied from the prediction parameter determination unit 21 and the reference layer encoded data DATA # R. The prediction information generation unit 22 supplies the generated prediction information to the variable length encoding unit 25. The prediction information generation unit 22 can refer to motion information stored in a frame memory 244 (described later) included in the texture information generation 24 when restoring the prediction parameter.

  Since the base decoding unit 23 is the same as the base decoding unit 16 of the hierarchical video decoding device 1, the description thereof is omitted here.

  The texture information generation unit 24 generates transform coefficient information including transform coefficients obtained by orthogonal transform / quantization of the prediction residual obtained by subtracting the predicted image from the input image PIN # T. The texture information generation unit 24 supplies the generated transform coefficient information to the variable length encoding unit 25. In the texture information generation 24, information on the restored decoded image is stored in an internal frame memory 244 (described later).

  The variable length coding unit 25 performs variable length coding on the prediction information supplied from the prediction information generation unit 22 and the transform coefficient information supplied from the texture information generation unit 24 to generate target layer encoded data DATA # T. The variable length encoding unit 25 supplies the generated target layer encoded data DATA # T to the NAL multiplexing unit 26.

  The NAL multiplexing unit 26 stores the target layer encoded data DATA # T and the reference layer encoded data DATA # R supplied from the variable length encoding unit 25 in the NAL unit, and thereby performs hierarchical video that has been NAL multiplexed. Image encoded data DATA is generated and output to the outside.

  Hereinafter, details of each of the prediction information generation unit 22 and the texture information generation unit 24 will be described.

(Prediction information generator)
The detailed configuration of the prediction information generation unit 22 will be described with reference to FIG. FIG. 23 is a functional block diagram illustrating the configuration of the prediction information generation unit 22.

  As shown in FIG. 23, the prediction information generation unit 22 includes a prediction type selection unit 221, a switch 222, an intra prediction mode derivation unit 223, a motion vector candidate derivation unit 224, a motion information generation unit 225, a merge candidate derivation unit (interlayer candidate). (Derivation means) 226, merge information generation unit 227, and PU partition type derivation unit 228.

  The prediction type selection unit 221 sends a switching instruction to the switch 222 according to the CU type or PU type, and controls the prediction parameter derivation process. Specifically, it is as follows.

  When intra CU or intra PU is designated, the prediction type selection unit 221 controls the switch 222 so that prediction information can be derived using the intra prediction mode deriving unit 223.

  When either inter CU (no merging) or inter PU (no merging) is specified, the prediction type selection unit 221 uses the motion information generation unit 225 to control the switch 222 so that a prediction parameter can be derived.

  When any one of the base skip CU, the base merge PU, the skip CU, and the merge PU is designated, the prediction type selection unit 221 controls the switch 222 so that a prediction parameter can be derived using the merge information generation unit 227.

  The switch 222 supplies the prediction parameter to any of the intra prediction mode deriving unit 223, the motion information generating unit 225, and the merge information generating unit 227 in accordance with an instruction from the prediction type selecting unit 221. A prediction parameter is derived at a supply destination of the prediction information.

  The intra prediction mode deriving unit 223 derives a syntax value related to the intra prediction mode. That is, the intra prediction mode restoration unit 143 generates a syntax value related to the prediction mode as the prediction information.

  The motion vector candidate derivation unit 224 uses the base decoding information to derive an estimated motion vector candidate by intra-layer motion estimation processing or inter-layer motion estimation processing. The motion vector candidate derivation unit 224 supplies the derived motion vector candidates to the motion information generation unit 225.

  The motion information generation unit 225 generates a syntax value related to motion information in each inter prediction partition that is not merged. That is, the motion information restoration unit 145 generates a syntax value related to motion information as prediction information. Specifically, the motion information generation unit 225 derives corresponding syntax element values inter_pred_flag, mvd, mvp_idx, and refIdx from the motion compensation parameter in each PU.

  Specifically, when the target PU is a base merge PU, the motion information generation unit 225 derives the syntax value based on the motion vector candidates supplied from the motion vector candidate derivation unit 224.

  On the other hand, when the target CU (PU) is an inter CU (inter PU) that does not perform merging, the motion information restoration unit 145 derives the syntax value based on the motion information included in the prediction parameter.

  The merge candidate derivation unit 226 uses motion information similar to a motion compensation parameter in each PU using decoded motion information supplied from a frame memory 155 described later and / or base decoding information supplied from the base decoding unit 23, and the like. A merge candidate having a compensation parameter is derived. The merge candidate derivation unit 226 supplies the derived merge candidates to the merge information generation unit 227. The configuration of the merge candidate derivation unit 226 is the same as the configuration of the merge candidate derivation unit 146 included in the hierarchical video decoding device 1, and thus the description thereof is omitted.

  The merge information generation unit 227 generates a syntax value related to motion information regarding each inter prediction partition to be merged. That is, the merge information generation unit 227 generates a syntax value related to motion information as prediction information. Specifically, the merge information generation unit 227 outputs a syntax element value merge_idx that specifies a merge candidate having a motion compensation parameter similar to the motion compensation parameter in each PU.

  The PU partition type deriving unit 228 estimates the PU partition type to the PU of the target CU in the target layer using the encoded information and the base decoding information. Since the configuration of the PU partition type deriving unit 228 is the same as the configuration of the PU partition type deriving unit 148 included in the hierarchical image decoding device 1, detailed description thereof is omitted.

(Texture information generator)
A detailed configuration of the texture information generation unit 24 will be described with reference to FIG. FIG. 24 is a functional block diagram illustrating the configuration of the texture information generation unit 24.

  As shown in FIG. 24, the texture information generation unit 24 includes a texture prediction unit 241, a subtractor 242, an orthogonal transformation / quantization unit 243, an inverse orthogonal transformation / inverse quantization unit 244, an adder 245, a loop filter unit 246, And a frame memory 247.

  The subtractor 242 generates a prediction residual D by subtracting the prediction image supplied from the texture prediction unit 241 from the input image PIN # T. The subtractor 242 supplies the generated prediction residual D to the transform / quantization unit 243.

  The orthogonal transform / quantization unit 243 generates a quantized prediction residual by performing orthogonal transform and quantization on the prediction residual D. Here, the orthogonal transform refers to an orthogonal transform from the pixel region to the frequency region. Further, examples of orthogonal transform include DCT transform (Discrete Cosine Transform), DST transform (Discrete Sine Transform), and the like. In addition, the specific quantization process is as described above, and the description thereof is omitted here. The orthogonal transform / quantization unit 243 supplies the generated transform coefficient information including the quantized prediction residual to the inverse transform / inverse quantization unit 244 and the variable length coding unit 25.

  The texture prediction unit 241, the inverse orthogonal transform / inverse quantization unit 244, the adder 245, the loop filter unit 246, and the frame memory 247 are respectively a texture prediction unit 152, an inverse orthogonal transform / Since it is similar to the inverse quantization unit 151, the adder 153, the loop filter unit 154, and the frame memory 155, the description thereof is omitted here. However, the texture prediction unit 241 supplies the predicted image not only to the adder 245 but also to the subtractor 242.

(Application example to other hierarchical video encoding / decoding systems)
The above-described hierarchical moving image encoding device 2 and hierarchical moving image decoding device 1 can be used by being mounted on various devices that perform transmission, reception, recording, and reproduction of moving images. The moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.

  First, it will be described with reference to FIG. 25 that the above-described hierarchical video encoding device 2 and hierarchical video decoding device 1 can be used for transmission and reception of video.

  FIG. 25A is a block diagram illustrating a configuration of a transmission device PROD_A in which the hierarchical video encoding device 2 is mounted. As illustrated in (a) of FIG. 25, the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_A1. Thus, a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided. The hierarchical moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.

  The transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1 An image processing unit A7 that generates or processes an image may be further provided. FIG. 25A illustrates a configuration in which the transmission apparatus PROD_A includes all of these, but a part of the configuration may be omitted.

  The recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.

  FIG. 25B is a block diagram illustrating a configuration of the receiving device PROD_B in which the hierarchical video decoding device 1 is mounted. As illustrated in (b) of FIG. 25, the reception device PROD_B includes a reception unit PROD_B1 that receives a modulation signal, a demodulation unit PROD_B2 that obtains encoded data by demodulating the modulation signal received by the reception unit PROD_B1, and a demodulation A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2. The above-described hierarchical video decoding device 1 is used as the decoding unit PROD_B3.

  The receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. PROD_B6 may be further provided. FIG. 25B illustrates a configuration in which the reception apparatus PROD_B includes all of these, but a part of the configuration may be omitted.

  The recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

  Note that the transmission medium for transmitting the modulation signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

  For example, a terrestrial digital broadcast broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting. Further, a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

  Also, a server (workstation or the like) / client (television receiver, personal computer, smartphone, etc.) such as a VOD (Video On Demand) service or a video sharing service using the Internet transmits and receives a modulated signal by communication. This is an example of PROD_A / reception device PROD_B (usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. The smartphone also includes a multi-function mobile phone terminal.

  Note that the client of the video sharing service has a function of encoding a moving image captured by a camera and uploading it to the server in addition to a function of decoding the encoded data downloaded from the server and displaying it on the display. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.

  Next, the fact that the above-described hierarchical video encoding device 2 and hierarchical video decoding device 1 can be used for video recording and reproduction will be described with reference to FIG.

  FIG. 26A is a block diagram illustrating a configuration of a recording apparatus PROD_C in which the above-described hierarchical video encoding apparatus 2 is mounted. As shown in (a) of FIG. 26, the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M. A writing unit PROD_C2 for writing. The hierarchical moving image encoding device 2 described above is used as the encoding unit PROD_C1.

  The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of a type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) Or a drive device (not shown) built in the recording device PROD_C.

  The recording device PROD_C is a camera PROD_C3 that captures moving images as a supply source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and reception for receiving moving images. The unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided. In FIG. 26A, a configuration in which all of these are provided in the recording apparatus PROD_C is illustrated, but a part may be omitted.

  The receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.

  Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images). . In addition, a camcorder (in this case, the camera PROD_C3 is a main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images), a smartphone (in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images) is also an example of such a recording device PROD_C.

  FIG. 26B is a block diagram illustrating a configuration of a playback device PROD_D in which the above-described hierarchical video decoding device 1 is mounted. As shown in (b) of FIG. 26, the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written to the recording medium PROD_M and a coded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained. The hierarchical moving image decoding apparatus 1 described above is used as the decoding unit PROD_D2.

  Note that the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.

  In addition, the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. PROD_D5 may be further provided. FIG. 26B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but a part may be omitted.

    The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

  Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images). . In addition, a television receiver (in this case, the display PROD_D3 is a main supply destination of moving images), a digital signage (also referred to as an electronic signboard or an electronic bulletin board), and the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images. Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image) A smartphone (which is a main image supply destination), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination), and the like are also examples of such a playback device PROD_D.

(About hardware implementation and software implementation)
Finally, each block of the hierarchical video decoding device 1 and the hierarchical video encoding device 2 may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central It may be realized by software using a Processing Unit).

  In the latter case, each of the devices includes a CPU that executes instructions of a control program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, and A storage device (recording medium) such as a memory for storing various data is provided. An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program for each of the above devices, which is software that realizes the above-described functions, is recorded so as to be readable by a computer. This can also be achieved by supplying to each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU (Micro Processing Unit)).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, CD-ROM (Compact Disc Read-Only Memory) / MO (Magneto-Optical) / Disks including optical disks such as MD (Mini Disc) / DVD (Digital Versatile Disk) / CD-R (CD Recordable), cards such as IC cards (including memory cards) / optical cards, mask ROM / EPROM (Erasable Programmable Read-only Memory (EEPROM) (Electrically Erasable and Programmable Read-only Memory) / Semiconductor memories such as flash ROM, or logic circuits such as PLD (Programmable Logic Device) and FPGA (Field Programmable Gate Array) Etc. can be used.

  Further, each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, an intranet, an extranet, a LAN (Local Area Network), an ISDN (Integrated Services Digital Network), a VAN (Value-Added Network), a CATV (Community Antenna Television) communication network, a virtual private network (Virtual Private Network), A telephone line network, a mobile communication network, a satellite communication network, etc. can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, infra-red such as IrDA (Infrared Data Association) or remote control, such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc. , Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, terrestrial digital network, etc. Is possible. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope shown in the claims, and the present invention is also applied to an embodiment obtained by appropriately combining technical means disclosed in the embodiment. It is included in the technical scope of the invention.

  The present invention relates to a hierarchical video decoding device that decodes encoded data in which image data is hierarchically encoded, and a hierarchical video encoding device that generates encoded data in which image data is hierarchically encoded. It can be suitably applied to. Further, the present invention can be suitably applied to the data structure of hierarchically encoded data that is generated by a hierarchical video encoding device and referenced by the hierarchical video decoding device.

1. Hierarchical video decoding device (image decoding device)
11 NAL demultiplexing unit 12 Variable length decoding unit (flag decoding means)
13 Base decoding unit 14 Prediction parameter restoration unit 15 Texture restoration unit 146 Merge candidate derivation unit 148 PU partition type derivation unit 1461 Merge candidate derivation control unit (determination means)
1462 Merge candidate storage unit 1463 Slice type determination unit 1464 Individual merge candidate derivation unit 1464A Spatial merge candidate derivation unit (target layer candidate derivation unit, spatial motion information candidate derivation unit)
1464B Inter-layer merge candidate derivation unit (interlayer candidate derivation means)
1464C temporal merge candidate derivation unit (target layer candidate derivation means, temporal motion information candidate derivation means)
1464D Join merge candidate derivation unit (target layer candidate derivation means)
1464E Zero merge candidate derivation unit (target layer candidate derivation means)
1464F
1481 Reference CU setting section (reference coding unit setting means)
1482 PU partition estimation unit (partition pattern determination means)
1482A Lower right pixel position determination unit 1482B Overlapping region determination unit 2 Hierarchical video encoding device (image encoding device)
21 Prediction parameter determination unit 22 Prediction information generation unit 23 Base decoding unit 24 Texture information generation 25 Variable length encoding unit 26 NAL demultiplexing unit 226 Merge candidate deriving unit (interlayer candidate deriving unit)

Claims (14)

An image decoding device that decodes hierarchically encoded data in which image information relating to images of different quality for each layer is hierarchically encoded, and restores an image in a target layer to be decoded,
For the target coding unit that is the decoding target coding unit in the target layer, the reference layer that is the decoded layer is based on the coding unit that has an overlapping portion with the reference region that is the region corresponding to the target coding unit. A reference coding unit setting means for setting a reference coding unit to determine the degree of overlap with the reference region,
A division pattern determining unit that determines a division pattern of the target coding unit into a prediction unit that is a processing unit of a predicted image according to a degree of overlap between the reference region and the reference coding unit. An image decoding apparatus characterized by the above.   The image decoding apparatus according to claim 1, wherein the reference coding unit is a coding unit in the reference layer including a predetermined pixel in the reference region.   The image decoding apparatus according to claim 2, wherein the predetermined pixel is an upper left pixel of the reference area.   The image decoding apparatus according to claim 1, wherein the reference coding unit is a coding unit having a maximum area among coding units having an overlapping portion with the reference region. The degree of overlap between the reference region and the reference coding unit indicates a magnitude relationship regarding the width and height of the overlapping portion between the reference region and the reference coding unit.
5. The image decoding device according to claim 1, wherein the division pattern determination unit determines the division pattern according to the magnitude relationship. The degree of overlap between the reference region and the reference coding unit indicates an inclusive relationship between the reference region and the reference coding unit,
5. The image decoding device according to claim 1, wherein the division pattern determination unit determines the division pattern according to the inclusion relation. The degree of overlap between the reference region and the reference coding unit indicates the positional relationship of the lower right pixel of the reference coding unit with respect to the reference region,
The image decoding apparatus according to claim 3, wherein the division pattern determination unit determines the division pattern according to the positional relationship.   The image decoding apparatus according to claim 1, wherein the division pattern determination unit determines a symmetric partition as the division pattern.   The image decoding apparatus according to claim 1, wherein the division pattern determination unit restricts or prohibits division when the target coding unit is a predetermined size or less.   The division pattern determination means derives the division pattern in the target coding unit based on the division pattern of the reference coding unit when the reference region is included in the reference coding unit. The image decoding device according to any one of claims 5 to 7   When the ratio of the resolution of the target layer with respect to the resolution of the reference layer is smaller than a predetermined value, the division pattern determination unit is configured according to a magnitude relationship between the size of the reference coding unit and the size of the target coding unit. The image decoding apparatus according to claim 1, wherein the division pattern is determined. Furthermore, it comprises flag decoding means for decoding a reference layer estimation flag, which is a flag indicating that the estimation processing from the reference layer is performed, from the hierarchically encoded data,
When the reference layer estimation flag decoded in the target coding unit indicates that estimation processing from a reference layer is performed,
Setting of the reference coding unit by the reference coding unit setting means; and
The image decoding apparatus according to claim 1, wherein the division pattern determination unit determines the division pattern. A data structure of hierarchically encoded data decoded by the image decoding device according to claim 12,
The reference layer estimation flag is encoded after a skip flag that is a flag indicating whether or not the mode is a skip mode and before a prediction mode flag that is a flag indicating the type of the prediction mode,
When the reference layer estimation flag decoded in the target coding unit indicates that the estimation process from the reference layer is performed, the encoding of the prediction mode flag is omitted. data structure. An image encoding device that hierarchically encodes image information related to images of different quality for each layer and generates encoded data including information related to an image in a target layer to be encoded,
For a target coding unit that is a coding unit to be coded in the target layer, a coding unit that has an overlapping portion with a reference region that is a region corresponding to the target coding unit in a reference layer that is a coded layer Based on the reference coding unit setting means for setting a reference coding unit to determine the degree of overlap with the reference region,
A division pattern determining unit that determines a division pattern of the target coding unit into a prediction unit that is a processing unit of a predicted image according to a degree of overlap between the reference region and the reference coding unit. An image encoding device characterized by the above.

Images