JP2015015626A - Image decoder and image encoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce redundancy of codes, associated with inter-layer prediction, in decoding processing and encoding processing on a multi-layer image.SOLUTION: An encoder and a decoder use a reference picture set including texture dependence with respect to a dependence type between layers associated with inter-layer prediction and a specifying method for reference picture, and use an alternative reference picture only for a case of the motion dependence. Further, an extension parameter associated with inter-layer prediction is defined in an extension slice header different from a slice header, and when the extension slice header is omitted, the decoder is configured to set a predetermined default value for the extension parameter and to use the default value for decoding of the inter-layer prediction.

Description

  The present invention relates to an image decoding device and an image encoding device.

  One of the basic methods for encoding moving images is inter-frame prediction encoding in which prediction is performed between images (pictures) continuous in the time direction. Pictures that are temporally close to each other have a high correlation, and when the difference between the pictures is taken, the difference value is concentrated in the vicinity of zero. Therefore, the code amount can be significantly reduced by using the difference value.

  A method for encoding a moving image composed of a plurality of layers is generally referred to as scalable encoding or hierarchical encoding. The layer here means a set of moving image data composed of a plurality of pictures continuous in the time direction. In scalable coding, high coding efficiency is realized by performing prediction between layers in addition to the above-described interframe prediction. A reference layer that does not use inter-layer prediction is called a base layer, and other layers are called enhancement layers. Scalable encoding in the case where a layer is composed of viewpoint images is referred to as view scalable encoding. At this time, the base layer is also called a base view, and the enhancement layer is also called a non-base view (dependent view).

  In addition to view scalable coding, scalable coding includes spatial scalable coding (processing low-resolution pictures as a base layer and high-resolution pictures as an extension layer), SNR scalable coding (image quality as a base layer). Low picture, high quality picture as an enhancement layer). In scalable coding, in addition to using temporally preceding and succeeding pictures already encoded / decoded in each layer as reference pictures, pictures in layers other than the target layer can be used as reference layers. Temporal prediction within a layer is called motion prediction, and a vector indicating a shift between pictures is a motion vector. Prediction between layers is called inter-layer prediction, and a vector indicating a shift between pictures is called a displacement vector. In addition, when a layer is comprised from viewpoint images, such as view scalable encoding, a displacement vector is a vector showing the parallax between viewpoint images, and is also called a parallax vector.

  In the international standard of the moving picture coding system, examples of scalable coding include SVC (Scalable Video Coding) and MVC (Multi-view Video Coding). These are standards that support multiple layers by extending H.264 / AVC (Advanced Video Coding) for single layers. Thereafter, with regard to H.265 / HEVC (High Efficiency Video Coding; Non-Patent Document 1) standardized as a next-generation encoding method, a discussion on a scalable encoding method for a plurality of layers has started as an extended standard.

  In such an extension of the encoding method, importance is attached to backward compatibility such that the already standardized encoding method of the base portion is used as it is without being changed as much as possible. In scalable coding standards based on H.265 / HEVC, it is desirable that the coding data structure and low-level processing procedures defined in the single-layer standard be maintained as much as possible. . In the standard for a single layer, there is a method called temporal motion prediction that performs prediction between a picture different from a target picture (here, a picture that is temporally different). In scalable coding for multiple layers, in addition to this, a method called sample prediction that performs prediction between pictures in different layers can be used, and these can be adaptively switched or combined for use. Is aimed at improving the coding efficiency (Non-patent Document 2, Non-patent Document 3).

“Recommendation ITU-T H.265 High efficiency video coding”, TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU, 04/2013 “SHVC Working Draft 2”, JCTVC-M1008, JCT-VC, Incheon, KR, 18-26 Apr. 2013 “MV-HEVC Draft Text 4”, JCT3V-D1004, JCT-3V, Incheon, KR, 20-26 Apr. 2013

  In scalable coding for a plurality of layers, a code for transmitting a flag and parameters related to the use of predictive coding between layers is required. The flags and parameters related to the inter-layer prediction are defined so as to be included in a data structure called a slice segment header in Non-Patent Document 2 and Non-Patent Document 3. However, since the slice segment header is a data structure already defined in the standard for single layer, that is, H.265 / HEVC, redefinition to include codes related to multiple layers is This affects the designed / implemented decoding device and software, and it is necessary to change the existing slice header processing function, which increases the implementation cost.

  In addition, depending on the layer dependency type of VPS, whether the dependency relationship between layers is sample prediction dependency, motion prediction dependency, or both (sample prediction dependency and motion prediction dependency) is specified. . However, in Non-Patent Document 2 and Non-Patent Document 3, since it can be added to the reference picture list (RPS) used for reference picture selection regardless of the layer-dependent type, the reference picture added to the RPS is sample-predicted. There is a problem that it is not possible or not guaranteed.

  Further, as a method of designating a reference picture colPic for temporal motion prediction, a first method of designating a picture in a reference picture list (RPL) generated from a reference picture list (RPS) and a reference picture index refIdx, and an alternative reference As a picture, there are two methods of specifying by a layer ID. However, in Non-Patent Document 2 and Non-Patent Document 3, regardless of whether the layer dependence type is motion prediction dependence or both (sample prediction dependence and motion prediction dependence), either the first method or the second method is used. In the case of temporal motion prediction, there is a problem that transmission of a code specifying the reference picture colPic becomes redundant.

  The present invention has been made in view of the above points, and in scalable coding for a plurality of layers, there is a problem in implementation cost when an encoding method for a single layer is extended, and in inter-layer prediction. Provided are an image decoding device, an image decoding method, an image decoding program, an image encoding device, an image encoding method, and an image encoding program that eliminate code redundancy related to designation of a reference image.

In order to solve the above-described problem, an image decoding apparatus according to the present invention includes:
(1) An image decoding device that decodes hierarchically encoded data,
Dependency type decoding means comprising means for decoding a layer dependency type and determining whether a layer corresponding to the layer dependency type is a layer that exhibits sample dependency;
A reference picture set decoding unit for extracting a sample dependent layer from the dependency layer indicating the sample dependence and deriving a reference picture set;
An inter-prediction image generation unit that generates a prediction image using a reference picture specified by a reference picture list derived using the inter-layer reference picture set is provided.

(2) In the above image decoding apparatus,
The dependence type decoding means decodes the layer dependence type from the encoded data of the parameter set, derives SamplePredEnableFlag, which is a flag indicating whether the dependence layer is a sample dependence layer, according to the layer dependence type,
The sample dependent layer decoding means decodes the inter-layer dependency identifier (inter_layer_pred_layer_idc) from the encoded data of the slice segment layer, and derives the sample dependent layer ActiveSamplePredRefLayerId from the dependent layer whose SamplePredEnableFlag is 1.

(3) The image decoding apparatus further includes an inter-layer prediction only flag decoding means for decoding an inter-layer prediction only flag inter_layer_sample_only flag,
The alternative reference picture decoding unit derives the number NumActiveSamplePredRefLayers of the sample dependent layers according to the layer dependent type,
When the inter-layer prediction only flag inter_layer_sample_only flag indicates only inter-layer prediction, the reference picture set decoding unit does not include pictures in the same layer in the reference picture list,
The inter-layer prediction only flag decoding means decodes the inter-layer prediction only flag inter_layer_sample_only flag when the number of motion-only dependent layers NumActiveSamplePredRefLayers is 1 or more.
(4) Further, an image decoding device having another configuration according to the present invention includes:
An image decoding device that decodes hierarchically encoded data,
Dependency type decoding means comprising means for decoding a layer dependent type and determining whether the layer corresponding to the layer dependent type is a motion dependent only layer indicating that it is motion dependent and not sample dependent;
An alternative reference picture decoding unit for decoding an alternative reference picture from the motion-dependent only layer;
An inter prediction parameter deriving unit for deriving a temporal motion vector using the alternative reference picture is provided.

(5) In the above image decoding apparatus,
The alternative reference picture decoding unit derives the number of motion dependent only layers NumActiveMotionPredRefLayers, which is a dependent layer indicating that it is motion dependent and not sample dependent, according to the layer dependent type, and the number of motion only dependent layers NumActiveMotionPredRefLayers is 1. In the case described above, the flag alt_collocated_indication_flag indicating whether or not to use the alternative reference picture is decoded from the encoded data of the slice segment layer.

(6) In the above image decoding apparatus,
The alternative reference picture decoding unit derives a motion-dependent only layer ActivnMotionPredRefLayerId, which is a dependent layer indicating that it is motion-dependent and not sample-dependent, according to the layer-dependent type, and a MotionPredEnableFlag that is a flag indicating only the motion-dependent layer The identifier collocated_ref_layer_idx that identifies the alternative reference picture is decoded from the motion-dependent only layer.

(7) An image encoding device according to the present invention includes:
An image encoding apparatus that hierarchically encodes multiple layers of image data,
A header information encoding means for encoding at least actual dependent layer information related to sample prediction between layers and actual dependent layer information related to motion prediction between layers as a parameter related to a coding method of a non-base layer picture;
The header information encoding means, based on the number of sample prediction reference layers (NumSamplePredRefLayers) for the target layer, the presence / absence of codes of the inter-layer prediction reference pictures (num_inter_layer_ref_pics_minus1) and the inter-layer dependency identifier (inter_layer_pred_layer_idc), the code length, and the value Encode the range and use the encoded inter-layer prediction reference picture number and the inter-layer dependency identifier for encoding non-base layer pictures as actual dependent layer information related to inter-layer sample prediction. It is characterized by.

(8) An image decoding apparatus having a further configuration according to the present invention is as follows:
An image decoding device that decodes hierarchically encoded data,
Parameter set decoding means for decoding at least the number of dependent layers (NumDirectRefLayers [nuh_layer_id]) related to the layer nuh_layer_id as a parameter related to the encoding method of the non-base layer picture;
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extended decoding means decodes each parameter indicating presence / absence of inter-layer prediction of a non-base layer picture (inter_layer_pred_enable_flag), number of inter-layer prediction reference pictures (num_inter_layer_ref_pics_minus1), and inter-layer dependency identifier (inter_layer_pred_layer_idc [i]) If each syntax is not included in the encoded data, inter_layer_pred_enable_flag is set to 1 for inter-layer prediction, num_inter_layer_ref_pics_minus1 is decoded to NumDirectRefLayers [nuh_layer_id]-1, and inter_layer_pred_layer_idc [i] is decoded using a parameter set The dependency layer is set to i indicating that the dependency layer is used as it is.

(9) An image decoding device having a further configuration according to the present invention is as follows:
An image decoding device that decodes hierarchically encoded data,
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means includes:
Motion-dependent layer replacement information decoding means for decoding motion-dependent layer replacement information (alt_collocated_indication_flag, collocated_ref_layer_idx) indicating a reference picture at the time of temporal motion vector prediction in a non-base layer picture;
A motion vector deriving unit for deriving a temporal motion vector using a picture specified by the decoded motion dependent layer alternative information;
The motion dependent layer alternative information decoding means decodes the motion dependent layer alternative information from the encoded data when the encoded data includes extended header information, and the encoded data does not include the extended header information In this case, alt_collocated_indication_flag = 0 indicating that an alternative reference layer is not used is decoded as motion-dependent layer alternative information.

(10) An image decoding device having still another configuration according to the present invention includes:
An image decoding device that decodes hierarchically encoded data,
Parameter set decoding means for decoding at least the number of motion-dependent layers (NumMotionPredRefLayers [i]) as a parameter related to the non-base layer picture encoding method;
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means includes:
Motion-dependent layer replacement information decoding means for decoding motion-dependent layer replacement information (alt_collocated_indication_flag, collocated_ref_layer_idx) indicating a reference picture at the time of temporal motion vector prediction in a non-base layer picture;
A motion vector deriving unit for deriving a temporal motion vector using a picture specified by the decoded motion dependent layer alternative information;
The motion dependent layer alternative information decoding means decodes the motion dependent layer alternative information from the encoded data when the encoded data includes extended header information, and the encoded data does not include the extended header information In this case, when the number of motion-dependent layers (NumMotionPredRefLayers [i]) derived by the parameter set decoding unit is 0, alt_collocated_indication_flag = 0 indicating that an alternative reference layer is not used is decoded as motion-dependent layer alternative information If the number of motion-dependent layers (NumMotionPredRefLayers [i]) is other than 0, alt_collocated_indication_flag = 1 indicating that an alternative reference layer is used is decoded.

(11) An image decoding device having a further configuration according to the present invention is as follows:
An image decoding device that decodes hierarchically encoded data,
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means includes:
Motion-dependent layer replacement information decoding means for decoding motion-dependent layer replacement information (alt_collocated_indication_flag, collocated_ref_layer_idx) indicating a reference picture at the time of temporal motion vector prediction in a non-base layer picture;
Reference picture information decoding means for decoding reference picture information (RPS, RPL);
A motion vector deriving unit for deriving a temporal motion vector using a picture specified by the decoded motion dependent layer alternative information;
The motion dependent layer alternative information decoding means decodes the motion dependent layer alternative information from the encoded data when the encoded data includes extended header information, and the encoded data does not include the extended header information In this case, when the reference picture information derived by the reference picture information decoding means includes a picture of the same target layer, alt_collocated_indication_flag = 0 indicating that the alternative reference layer is not used is decoded as motion dependent layer alternative information. When the reference picture information derived by the reference picture information decoding means does not include a picture of the same target layer, alt_collocated_indication_flag = 1 indicating that an alternative reference layer is used is decoded.

(12) An image decoding device having still another configuration according to the present invention includes:
An image decoding device that decodes hierarchically encoded data,
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
When the extension header information is included in the encoded data, the slice segment header extension decoding means includes at least parameters indicating a reference picture at the time of temporal motion vector prediction (alt_collocated_indication_flag, collocated_ref_layer_idx )
When extended header information is not included in the encoded data, it is an encoding parameter included in the basic header information, and is based on parameters (collocated_from_0_flag, collocated_ref_idx) indicating a reference picture in temporal motion vector prediction of the base layer, Each of the parameters in the non-base layer picture is determined.

(13) Another configuration of the image coding apparatus according to the present invention is as follows:
An image encoding apparatus that hierarchically encodes multiple layers of image data,
Basic header information encoding means for encoding encoding parameters common to the base layer and the non-base layer;
An extension header information encoding means for encoding encoding parameters for the non-base layer,
The extension header information encoding means omits encoding of the extension header information when the values of a plurality of parameter groups related to inter-layer prediction among the encoding parameters related to the non-base layer are a predetermined combination. To do.

  According to the present invention, in scalable encoding / decoding for a plurality of layers, backward compatibility with an encoding / decoding scheme for a single layer is ensured, particularly in transmission of flags and parameters related to inter-layer prediction. It is possible to reduce redundancy and reduce the amount of transmission code.

It is the schematic which shows the structure of the image transmission system which concerns on this invention. It is a figure which shows the hierarchical structure of the data of the encoding stream which concerns on this invention. It is a conceptual diagram which shows an example of a reference picture list. It is a conceptual diagram which shows the example of a reference picture. It is the schematic which shows the structure of the image decoding apparatus which concerns on one Embodiment of this invention. It is the schematic which shows the structure of the image decoding apparatus which concerns on another embodiment of this invention. It is the schematic which shows the structure of the image decoding apparatus which concerns on another embodiment of this invention. It is the schematic which shows the structure of the image data decoding part which concerns on this embodiment. It is the schematic which shows the structure of the image coding apparatus which concerns on embodiment of this invention. It is the schematic which shows the structure of the image data encoding part which concerns on this embodiment. It is an example of the syntax table of the extended video parameter set which concerns on this invention. It is an example of the syntax table of the slice segment header which concerns on this invention. It is an example of the derivation | leading-out procedure of the real dependence layer information by the real dependence layer decoding part which concerns on embodiment of this invention. It is an example of the syntax table of the slice segment header extension according to the present invention. It is a figure explaining the relationship between the layer dependence type in a comparative example, a sample dependence flag, and a motion dependence flag. It is a figure explaining the relationship between the layer dependence type which concerns on embodiment of this invention, a sample dependence flag, and a motion dependence flag. It is a figure explaining the relationship between the layer dependence type in a comparative example, a sample dependence flag, a motion dependence flag, and a reference picture set. It is an example of a procedure for deriving the actual reference layer picture number, the reference layer identifier, the inter-layer reference picture set, and the total reference picture number in the comparative example. It is a conceptual diagram explaining the relationship of dependency between layers in a comparative example. It is a figure explaining the relationship between the layer dependence type which concerns on the 1st Embodiment of this invention, a sample dependence flag, a motion dependence flag, and a reference picture set. It is an example of a procedure for deriving the actual reference layer picture number, the reference layer identifier, the inter-layer reference picture set, and the total reference picture number according to the first embodiment of the present invention. It is a conceptual diagram explaining the relationship of dependency between layers which concerns on the 1st Embodiment of this invention. It is an example of the derivation | leading-out procedure of the reference picture list which concerns on the 1st Embodiment of this invention. It is a figure explaining the relationship between the layer dependence type in a comparative example, and an alternative collocated picture. It is a conceptual diagram explaining the relationship of dependency between layers in a comparative example. It is a figure explaining the relationship between the layer dependence type which concerns on the 1st Embodiment of this invention, and an alternative collocated picture. It is a conceptual diagram explaining the relationship of dependency between layers which concerns on the 1st Embodiment of this invention.

(First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system 1 according to the present embodiment.

  The image transmission system 1 is a system that transmits a code obtained by encoding a plurality of layer images and displays an image obtained by decoding the transmitted code. The image transmission system 1 includes an image encoding device 11, a network 21, an image decoding device 31, and an image display device 41.

  A signal T indicating a plurality of layer images (also referred to as texture images) is input to the image encoding device 11. A layer image is an image that is viewed or photographed at a certain resolution and a certain viewpoint. When performing view scalable coding in which a three-dimensional image is coded using a plurality of layer images, each of the plurality of layer images is referred to as a viewpoint image. Here, the viewpoint corresponds to the position or observation point of the photographing apparatus. For example, the plurality of viewpoint images are images taken by the left and right photographing devices toward the subject. The image encoding device 11 encodes each of these signals to generate an encoded stream Te (encoded data). Details of the encoded stream Te will be described later. A viewpoint image is a two-dimensional image (planar image) observed at a certain viewpoint. The viewpoint image is indicated by, for example, a luminance value or a color signal value for each pixel arranged in a two-dimensional plane. Hereinafter, one viewpoint image or a signal indicating the viewpoint image is referred to as a picture. In addition, when performing spatial scalable coding using a plurality of layer images, the plurality of layer images include a base layer image having a low resolution and an enhancement layer image having a high resolution. When SNR scalable encoding is performed using a plurality of layer images, the plurality of layer images are composed of a base layer image with low image quality and an extended layer image with high image quality. Note that view scalable coding, spatial scalable coding, and SNR scalable coding may be arbitrarily combined. In the present embodiment, encoding and decoding of an image including at least a base layer image and an image other than the base layer image (enhancement layer image) is handled as the plurality of layer images. Of the multiple layers, for two layers that have a reference relationship (dependency relationship) in the image or encoding parameter, the image on the reference side is referred to as a first layer image, and the image on the reference side is referred to as a second layer image. . For example, when there is an extended image (other than the base layer) encoded with reference to the base layer, the base layer image is handled as a first layer image and the extended image is handled as a second layer image. Examples of enhancement layer images include viewpoint images other than the base view, depth images, and high resolution images.

  The network 21 transmits the encoded stream Te generated by the image encoding device 11 to the image decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional or bidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may be replaced with a storage medium that records an encoded stream Te such as a DVD (Digital Versatile Disc) or a BD (Blue-ray Disc).

  The image decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates a plurality of decoded layer images Td (decoded viewpoint images Td) respectively decoded.

  The image display device 41 displays all or part of the plurality of decoded layer images Td generated by the image decoding device 31. For example, in view scalable coding, a 3D image (stereoscopic image) and a free viewpoint image are displayed in all cases, and a 2D image is displayed in some cases. The image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. In addition, in the spatial scalable coding and SNR scalable coding, when the image decoding device 31 and the image display device 41 have a high processing capability, a high-quality enhancement layer image is displayed and only a lower processing capability is provided. Displays a base layer image that does not require higher processing capability and display capability as an extension layer.

<Structure of encoded stream Te>
Prior to detailed description of the image encoding device 11 and the image decoding device 31 according to the present embodiment, a data structure of an encoded stream Te generated by the image encoding device 11 and decoded by the image decoding device 31 will be described. .

  FIG. 2 is a diagram illustrating a hierarchical structure of data in the encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. (A) to (f) of FIG. 2 respectively show a sequence layer that defines a sequence SEQ, a picture layer that defines a picture PICT, a slice layer that defines a slice S, a slice data layer that defines slice data, and slice data. It is a figure which shows the encoding tree layer which prescribes | regulates the encoding tree layer and coding unit (Coding Unit; CU) contained in a coding tree which prescribe | regulate the coding tree unit contained. The encoded data of each layer constitutes a data structure called a NAL (Network Abstraction Layer) unit for each data type. The NAL unit type (nal_unit_type) included in the header part of the NAL unit indicates the type of data constituting the target NAL unit. The NAL unit header includes a layer identifier (nuh_layer_id). The layer identifier is information for distinguishing a plurality of layer images (multi-viewpoint images and the like) that are targets of scalable encoding.

(Sequence layer)
In the sequence layer, a set of data referred to by the image decoding device 31 in order to decode a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined. As shown in FIG. 2A, the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, It includes supplemental enhancement information (SEI). Here, the value indicated after # indicates the layer ID. FIG. 2 shows an example in which encoded data of # 0 and # 1, that is, layer 0 and layer 1, exists, but the type of layer and the number of layers are not limited to this.

  In the video parameter set VPS, in a moving image composed of a plurality of layers, a set of encoding parameters common to the plurality of moving images, a plurality of layers included in the moving image, and encoding parameters related to the individual layers. A set is defined.

  In the sequence parameter set SPS, a set of encoding parameters referred to by the image decoding device 31 in order to decode the target sequence is defined. For example, the width and height of the picture in the target sequence are defined.

  In the picture parameter set PPS, a set of encoding parameters referred to by the image decoding device 31 in order to decode each picture in the target sequence is defined. For example, a reference value of a quantization width used for picture decoding and a flag indicating application of weighted prediction are included. A plurality of PPSs may exist in the same layer. In this case, it is selected which of a plurality of PPSs is applied from each picture in the target sequence.

(Picture layer)
In the picture layer, a set of data referred to by the image decoding device 31 for decoding a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 2B, the picture PICT includes slices S0 to SNS-1 (NS is the total number of slices included in the picture PICT).

  Hereinafter, when it is not necessary to distinguish each of the slices S0 to SNS-1, the subscripts may be omitted. The same applies to data included in an encoded stream Te described below and to which other subscripts are attached.

(Slice layer)
In the slice layer, a set of data referred to by the image decoding device 31 for decoding the slice S to be processed (also referred to as a target slice) is defined. As shown in FIG. 2C, the slice S includes a slice header SH and slice data SDATA.

  The slice header SH includes an encoding parameter group that is referred to by the image decoding device 31 in order to determine a decoding method of the target slice. Slice type designation information (slice_type) for designating a slice type is an example of an encoding parameter included in the slice header SH.

  As slice types that can be specified by the slice type specification information, (1) I slice using only intra prediction at the time of encoding, (2) P slice using unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

  The slice header SH includes collocated information collocated_from_l0_flag and collocated_ref_idx for designating temporal motion information described later when the slice type is P slice or B slice.

  Note that the slice header SH may include a PPS identifier (pic_parameter_set_id) indicating a reference to the picture parameter set PPS included in the sequence layer.

(Slice data layer)
In the slice data layer, a set of data referred to by the image decoding device 31 for decoding the slice data SDATA to be processed is defined. The slice data SDATA includes a coding tree unit (CTU) as shown in (d) of FIG. The coding tree unit CTU is an image area having a fixed size (for example, 64 × 64 pixels) that constitutes a slice. An image block corresponding to the coding tree unit CTU is referred to as a coding tree block (CTB).

  The slice layer may be handled in units of slice segment layers obtained by further dividing the slice. In this case, the aforementioned slice header SH is called a slice segment header, and similarly, the slice data SDATA is called slice segment data. In the following description, for the sake of simplicity, it will be described as a slice header and slice data, but can be replaced with a slice segment header and slice segment data, respectively. Also, the slice segment header and slice segment data can be replaced with the slice header and slice data, respectively.

(Encoding tree layer)
As shown in FIG. 2E, the coding tree layer defines a set of data that the image decoding device 31 refers to in order to decode the coding tree unit to be processed. The coding tree unit is divided by recursive quadtree division. A tree structure obtained by recursive quadtree partitioning is called a coding tree. The coding tree unit CTU includes a split flag (split_flag). When the split_flag is 1, the coding tree unit CTU is further divided into four CTUs. When split_flag is 0, the coding tree unit CTU is divided into four coding units (CU: Coding Unit). The coding unit CU is a terminal node of the coding tree layer and is not further divided in this layer. The encoding unit CU is a basic unit of encoding / decoding processing.

  When the size of the encoding tree unit CTU is 64 × 64 pixels, the size of the encoding unit CU is 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, or 8 × 8 pixels. It can take. Note that an image block corresponding to the coding unit CU is referred to as a coding block (CB).

(Encoding unit layer)
As shown in (f) of FIG. 2, the encoding unit layer defines a set of data referred to by the image decoding device 31 in order to decode the processing target encoding unit. Specifically, the encoding unit is a prediction tree composed of a CU header CUH, one or more prediction units (PU: Prediction Unit), and a transform composed of one or more transform units (TU: Transform Unit). Consists of a tree.

  The CU header CUH defines whether the encoding unit is a unit that uses intra prediction or a unit that uses inter prediction.

  In the prediction tree, the encoding unit CU is divided into one or a plurality of prediction units PU, and the position and size of each prediction unit are defined. Broadly speaking, there are two types of division in the prediction tree: intra prediction and inter prediction. Intra prediction is prediction within the same picture, and inter prediction refers to prediction processing performed between different pictures (for example, between display times and between layer images).

  In the case of intra prediction, there are 2N × 2N (the same size as the encoding unit) and N × N division methods.

  In the case of inter prediction, the division method is defined by the division mode (part_mode) included in the CU header CUH, 2N × 2N (the same size as the encoding unit), 2N × N, 2N × nU, 2N × nD, N × 2N, nL × 2N, nR × 2N, and N × N. Note that 2N × nU indicates that a 2N × 2N encoding unit is divided into two regions of 2N × 0.5N and 2N × 1.5N in order from the top. 2N × nD indicates that a 2N × 2N encoding unit is divided into two regions of 2N × 1.5N and 2N × 0.5N in order from the top. nL × 2N indicates that a 2N × 2N encoding unit is divided into two regions of 0.5N × 2N and 1.5N × 2N in order from the left. nR × 2N indicates that a 2N × 2N coding unit is divided into two regions of 1.5N × 2N and 0.5N × 2N in order from the left. Since the number of divisions is one of 1, 2, and 4, PUs included in the CU are 1 to 4. These PUs are expressed as PU0, PU1, PU2, and PU3 in order.

  In the conversion tree, the encoding unit CU is divided into one or a plurality of conversion units TU, and the position and size of each conversion unit are defined. There are two types of division in the conversion tree: one in which an area having the same size as the encoding unit is allocated as a conversion unit, and the other in division by recursive quadtree division, similar to the above-described division in the encoding tree block.

  Note that the image blocks corresponding to the prediction unit PU and the transform unit TU are referred to as a prediction block (PB: Prediction Block) and a transform block (TB: Transform Block), respectively.

(Prediction parameter)
The prediction image of the prediction unit is derived by a prediction parameter associated with the prediction unit. The prediction parameters include a prediction parameter for intra prediction or a prediction parameter for inter prediction. Hereinafter, prediction parameters for inter prediction (inter prediction parameters) will be described. The inter prediction parameter includes prediction list use flags predFlagL0 and predFlagL1, reference picture indexes refIdxL0 and refIdxL1, and vectors mvL0 and mvL1. Prediction list use flags predFlagL0 and predFlagL1 are flags indicating whether or not reference picture lists called L0 list RefPicList0 and L1 list RefPicList1 are used. When the value is 1, the reference picture list corresponding to each flag is used. Used. In this specification, when “flag indicating whether or not XX” is described, it is assumed that the value is XX when the value is 1, and that it is not XX when the value is 0. In products and the like, 1 is treated as true and 0 is treated as false (the same applies hereinafter). However, other values may be used as a true value and a false value in an actual apparatus or method. When two reference picture lists are used, that is, when predFlagL0 = 1 and predFlagL1 = 1, the prediction method is called bi-prediction, and when one reference picture list is used, that is, (predFlagL0, predFlagL1) = (1, 0) or (predFlagL0, predFlagL1) = (0,1), the prediction method is called single prediction. Note that the prediction list use flag information can also be expressed by an inter prediction flag inter_pred_idc described later.

  Syntax elements for deriving inter prediction parameters included in the encoded data include, for example, a partition mode part_mode, a merge flag merge_flag, a merge index merge_idx, an inter prediction flag inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, and a difference There is a vector mvdLX.

(Reference picture list)
Next, an example of the reference picture list Ref will be described. The reference picture list is an information set including reference pictures stored in a reference picture memory 306 (FIG. 8) described later. FIG. 3 is a conceptual diagram illustrating an example of a reference picture list. In the reference picture list 601, five rectangles arranged in a line on the left and right indicate reference pictures, respectively. Reference symbols P0, P1, Q2, P3, and P4 shown in order from the left end to the right are symbols indicating respective reference pictures. P such as P1 indicates a certain layer P, and Q of Q2 indicates a layer Q different from the layer P. The subscripts P and Q indicate a picture order number POC (Picture Order Count). The reference picture list is an information set that can be updated in units of slice layers. Which reference picture in the reference picture is referred to is separately indicated by a reference picture index refIdxLX in the coding unit layer.

(Reference picture example)
Next, an example of a reference picture used for deriving a vector will be described. FIG. 4 is a conceptual diagram illustrating an example of a reference picture. In FIG. 4, the horizontal axis indicates display time, and the vertical axis indicates layers (for example, moving images with different viewpoints). Each of the illustrated rectangles represents a picture. Of the illustrated pictures, the picture P2 in the lower row is a decoding target picture (target picture). A reference picture Q2 indicated by an upward arrow from the target picture is a picture that has the same display time as the target picture and a different viewpoint. In the displacement prediction based on the target picture, the reference picture Q0 is used. Reference pictures P0 and P1 indicated by left-pointing arrows from the target picture are past pictures at the same viewpoint as the target picture. Reference pictures P3 and P4 indicated by right-pointing arrows from the target picture are the same viewpoint as the target picture and are future pictures. In motion prediction based on the target picture, reference pictures P0, P1, P3, and P4 are used.

(Inter prediction flag and prediction list usage flag)
The relationship between the inter prediction flag and the prediction list use flags predFlagL0 and predFlagL1 can be mutually converted as follows. Therefore, as an inter prediction parameter, a prediction list use flag may be used, or an inter prediction flag may be used. In addition, hereinafter, the determination using the prediction list use flag may be replaced with the inter prediction flag. Conversely, the determination using the inter prediction flag can be performed by replacing the prediction list use flag.

Inter prediction flag = (predFlagL1 << 1) + predFlagL0
predFlagL0 = Inter prediction flag & 1
predFlagL1 = Inter prediction flag >> 1
Here, >> is a right shift, and << is a left shift.

  The inter prediction flag inter_pred_idc is data indicating the type and number of reference pictures, and takes any value of Pred_L0, Pred_L1, and Pred_Bi. Pred_L0 and Pred_L1 indicate that reference pictures stored in the reference picture list RefPicList0 and the reference picture list RefPicList1, respectively, are used, and both indicate that one reference picture is used (single prediction). Prediction using the reference picture list RefPicList0 and the reference picture list RefPicList1 is referred to as L0 prediction and L1 prediction, respectively. Pred_Bi indicates that two reference pictures are used (bi-prediction), and indicates that two reference pictures specified by the reference picture list RefPicList0 and the reference picture list RefPicList1 are used. The prediction vector index mvp_LX_idx is an index indicating a prediction vector, and the reference picture index refIdxLX is an index indicating a reference picture stored in the reference picture list. Note that LX is a description method used when L0 prediction and L1 prediction are not distinguished. By replacing LX with L0 and L1, parameters for the reference picture specified in the reference picture list RefPicList0 and the reference picture list RefPicList1 are used. Differentiate the parameters for the specified reference picture. For example, refIdxL0 is a reference picture index used for L0 prediction, refIdxL1 is a reference picture index used for L1 prediction, and refIdx (or refIdxLX) is a notation used when refIdxL0 and refIdxL1 are not distinguished.

(Motion vector and displacement vector)
The vector mvLX includes a motion vector and a displacement vector (disparity vector). A motion vector is a position between a block position in a picture at a certain display time of a layer and a corresponding block position in a picture at a different display time (for example, adjacent discrete time) in the same layer. It is a vector indicating a deviation. The displacement vector is a vector indicating a positional shift between the position of a block in a picture at a certain display time of a certain layer and the position of a corresponding block in a picture of a different layer at the same display time. The pictures in different layers may be pictures from different viewpoints or pictures with different resolutions. In particular, a displacement vector corresponding to pictures of different viewpoints is called a disparity vector. In the following description, when a motion vector and a displacement vector are not distinguished, they are simply referred to as a vector mvLX. A prediction vector and a difference vector related to the vector mvLX are referred to as a prediction vector mvpLX and a difference vector mvdLX, respectively. Whether the vector mvLX and the difference vector mvdLX are motion vectors or displacement vectors is distinguished using a reference picture index refIdxLX attached to the vector.

(VPS expansion)
Next, parameters indicating a reference relationship between layers in inter-layer prediction will be described. The reference relationship between layers in inter-layer prediction is specified by a VPS extension (vps_extension) that is a data set obtained by extending a VPS. FIG. 11 shows an example of the data structure of VPS extension. Among these, the layer dependence flag direct_dependency_flag [i] [j] and the layer dependence type direct_dependency_type [i] [j] indicate the reference relationship between layers. Specifically, when the layer dependence flag direct_dependency_flag [i] [j] is 1, the target layer i refers to another layer j, and when the layer dependence flag is 0, the target layer i does not refer to the layer j. . When the layer dependency flag is 1, the layer dependency type direct_dependency_type [i] [j] indicates the layer dependency type of the reference layer j with respect to the target layer i. The layer dependent type can specify sample prediction only, motion prediction only, or both. The relationship between the value of direct_dependency_type [i] [j] and the value of the layer dependency type is shown below.

direct_dependency_type [i] [j] = 0… sample prediction and motion prediction
direct_dependency_type [i] [j] = 1… motion prediction only
direct_dependency_type [i] [j] = 2 ... Sample prediction only From the above codes, parameters related to inter-layer prediction are derived in a parameter set decoding unit (FIGS. 5 to 7) described later.

(RPS information)
Syntax values related to RPS (hereinafter referred to as RPS information) include the following.
1. 1. SPS short-term RPS information: short-term reference picture set information included in the SPS 2. SPS long-term RP information: long-term reference picture information included in SPS SH short-term RPS information: short-term reference picture set information included in the slice header SH long-term RP information: long-term reference picture information included in the slice header ILRP information: Inter-layer reference picture information (1. SPS short-term RPS information)
The SPS short-term RPS information includes information on a plurality of short-term RPSs that can be used from each picture referring to the SPS. Note that the short-term RPS is a set of pictures that can be a reference picture (short-term reference picture) specified by a relative position with respect to the target picture (for example, a POC difference from the target picture).

  The short-term RPS information includes the number of short-term reference pictures (num_negative_pics) whose display order is earlier than that of the target picture and the number of short-term reference pictures (num_positive_pics) whose display order is later than that of the target picture. In the following, a short-term reference picture whose display order is earlier than the target picture is referred to as a front short-term reference picture, and a short-term reference picture whose display order is later than the target picture is referred to as a rear short-term reference picture.

  The short-term RPS information includes, for each forward short-term reference picture, the absolute value of the POC difference with respect to the target picture (delta_poc_s0_minus1 [i]), and the presence / absence of use as a reference picture of the target picture (used_by_curr_pic_s0_flag [ i]). In addition, for each backward short-term reference picture, the absolute value of the POC difference with respect to the target picture (delta_poc_s1_minus1 [i]) and the possibility of being used as the reference picture of the target picture (used_by_curr_pic_s1_flag [i]) are included It is.

(2. SPS long-term RP information)
The SPS long-term RP information includes information on a plurality of long-term reference pictures that can be used from each picture that references the SPS. A long-term reference picture is a picture specified by an absolute position (for example, POC) in a sequence.

  The SPS long-term RP information includes information (long_term_ref_pics_present_flag) indicating the presence or absence of a long-term reference picture transmitted in the SPS, the number of long-term reference pictures included in the SPS (num_long_term_ref_pics_sps), and information on each long-term reference picture. The long-term reference picture information includes the POC (lt_ref_pic_poc_lsb_sps [i]) of the reference picture and the possibility of being used as the reference picture of the target picture (used_by_curr_pic_lt_sps_flag [i]).

  The POC of the reference picture may be the POC value itself associated with the reference picture, or the LSB (Least Significant Bit) of the POC, that is, the remainder obtained by dividing the POC by a predetermined power of 2 The value of may be used.

(3. SH short-term RPS information)
The SH short-term RPS information includes information of a single short-term RPS that can be used from a picture that references a slice header.

(4. SH long-term RP information)
The SH long-term RP information includes information on a long-term reference picture that can be used from a picture that references a slice header.

(5. ILRP information)
ILRP information is information on an inter-layer reference picture that can be referred to in inter-layer prediction. The ILRP information includes an inter-layer prediction valid flag (inter_layer_pred_enabled_flag), the number of inter-layer reference pictures (NumActiveRefLayerPics), and a layer identifier (inter_layer_pred_layer_idc [i]) indicating a layer to which each inter-layer reference picture belongs.

(Configuration of comparative example)
Before describing the configuration of the image decoding device 31 according to an embodiment of the present invention, a comparative example will be described. FIG. 15 is a diagram for explaining the relationship between the layer dependence type, the sample dependence flag SamplePredEnableFlag, and the motion dependence flag SamplePredEnableFlag according to the comparative example. FIG. 15A shows that, in the comparative example, the layer dependence type is other than 0 (here, 2), and both the sample dependence flag SamplePredEnableFlag and the motion dependence flag SamplePredEnableFlag are 1. FIG. These show the derivation | leading-out processing of the sample dependence flag SamplePredEnableFlag and the motion dependence flag SamplePredEnableFlag in a comparative example.

  FIG. 17 shows the state of the layer dependent type of pictures that can be included in the reference picture set RPS in the comparative example. As shown in FIG. 17, in the comparative example, the reference picture set RPS can be included regardless of the layer-dependent type state (sample-dependent flag SamplePredEnableFlag and motion-dependent flag MotionPredEnableFlag).

  FIG. 18 is a diagram illustrating a derivation process of the reference picture set RPS in the comparative example. FIG. 18A shows processing for deriving the actual dependent layer number NumActiveRefLayerPics in the comparative example. FIG. 18B shows processing for deriving the reference picture layer RefPicLayerId in the comparative example. FIG. 18C shows RPS derivation processing in the comparative example. As shown in FIG. 18 (c), RefPicSetInterLayer [i] is marked with a long-term prediction reference “used for long-term reference”, whereby a reference picture is put into the RPS. In the process of FIG. 18C, the layer of nuh_layer_id included in the reference picture layer RefPicLayerId [i] derived by the process of FIG. 18B is put in the reference picture, but the reference picture layer RefPicLayerId [i] Does not depend on the layer dependency type. Therefore, in the comparative example, as shown in FIG. 17, the reference picture set RPS can be included regardless of the layer-dependent type.

  FIG. 19 is a diagram illustrating RPS derivation processing in the comparative example. It can be seen that the actual dependent layer number NumActiveRefLayerPics is set to 3, and the reference picture layer RefPicLayerId [] [] is set to nuh_layer_id = 0,2,3.

  FIG. 24 shows a state of a layer dependent type of a picture that can be used for the alternative collocated picture colPic in the comparative example. As shown in FIG. 24A, in the comparative example, even when the sample dependence flag SamplePredEnableFlag is 1 as the layer dependence type state, if the motion dependence flag MotionPredEnableFlag is 1, it is used for the alternative collocated picture colPic. Can do. FIG. 24B shows a derivation process of a motion-dependent layer (actual motion-dependent layer identifier ActiveMotionPredRefLayerId []) and the number of motion-dependent layers (actual motion-dependent layer number NumActiveMotionPredRefLayers) used in the derivation process of the alternative collocated picture colPic in the comparative example. Indicates. In the comparative example, the motion dependent layer is set regardless of the sample dependent flag SamplePredEnableFlag.

  Comparative examples shown in FIGS. 15, 17 to 19, and 24 to 25 correspond to FIGS. 16, 20 to 22, and 26 to 27, respectively, of the embodiment of the present invention.

(Configuration of image decoding device)
Next, the configuration of the image decoding device 31 according to an embodiment of the present invention will be described. FIG. 5 is a schematic diagram illustrating a configuration of the image decoding device 31 according to the present embodiment. The image decoding device 31 includes an image data decoding unit 300 and a header decoding unit 400.

  The image data decoding unit 300 decodes image data from the input encoded stream Te based on a slice layer encoding parameter input from the header decoding unit 400 described later. The decoded image data is output to the outside of the image decoding apparatus as a decoded layer image Td (decoded viewpoint image Td). Details of the image data decoding unit 300 will be described later.

  The header decoding unit 400 includes a parameter set decoding unit 401 and a slice segment header decoding unit 402. The parameter set decoding unit 401 converts encoded data of layers higher than the slice layer, such as a video parameter set VPS, a sequence parameter set SPS, and a picture parameter set PPS, included in the encoded stream Te, according to a predetermined syntax rule. Decode and determine parameters required for decoding at each layer. For example, the VPS extension described above is decoded to derive the layer dependency flag direct_dependency_flag [i] [j], the layer dependency type direct_dependency_type [i] [j], and the like.

  The parameter set decoding unit 401 includes a dependency type decoding unit internally.

  FIG. 16 is a diagram for explaining the relationship between the layer dependence type, the sample dependence flag SamplePredEnableFlag, and the motion dependence flag SamplePredEnableFlag according to the embodiment of the present invention. FIG. 16A shows that when the layer dependency type is 0, both the sample dependency flag SamplePredEnableFlag and the motion dependency flag SamplePredEnableFlag are 1.

  FIG. 16B shows the parameter set decoding unit 401 in the dependency type decoding unit derivation processing of the sample dependency flag SamplePredEnableFlag and the motion dependency flag SamplePredEnableFlag.

  As shown in FIG. 16B, the parameter set decoding unit 401 performs SamplePredEnableFlag [iNuhLid] [j] and MotionPredEnableFlag [iNuhLid] [j] for each target layer i and dependent layer j according to the following equations. Is derived.

SamplePredEnabledFlag [iNuhLId] [j] =
((3-(direct_dependency_type [i] [j] & 3)) & 1)
MotionPredEnabledFlag [iNuhLId] [j] =
((((3-(direct_dependency_type [i] [j] & 3)) & 2) >> 1)
In the above equation, the symbol & represents a bit product (logical product), and the symbol >> represents a right bit shift.

  Thereby, SamplePredEnabledFlag and MotionPredEnabledFlag can be derived according to the relationship between the value of direct_dependency_type [i] [j] and the value of the layer dependence type.

  According to the parameter set decoding unit 401 including the above processing, when direct_dependency_type [i] [j] is 0, there is sample prediction dependency, that is, the corresponding SamplePredEnabledFlag [iNuhLId] [j] is 1 and the motion The sample dependency flag SamplePredEnableFlag and the motion dependency flag SamplePredEnableFlag are derived so that there is prediction dependency, that is, the corresponding MotionPredEnabledFlag [iNuhLId] [j] is 1. As a result, the code amount of direct_dependency_type [i] [j] is reduced because 0 represented by the shortest bit length is assigned to the case where there is dependency on the most frequently used sample prediction and motion prediction. There is an effect.

  Note that the SamplePredEnabledFlag and the MotionPredEnabledFlag are derived after taking the logical product (&) of the direct_dependency_type [i] [j] and 3, and the SamplePredEnabledFlag and the DirectPredependency_type [i] [j] MotionPredEnabledFlag can be set. As a result, the low-order 2 bits can express sample dependency and motion dependency, and the bits above the low-order 2 bits can express the presence or absence of dependencies other than sample dependency and motion dependency.

In the above description, nuh_layer_id and iNuhLid are both layer identifiers corresponding to the target layer i. If specified in vps_extension (), use the value (layer_id_in_nuh [i]), otherwise use the NAL unit described above. The layer identifier nuh_layer_id specified in the header is used as it is. These derived parameters are output to the slice segment header decoding unit 402.
(Slice segment header decoding unit)
The slice segment header decoding unit 402 decodes the slice segment header included in the input encoded stream Te based on the parameters of the upper layer input from the parameter set decoding unit 401, and the moving image data of the target slice segment Deriving slice layer coding parameters necessary for decoding. The slice segment header decoding unit 402 further includes an actual dependency layer decoding unit 403, a reference picture set deriving unit 404, and a collocated picture decoding unit 405.

  FIG. 20 shows the state of the layer dependent type of pictures that can be included in the reference picture set RPS according to the embodiment of the present invention. As shown in FIG. 20, in the embodiment of the present invention, only a picture having a sample dependency flag SamplePredEnableFlag derived from the layer dependency type can be included in the reference picture set RPS.

  FIG. 21 is a diagram showing a derivation process of the reference picture set RPS according to the embodiment of the present invention. FIG. 21A shows processing for deriving the actual dependent layer number NumActiveRefLayerPics according to the embodiment of the present invention. This is the same as the comparative example of FIG. FIG. 21B shows a derivation process of the sample dependent layer (actual sample dependent layer identifier ActiveSamplePredRefLayerId []) and the number of sample dependent layers (actual sample dependent layer number NumActiveSamplePredRefLayers) according to the embodiment of the present invention. FIG. 21C illustrates the RPS derivation process according to the embodiment of this invention. As shown in FIG. 21 (c), RefPicSetInterLayer [i] is marked with a long-term prediction reference “used for long-term reference”, whereby a reference picture is put into the RPS. In the process of FIG. 21C, the layer of nuh_layer_id included in the sample dependent layer (actual sample dependent layer identifier ActiveSamplePredRefLayerId []) derived in the process of FIG. Since the sample-dependent layer is a picture whose sample-dependent flag SamplePredEnableFlag is 1, it is determined whether the sample-dependent layer can be put into the RPS according to the state of the layer-dependent type.

  FIG. 22 is a diagram illustrating RPS derivation processing according to the embodiment of the present invention. It can be seen that the sample dependent layer number (actual sample dependent layer number NumActiveSamplePredRefLayers) is 2, and the sample dependent layer (actual sample dependent layer identifier ActiveSamplePredRefLayerId []) is nuh_layer_id = 0,3. Unlike FIG. 19 of the comparative example, a layer with nuh_layer_id = 2 that is not a sample-dependent layer but a motion-dependent layer is not included in the RPS.

  FIG. 23 is a diagram for explaining the inter-layer prediction limitation flag derivation process according to the embodiment of the present invention. As shown in FIG. 23A, when the inter-layer prediction only flag inter_layer_sample_pred_only_flag is 1, the inter prediction RPL valid flag InterRefEnabledInRPLFlag is set to 0. As shown in FIG. 23B, only when the inter prediction RPL valid flag InterRefEnabledInRPLFlag is 1, the temporary reference picture list RefPicListTempX [] includes a short-term forward reference picture set RefPicSetStCurrBefore, a short-term backward reference picture set RefPicSetStCurrAfter [i], In addition, a reference picture list of the long-term reference picture set RefPicSetLtCurr [i] is stored. The reference picture of the inter-layer reference picture list RefPicSetInterLayer is stored regardless of the inter prediction RPL enabled flag InterRefEnabledInRPLFlag. A reference picture list is constructed from the temporary reference picture list RefPicListTempX [].

  FIG. 26 shows a state of a layer dependent type of a picture that can be used for an alternative collocated picture according to an embodiment of the present invention. As shown in FIG. 26A, in the embodiment of the present invention, when the sample dependent flag SamplePredEnableFlag is 1 as the layer dependent type state, even if the motion dependent flag MotionPredEnableFlag is 1, the alternative collocated picture Not used for. This is because a dependent layer whose sample dependency flag SamplePredEnableFlag is 1 can specify colPic using the reference picture index refIdx stored in the reference picture set RPS. In the present embodiment, the reference picture stored in the RPS is specified using the reference picture index identifier collocated_ref_idx, and the reference picture not stored in the RPS is specified using the layer ID identifier collocated_ref_layer_idx, so the colPic is specified. Therefore, there is no redundancy in the information. FIG. 26B shows a motion-dependent limited layer (actual motion-dependent layer identifier ActiveMotionPredRefLayerId []) and a motion-dependent limited layer number (actual motion-dependent layer number NumActiveMotionPredRefLayers) used in the derivation process of the collocated picture colPic according to the embodiment of the present invention. )) Derivation processing is shown. In the present embodiment, the motion dependence limited layer is not set when the sample dependence flag SamplePredEnableFlag is 1, unlike the motion dependence layer of the comparative example shown in FIG. The derivation of the layer-dependent type in FIG. 16, the reference picture set in FIGS. 20 to 22, the reference picture list derivation, the inter-layer prediction limitation in FIG. 23, and the alternative collocated picture derivation in FIG. 26 are used independently. You may use it simultaneously.

  FIG. 12 shows an example of the data structure of the slice segment header decoded by the slice segment header decoding unit 402. The slice segment header decoding unit 402, when a dependent layer exists for a non-base layer (a layer having a layer identifier nuh_layer_id> 0), that is, when the number of dependent layers NumDirectRefLayers [nuh_layer_id] is greater than 0, the actual dependent layer in the target slice segment The syntax for is decoded as follows: First, the inter-layer dependency flag inter_layer_pred_enabled_flag indicating the presence / absence of inter-layer prediction is decoded. When inter-layer prediction is enabled, that is, when inter_layer_pred_enabled_flag = 1, the number of reference pictures between layers is further decoded by num_inter_layer_ref_pics_minus1. However, when the number of dependent layers NumDirectRefLayers [nuh_layer_id] is 1 or when max_one_active_ref_layer_flag specified in the VPS extension is 1, that is, when the actual number of dependent layers is limited to 1, num_inter_layer_ref_pics_minus1 is not decoded.

  The actual dependency layer decoding unit 403 included in the slice segment header decoding unit 402 derives a parameter NumActiveRefLayerPics indicating the number of actual dependency layer pictures related to the target slice segment (target picture) by the process shown in FIG. Specifically, if the target layer is a base layer (nuh_layer_id == 0), there is no reference layer (NumDirectRefLayers [nuh_layer_id] == 0), or inter-layer prediction is not enabled (inter_layer_pred_enabled_flag == 0) Sets NumActiveRefLayerPics = 0. Also, when the number of reference layers is limited to 1 (max_one_active_ref_layer = 1) or when the number of dependent layers NumDirectRefLayers [nuh_layer_id] is 1, NumActiveRefLayerPics = 1 is set. In other cases, num_inter_layer_ref_pics_minus1 is decoded from the encoded data, and the value of num_inter_layer_ref_pics_minus + 1 is set in NumActiveRefLayerPics. The actual dependency layer decoding unit 403 further decodes an inter-layer dependency identifier inter_layer_pred_layer_idc [i] indicating a dependency layer of inter-layer prediction corresponding to the derived actual dependency layer picture number. However, if the target layer is a base layer (nuh_layer_id == 0), there are no reference layers (NumDirectRefLayers [nuh_layer_id] == 0), or inter-layer prediction is not enabled (inter_layer_pred_enabled_flag == 0) In this case, inter_layer_pred_layer_idc [i] is not decoded.

  The actual dependency layer decoding unit 403 uses the reference picture layer identifier RefPicLayerId [], which is layer identification information for each actual dependency layer picture, as actual dependency layer information regarding inter-layer prediction in the target slice segment. NumActiveSamplePredRefLayers and actual motion dependent layer number NumActiveMotionPredRefLayers indicating the number of layers to be referenced in sample prediction and motion prediction, and identification information of the dependent layer corresponding to the actual dependent layer number of each of sample prediction and motion prediction The real sample dependent layer identifier ActiveSamplePredRefLayerId [] and the actual motion dependent layer identifier ActiveMotionPredRefLayerId [] are derived. The procedure for deriving these parameters is shown in FIG. 21 (b) and FIG.

  As shown in FIG. 21B, the actual dependency layer decoding unit 403 performs the following processing on each actual dependency layer i that takes a value of 0 from the actual dependency layer number NumActiveRefLayerPics. First, the dependent layer identifier inter_layer_pred_layer_idc [i] is specified from the actual dependent layer i.

  Specifically, the actual dependency layer decoding unit 403 refers to the sample dependency flag SamplePredEnabledFlag [] [] derived by the parameter set decoding unit 401 corresponding to each actual dependency layer i, and refers to the reference layer for the target layer nuh_layer_id If inter_layer_pred_layer_idc [i] indicates sample dependency, that is, SamplePredEnabledFlag [nuh_layer_id] [inter_layer_pred_layer_idc [i]] is 1, the sample dependent layer (here, layer ID, RefLayerId [nuh_layer_id] [inter_layer_pred_layer_idc [i] ) Is added to the actual sample prediction layer according to the following equation:

ActiveSamplePredRefLayerId [k ++] = RefLayerId [nuh_layer_id] [inter_layer_pred_layer_idc [i]]
The count k of the actual sample dependent layer number NumActiveSamplePredRefLayers is incremented by one.

  The actual sample dependent layer number NumActiveSamplePredRefLayers is obtained as the count k at the time when all the actual dependent layers i have been processed.

  As a result of the above processing, an actual sample dependent layer ActiveSamplePredRefLayerId [] is obtained that includes only a dependency layer whose layer dependency type indicates sample dependency.

  ActiveSamplePredRefLayerId [] is called actual dependency layer information related to sample prediction.

As shown in FIG. 26B, the actual dependency layer decoding unit 403 performs the following processing for each actual dependency layer i that takes a value of 0 from the actual dependency layer number NumActiveRefLayerPics. First, the dependent layer identifier inter_layer_pred_layer_idc [i] is specified from the actual dependent layer i.
When the sample dependency flag SamplePredEnabledFlag [] [] derived by the parameter set decoding unit 401 is referred to and the reference layer inter_layer_pred_layer_idc [i] with respect to the target layer nuh_layer_id indicates motion limited dependency, that is, SamplePredEnabledFlag [nuh_layer_id] [inter_layer_pred_layer_idc [i ]] Is 0 and MotionPredEnabledFlag [nuh_layer_id] [inter_layer_pred_layer_idc [i]] is 1, the motion-dependent layer (here, layer ID, RefLayerId [nuh_layer_id] [inter_layer_pred_layer_idc [i]]) is To the actual motion prediction layer.

ActiveMotionPredRefLayerId [k ++] = RefLayerId [nuh_layer_id] [inter_layer_pred_layer_idc [i]]
The count j of the actual motion dependent layer number NumActiveMotionPredRefLayers is incremented by one.

  The actual motion dependent layer number NumActiveMotionPredRefLayers is obtained as the count j when all the actual dependent layers i have been processed.

  With the above processing, an actual motion dependent layer ActiveMotionPredRefLayerId [] including only a dependent layer whose layer dependence type indicates motion limited dependence is obtained.

  The actual dependency layer decoding unit 403 including the above processing sets the actual motion dependency layer identifier ActiveMotionPredRefLayerId only when the sample dependency flag SamplePredEnabledFlag [] [] corresponding to the dependency layer is 0. With this configuration, as will be described later, only the layer set in the actual motion dependent layer identifier ActiveMotionPredRefLayerId is used for the derivation process of the alternative collocated picture colPic. There is an effect that the processing can be omitted.

  If there is a sample dependent layer for the target picture, that is, if the actual sample dependent layer number NumActiveSamplePredRefLayers is greater than 0, the actual dependent layer decoding unit 403 further indicates whether or not to limit inter-layer prediction to sample prediction. The inter-sample prediction only flag inter_layer_sample_pred_only_flag is decoded. When the inter-layer sample prediction only flag is 1, inter-frame prediction within the same image layer is not used when the target picture is decoded, and when the flag is 0, inter-frame prediction within the same image layer is used. It is done.

  The reference picture set deriving unit 404 derives a reference picture set RPS used for the decoding process of the current picture based on the derived actual dependency layer information.

  Details of the RPS deriving process performed by the reference picture set deriving unit 404 will be described.

  The RPS derivation process can divide a plurality of RPS subsets into derivation processes. A subset of RPS is defined as follows:

The RPS is divided into the following two subsets depending on the type of the referenceable picture.
-Current picture reference picture set ListCurr: List of pictures that can be referred to in the target picture-Subsequent picture reference picture set ListFoll: List of pictures that are not referenced in the target picture but can be referred to in subsequent pictures in decoding order in the target picture The number of pictures included in the picture reference picture set is called the current picture referenceable picture number NumCurrList.

The current picture reference picture set is further composed of four partial lists.
・ Long-term reference picture set RefPicSetLtCurr: Picture that can be referred to the current picture specified by SPS long-term RP information or SH long-term RP information ・ Short-term forward reference picture set ListStCurrBefore: Current picture reference specified by SPS short-term RPS information or SH short-term RPS information A picture whose display order is earlier than the target picture. Short-term backward reference picture set ListStCurrAfter: current picture referenceable picture specified by SPS short-term RPS information or SH short-term RPS information, and the display order is earlier than the target picture. Inter-layer reference picture set RefPicSetInterLayer: Current picture referenceable picture specified by ILRP information The subsequent picture reference picture set is further composed of two partial lists.
Subsequent picture long-term reference picture set ListLtFoll: A picture that can be referred to a subsequent picture specified by SPS long-term RP information or SH long-term RP information.
Subsequent picture short-term reference picture set ListStFoll: Picture that can be referred to the current picture specified by SPS short-term RPS information or SH short-term RPS information.

Short-term forward reference picture set ListStCurrBefore, short-term backward reference picture set ListStCurrAfter, long-term reference picture set RefPicSetLtCurr, inter-layer reference picture set RefPicSetInterLayer, subsequent picture short-term reference picture set RefPicSetFoll, and subsequent picture long-term reference picture set ListLtFoll Generate by the following procedure. In addition, the current picture referenceable picture number NumPocTotalCurr is derived. Each reference picture set is set to be empty before starting the following processing.
(S101) Based on the SPS short-term RPS information and the SH short-term RPS information, a single short-term reference picture set used for decoding the current picture is specified. Specifically, when the value of short_term_ref_pic_set_sps included in the SH short-term RPS information is 0, the short-term RPS explicitly transmitted in the slice segment header included in the SH short-term RPS information is selected. Other than that (when the value of short_term_ref_pic_set_sps is 1, the short-term RPS indicated by short_term_ref_pic_set_idx included in the SH short-term RPS information is selected from a plurality of short-term RPSs included in the SPS short-term RPS information.
(S102) The POC of each reference picture included in the selected short-term RPS is derived. When the reference picture is a forward short-term reference picture, the reference picture POC is derived by subtracting the value of “delta_poc_s0_minus1 [i] +1” from the POC of the target picture. On the other hand, when the reference picture is a backward short-term reference picture, it is derived by adding the value of “delta_poc_s1_minus1 [i] +1” to the POC of the target picture.
(S103) The forward reference pictures included in the short-term RPS are confirmed in the order of transmission, and when the value of used_by_curr_pic_s0_flag [i] associated with the forward reference picture is 1, the forward reference picture is added to the short-term forward reference picture set RefPicSetCurrBefore. Otherwise (used_by_curr_pic_s0_flag [i] value is 0), the forward reference picture is added to the subsequent picture short-term reference picture set RefPicSetFoll.
(S104) The backward reference pictures included in the short-term RPS are confirmed in the order of transmission. If the value of used_by_curr_pic_s1_flag [i] associated with the backward reference picture is 1, the backward reference picture is added to the short-term backward reference picture set RefPicSetCurrAfter. Other than that (when the value of used_by_curr_pic_s1_flag [i] is 0, the forward reference picture is added to the subsequent picture short-term reference picture set RefPicSetFoll.
(S105) Based on the SPS long-term RP information and the SH long-term RP information, a long-term reference picture used for decoding the current picture is specified. Specifically, num_long_term_sps reference pictures are selected from the reference pictures included in the SPS long-term RP information, and are sequentially added to the long-term RPS. The selected reference picture is the reference picture indicated by lt_idx_sps [i]. Subsequently, reference pictures included in the SH long-term RP information are sequentially added to the long-term RPS as many reference pictures as num_long_term_pics.
(S106) The POC of each reference picture included in the long-term RPS is derived. The POC of the long-term reference picture is directly derived from the value of poc_lst_lt [i] or lt_ref_pic_poc_lsb_sps [i] decoded in association with each other.
(S107) Check the reference pictures included in the long-term RPS in order, and if the value of associated used_by_curr_pic_lt_flag [i] or used_by_curr_pic_lt_sps_flag [i] is 1, add the long-term reference picture to the long-term reference picture set RefPicSetLtCurr To do. In other cases (used_by_curr_pic_lt_flag [i] or used_by_curr_pic_lt_sps_flag [i] has a value of 0), the long-term reference picture is added to the subsequent picture long-term reference picture set ListLtFoll.
(S108) The reference picture (inter-layer reference picture) is added to the inter-layer reference picture set RefPicSetInterLayer according to the real dependence layer information (ILRP information) decoded by the real dependence layer decoding unit 403. Specifically, as shown in FIG. 21C, when a picture picX of the same time as the target picture is in the decoded picture buffer and the layer nuh_layer_id of the picX is equal to ActiveSamplePredRefPicLayerId [i], an inter-layer reference picture set PicX is set to RefPicSetInterLayer [i], and picX puts a mark indicating a used for long-term reference. If the above condition is not satisfied, a mark indicating that there is no reference picture is attached to RefPicSetInterLayer [i]. The reference picture marked with the long-term reference picture in the above process is added to the inter-layer reference picture set RefPicSetInterLayer as an inter-layer reference picture.
(S109) The value of the variable NumPocTotalCurr is set to the sum of reference pictures that can be referenced from the current picture. That is, the value of the variable NumPocTotalCurr is set to the sum of the numbers of elements in the four lists of the short-term forward reference picture set RefPicSetCurrBefore, the short-term backward reference picture set RefPicSetCurrAfter, the long-term reference picture set RefPicSetLtCurr, and the inter-layer reference picture set RefPicSetInterLayer. The addition of the number of elements of the inter-layer reference picture set RefPicSetInterLayer is performed by adding the number of sample dependent layers (number of actual sample dependent layers NumActiveSamplePredRefLayers) as shown in FIG. The reference picture set deriving unit 404 constructs a reference picture list (RPL). Details of the reference picture list construction process will be described below. The reference picture list deriving unit generates a reference picture list RPL based on the reference picture set RPS and the RPL modification information.

The reference picture list is composed of two lists, an L0 reference list and an L1 reference list. First, the construction procedure of the LX reference list (X is 0 or 1) will be described. As shown in FIG. 23 (b), the L0 reference list is constructed according to the procedure shown in S301 to S307 below.
(S301) A provisional LX reference list RefPicListTempX is generated and initialized to an empty list.
(S302) When the inter prediction RPL valid flag InterRefEnabledInRPLFlag is 1, reference pictures included in the short-term forward reference picture set RefPicSetStCurrBefore are sequentially added to the provisional LX reference list RefPicListTempX.
(S303) When the inter prediction RPL valid flag InterRefEnabledInRPLFlag is 1, reference pictures included in the short-term backward reference picture set RefPicSetStCurrAfter are sequentially added to the provisional LX reference list.
(S304) When the inter prediction RPL valid flag InterRefEnabledInRPLFlag is 1, reference pictures included in the long-term reference picture set RefPicSetLtCurr are sequentially added to the provisional LX reference list.
(S305) Reference pictures included in the inter-picture layer reference picture set RefPicSetInterLayer are sequentially added to the provisional LX reference list.
(S306) When the lists_modification_present_flag decoded from the encoded data is 1, the provisional LX reference list is modified.
(S307) The provisional LX reference list is set as an LX reference list.

  Next, the L1 reference list is constructed by a similar process. However, in the construction of the L1 reference list, the processing order of S303 and S302 is reversed. That is, the short-term forward reference picture set RefPicSetStCurrAfter is stored before the short-term forward reference picture set RefPicSetStCurrBefore.

  According to the reference picture set deriving unit 404 having the above configuration, it is guaranteed that all the reference pictures stored in the RPS can be sample-predicted, so that the inter-layer sample prediction process can be simplified. There is an effect.

  When temporal motion prediction is valid in the target slice segment (slice_temporal_mvp_enabled_flag == 1), the collocated picture decoding unit 405 decodes a parameter for specifying the reference picture colPic for temporal motion prediction from the slice segment header. If the target layer is the base layer, the actual motion-dependent layer number NumActiveMotionPredRefLayers is 0, or the alternative collocated indication flag alt_collocated_indication_flag is 0, the reference picture index of the temporal motion prediction is used. If it is specified and alt_collocated_indication_flag is 1, it is specified using a collocated reference layer indicator collocated_ref_layer_idx.

  As shown in FIG. 12, the collocated picture decoding unit 405 substitutes when temporal motion prediction is effective, the target layer is a non-base layer (nuh_layer_id> 0), and an actual motion dependent layer exists (NumActiveMotionPredRefLayers> 0). The collocated indication flag alt_collocated_indication_flag is decoded. When the alternative collocation instruction flag is not decoded, the collocated picture decoding unit 405 sets 0 to alt_collocated_indication_flag. When the alternative collocation indication flag alt_collocated_indication_flag is 1, the collocated picture decoding unit further decodes the collocated reference layer indicator collocated_ref_layer_idx. However, when the number of actual motion-dependent layers is 1 (NumActiveMotionPredRefLayers == 1), the collocated picture decoding unit does not decode the collocated reference layer indicator and sets collocated_ref_layer_idx to 0.

  When the alternative collocation indication flag is not valid (alt_collocated_indication_flag is 0), the collocated picture decoding unit 405 sets the picture indicated by the index collocated_ref_idx of the reference picture list selected by collocated_from_l0_flag as the reference picture colPic for temporal motion prediction . Specifically, when collocated_from_l0_flag is 0, the collocated picture decoding unit 405 sets the picture indicated by RefPicList1 [collocated_ref_idx] to the reference picture colPic, and when collocated_from_l0_flag is 1, it is indicated by RefPicList0 [collocated_ref_idx] Set the picture to be the reference picture colPic.

  When the alternative collocation indication flag is valid (alt_collocated_indication_flag is 1), the collocated picture decoding unit 405 at the same time in the layer indicated by the actual motion dependent layer identifier ActiveMotionPredRefLayerId [collocated_ref_layer_idx] corresponding to the collocated reference layer indicator collocated_ref_layer_idx The picture is set as a reference picture colPic for temporal motion prediction.

(Configuration of image data decoding unit)
Next, the configuration of the image data decoding unit 300 will be described.

  FIG. 8 is a schematic diagram illustrating a configuration of the image data decoding unit 300 according to the present embodiment. The image data decoding unit 300 includes an entropy decoding unit 301, a prediction parameter decoding unit 302, a reference picture memory (reference image storage unit, frame memory) 306, a prediction parameter memory (prediction parameter storage unit, frame memory) 307, and a prediction image generation unit. 308, an inverse quantization / inverse DCT unit 311, an adder 312, and an inter-layer motion mapping unit 313.

  The prediction parameter decoding unit 302 includes an inter prediction parameter decoding unit 303 and an intra prediction parameter decoding unit 304. The predicted image generation unit 308 includes an inter predicted image generation unit 309 and an intra predicted image generation unit 310.

  The entropy decoding unit 301 performs entropy decoding on the input encoded data, and separates and decodes individual codes (syntax elements). The entropy decoding unit uses the information decoded by the header decoding unit 400 to determine the presence or absence of some codes. As information used at that time, for example, there is slice_type indicating the coding type of the target slice. The separated codes include prediction information for generating a prediction image and residual information for generating a difference image.

  The entropy decoding unit 301 outputs a part of the separated code to the prediction parameter decoding unit 302. Some of the separated codes are, for example, a prediction mode predMode, a partition mode part_mode, a merge flag merge_flag, a merge index merge_idx, an inter prediction flag inter_pred_idc, a reference picture index refIdxLX, a prediction vector index mvp_LX_idx, a difference vector mvdLX, and the like. Control of which code to decode is performed based on an instruction from the prediction parameter decoding unit 302. The entropy decoding unit 301 outputs the quantization coefficient to the inverse quantization / inverse DCT unit 311. This quantization coefficient is a coefficient obtained by performing quantization and performing DCT (Discrete Cosine Transform) on the residual signal in the encoding process.

  The prediction parameter decoding unit 302 decodes the inter prediction parameter or the intra prediction parameter based on the slice layer coding parameter input from the header decoding unit 400 and the code input from the entropy decoding unit 301. The decoded prediction parameter is output to the prediction image generation unit 308 and stored in the prediction parameter memory 307.

The inter prediction parameter decoding unit 303 refers to the prediction parameters stored in the prediction parameter memory 307 and outputs inter prediction parameters. The inter prediction parameters include, for example, a prediction list use flag predFlagLX, a reference picture index refIdxLX, a motion vector mvLX, and the like. For example,
The inter prediction parameter decoding unit 303 includes a temporal motion vector deriving unit. When the temporal motion prediction is valid, the temporal motion vector deriving unit reads the motion parameter of the spatial neighborhood block on the picture specified by the reference picture colPic set by the collocated picture decoding unit 405 from the prediction parameter memory, and reads the read motion parameter Based on the above, the motion vector mvLX of the target block is derived.

  Specifically, the temporal motion vector deriving unit identifies a collocated block from the reference picture colPic, and derives an availability flag availableFlagLXCol indicating the availability of the temporal motion vector mvLXCol and the temporal motion vector mvLXCol from the collocated block.

  Next, the temporal motion vector deriving unit obtains the coordinates of the block colBr adjacent to the lower right of the block having the same coordinates as the target block in the reference picture colPic. When the coordinates of the target block are (xPb, yPb), the width of the target block is nPbW, and the height is nPbH, the coordinates (xColBr, yColBr) of the block colBr are obtained by the following equations.

xColBr = xPb + nPbW
yColBr = yPb + nPbH
Next, the temporal motion vector deriving unit determines whether or not the coordinates of the block colBr are present in the coding tree unit row including the coordinates of the target block, and the coordinates of the block Br indicate the inside of the picture. Whether or not the coordinates of the block colBr are present in the coding tree unit row including the coordinates of the target block is determined by the following equation (A-1) when the logarithm of 2 of the size of the coding tree unit is CtbLog2SizeY. To do.

(yPb >> CtbLog2SizeY) == (yColBr >> CtbLog2SizeY) (A-1)
Whether or not the block colBr indicates the inside of a picture is determined by the following equation (A-2) when the horizontal width of the picture is pic_width_in_luma_samples and the height is pic_height_in_luma_samples.

xColPb <pic_width_in_luma_samples && yColPb <pic_height_in_luma_samples (A-2)
The temporal motion vector derivation unit sets the coordinates (xColPb, yColPb) of the collocated block by the following formulas when the results of formulas (A-1) and (A-2) are both true.

xColPb = (xColBr >> 4) << 4
yColPb = (yColBr >> 4) << 4
The temporal motion vector deriving unit derives a temporal motion vector mvLXCol from the motion vector mvLX included in the collocated block, and sets 1 to the availability flag availableFlagLXCol. However, when the prediction mode of the collocated block is the intra mode, 0 is set to the temporal motion vectors mvLXCol [0] and mvLXCol [1], and 0 is set to the availability flag availableFlagLXCol.

  On the other hand, the temporal motion vector deriving unit sets 0 to the temporal motion vectors mvLXCol [0] and mvLXCol [1] when either of the results of the expressions (A-1) and (A-2) is false. The availability flag availableFlagLXCol is set to 0.

  When the availability flag availableFlagLXCol is 0, the temporal motion vector derivation unit sets the block at the center of the block having the same coordinates as the target block in the reference picture colPic as a new collocated block. When the coordinates of the target block are (xPb, yPb), the width of the target block is nPbW, and the height is nPbH, the coordinates of the collocated block (xColPb, yColPb) are obtained by the following equations.

xColPb = ((xPb + (nPbW >> 1)) >> 4) << 4
yColPb = ((yPb + (nPbH >> 1)) >> 4) << 4
Next, the temporal motion vector derivation unit derives a temporal motion vector mvLXCol from the motion vector mvLX included in the collocated block, and sets 1 to the availability flag availableFlagLXCol. However, when the prediction mode of the collocated block is the intra mode, 0 is set to the temporal motion vectors mvLXCol [0] and mvLXCol [1], and 0 is set to the availability flag availableFlagLXCol.

  The derived temporal motion vector mvLXCol is used as a motion vector of the block when the block is in the merge mode. The temporal motion vector mvLXCol is also used as a predicted motion vector, and a motion vector mvLX is derived by taking the sum of the difference vector obtained from the entropy decoding unit 301.

  Based on the code input from the entropy decoding unit 301, the intra prediction parameter decoding unit 304 refers to the prediction parameter stored in the prediction parameter memory 307 and decodes the intra prediction parameter. The intra prediction parameter is a parameter used in a process of predicting a picture block within one picture, for example, an intra prediction mode IntraPredMode. The intra prediction parameter decoding unit 304 outputs the decoded intra prediction parameter to the prediction image generation unit 308 and stores it in the prediction parameter memory 307.

  The reference picture memory 306 stores a reference picture block (reference picture block) generated by an adder 312 described later at a predetermined position for each picture and block to be decoded.

  The prediction parameter memory 307 stores the prediction parameter at a predetermined position for each picture and block to be decoded. Specifically, the prediction parameter memory 307 stores the inter prediction parameter decoded by the inter prediction parameter decoding unit 303, the intra prediction parameter decoded by the intra prediction parameter decoding unit 304, and the prediction mode predMode separated by the entropy decoding unit 301. . The stored inter prediction parameters include, for example, a prediction list use flag predFlagLX (inter prediction flag inter_pred_idc), a reference picture index refIdxLX, and a vector mvLX.

  The prediction image generation unit 308 receives the prediction mode predMode input from the entropy decoding unit 301, and also receives prediction parameters from the prediction parameter decoding unit 302. Further, the predicted image generation unit 308 reads a reference picture from the reference picture memory 306. The predicted image generation unit 308 generates a predicted picture block P (predicted image) using the input prediction parameter and the read reference picture according to the prediction mode indicated by the prediction mode predMode.

  Here, when the prediction mode predMode indicates the inter prediction mode, the inter prediction image generation unit 309 uses the inter prediction parameter input from the inter prediction parameter decoding unit 303 and the read reference picture to perform the prediction picture block P by inter prediction. Is generated. The predicted picture block P corresponds to the PU. The PU corresponds to a part of a picture composed of a plurality of pixels as a unit for performing the prediction process as described above, that is, a decoding target block on which the prediction process is performed at once.

  The inter prediction image generation unit 309 generates a prediction image using a reference picture specified by a reference picture list (RPL) derived from a reference picture set (RPS).

  That is, for a reference picture list (RefPicListL0 or RefPicListL1) whose prediction list usage flag predFlagLX is 1, a vector mvLX indicates from a reference picture (RefPicListLX [refIdxLX]) indicated by a reference picture index refIdxLX with reference to a decoding target block The reference picture block at the position is read from the reference picture memory 306. The inter prediction image generation unit 309 performs prediction on the read reference picture block to generate a prediction picture block P. The inter prediction image generation unit 309 outputs the generated prediction picture block P to the addition unit 312.

  When the prediction mode predMode indicates the intra prediction mode, the intra predicted image generation unit 310 performs intra prediction using the intra prediction parameter input from the intra prediction parameter decoding unit 304 and the read reference picture. Specifically, the intra predicted image generation unit 310 reads, from the reference picture memory 306, a reference picture block that is a decoding target picture and is in a predetermined range from the decoding target block among blocks that have already been decoded. The predetermined range is, for example, any of the left, upper left, upper, and upper right adjacent blocks when the decoding target block sequentially moves in a so-called raster scan order, and varies depending on the intra prediction mode. The raster scan order is an order in which each row is sequentially moved from the left end to the right end in each picture from the upper end to the lower end.

  The intra predicted image generation unit 310 generates a predicted picture block by performing prediction in the prediction mode indicated by the intra prediction mode IntraPredMode for the read reference picture block. The intra predicted image generation unit 310 outputs the generated predicted picture block P to the addition unit 312.

  The inverse quantization / inverse DCT unit 311 performs inverse quantization on the quantization coefficient input from the entropy decoding unit 301 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 311 performs inverse DCT (Inverse Discrete Cosine Transform) on the obtained DCT coefficient to calculate a decoded residual signal. The inverse quantization / inverse DCT unit 311 outputs the calculated decoded residual signal to the adder 312.

  The adder 312 outputs the prediction picture block P input from the inter prediction image generation unit 309 and the intra prediction image generation unit 310 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 311 for each pixel. Addition to generate a reference picture block. The adder 312 stores the generated reference picture block in the reference picture memory 306, and outputs a decoded layer image Td in which the generated reference picture block is integrated for each picture to the outside.

  In one configuration of the image decoding device 31 having the above configuration, as described with reference to FIGS. 20 to 22, the layer dependence type is decoded, and whether the layer corresponding to the layer dependence type is a layer showing sample dependence is determined. A dependency type decoding unit having a means for determining, a dependency layer deriving unit (actual dependency layer decoding unit 403) for extracting a sample dependency layer from the dependency layer indicating the sample dependency, and deriving a reference picture set from the sample dependency layer A reference picture set deriving unit 404 and an inter predicted image generating unit 309 that generates a predicted image using a reference picture specified by a reference picture list derived using the inter-layer reference picture set. To do.

  More specifically, in the image decoding device 31 configured as described above, the dependency type decoding unit decodes the layer dependent type from the encoded data of the parameter set, and the dependent layer is a sample dependent layer according to the layer dependent type. SamplePredEnableFlag, which is a flag indicating whether or not, is derived, and in the dependent layer deriving unit (actually dependent layer decoding unit 403), the inter layer dependency identifier (inter_layer_pred_layer_idc) of the inter layer prediction reference picture is decoded from the encoded data of the slice segment layer Then, from the dependency layer whose SamplePredEnableFlag is 1, the sample dependency layer ActiveSamplePredRefLayerId is derived.

  In one configuration of the image decoding device 31 having the above configuration, as described with reference to FIG. 26, an inter-layer prediction limited flag decoding unit (actually dependent layer decoding unit 403) that decodes an inter-layer prediction limited flag inter_layer_sample_only flag is provided. The sample dependent layer decoding means (real dependent layer decoding unit 403) derives the number of sample dependent layers NumActiveSamplePredRefLayers according to the layer dependent type, and the reference picture set deriving unit 404 includes the inter-layer prediction limited flag. When the inter_layer_sample_only flag indicates only inter-layer prediction, a picture of the same layer is not included in the reference picture list, and the inter-layer prediction limited flag decoding unit is configured such that when the number of motion-dependent dependent layers NumActiveSamplePredRefLayers is 1 or more, Special feature for decoding inter-layer prediction only flag inter_layer_sample_only flag To.

  In one configuration of the image decoding device 31 configured as described above, as described with reference to FIGS. 26 and 12, the layer-dependent type is decoded, and the layer corresponding to the layer-dependent type is determined not to be motion-dependent and sample-dependent. A dependency type decoding unit comprising means for determining whether the motion dependence limited layer is indicated, an alternative reference picture decoding unit (collocated picture decoding unit 405) for decoding an alternative reference picture from the motion dependency limited layer, and the alternative reference An inter prediction parameter decoding unit 303 that derives a temporal motion vector using a picture is provided.

  In one configuration of the image decoding device 31 having the above configuration, as described with reference to FIG. 25, the alternative reference picture decoding unit (collocated picture decoding unit 405) is motion-dependent and sample-dependent depending on the layer-dependent type. Deriving the actual motion dependent layer number NumActiveMotionPredRefLayers, which is a dependent layer indicating that, if the actual motion dependent layer number NumActiveMotionPredRefLayers is 1 or more, whether to use an alternative reference picture from the encoded data of the slice segment layer It is characterized by decoding the indicated flag alt_collocated_indication_flag.

(Second Embodiment)
Next, a second embodiment of the image decoding apparatus according to the present invention will be described. FIG. 6 is a schematic diagram illustrating the configuration of the image decoding device 31a according to the present embodiment. The image decoding device 31 includes an image data decoding unit 300 and a header decoding unit 400a.

  Since the image data decoding unit 300 is the same as that of the first embodiment described above, description thereof is omitted. Similarly, functional blocks having the same reference numerals as those already described function in the same manner in this embodiment, and thus description thereof will be omitted.

  The header decoding unit 400a includes a parameter set decoding unit 401, a slice segment header basic decoding unit 4021, and a slice segment header extended decoding unit 4022a. Based on the flags and parameters decoded and determined by the parameter set decoding unit 401, the slice segment header basic decoding unit 4021 decodes the encoding parameters common to the base layer and the non-base layer in the slice header SH. Common encoding parameters are, for example, slice_type indicating the encoding type of the target slice, slice_temporal_mvp_enabled_flag indicating whether temporal motion prediction is valid, or the like. The slice segment header extension decoding unit 4022a decodes the encoding parameters related to the non-base layer. The encoding parameters related to the non-base layer are, for example, a flag inter_layer_pred_enabled_flag indicating whether inter-layer prediction is enabled, alt_collocated_indication_flag indicating col- lation prediction information, collocated_ref_layer_idx, or the like.

  The encoded stream input to the header decoding unit may not include the slice segment header extension itself. Whether or not the slice segment header extension is included is determined by the parameter set decoding unit. For example, the parameter set decoding unit decodes a flag slice_segment_header_extension_present_flag included in the picture parameter set PPS that indicates whether the slice segment header extension is valid. If the flag is valid, the slice segment header extension is encoded. It is determined that it is included in the stream. In that case, the slice segment header extension decoding unit decodes the slice segment header extension.

  FIG. 14 shows an example of the data structure of the slice segment header extension.

  The slice segment header extension decoding unit 4022a decodes the presence / absence of inter-layer prediction (inter_layer_pred_enable_flag), the number of inter-layer prediction reference pictures (num_inter_layer_ref_pics_minus1), and the inter-layer dependency identifier (inter_layer_pred_layer_idc [i]) from the input slice segment header extension . The decoding procedure is the same as that of the slice segment header decoding unit 402 described in the above embodiment. When inter_layer_pred_enable_fla is not decoded from the encoded data, the slice segment header extension decoding unit sets a value indicating that inter-layer prediction is performed as in the following equation.

inter_layer_pred_enable_flag = 1
Further, the slice segment header extension decoding unit, when not decoding num_inter_layer_ref_pics_minus1 indicating the number of inter-layer prediction reference pictures from the encoded data, the NumDirectRefLayers [nuh_layer_id, which is the number of dependent layers decoded from the VPS extension, to the number of inter-layer prediction reference pictures ] Is set. Specifically, since num_inter_layer_ref_pics_minus1 is a value obtained by subtracting 1 from the number of inter-layer prediction reference pictures, it is set as the following equation.

num_inter_layer_ref_pics_minus1 = NumDirectRefLayers [nuh_layer_id] -1
Further, the slice segment header extension decoding unit sets the inter-layer dependency identifier of the inter-layer prediction reference picture so as to indicate that the dependency layer decoded from the VPS extension is used as it is. Specifically, the following equation is set.

inter_layer_pred_layer_idc [i] = i
According to the above configuration, when the slice segment header extension itself is omitted and not input, inter_layer_pred_enable_flag, num_inter_layer_ref_pics_minus1, and inter_layer_pred_layer_idc are not decoded from the encoded data, and thus the values described above are set.

(Collocate picture decoding unit 405)
The slice segment header extended decoding unit 4022a may be configured to include a collocated picture decoding unit 405a. The collocated picture decoding unit 405a decodes the alternative collocation indication flag (alt_collocated_indication_flag) and the collocated reference layer indicator (collocated_ref_layer_idx) from the input slice segment header extension. The decoding procedure is the same as that of the collocated picture decoding unit 405 described in the above embodiment. When the alternative collocation indication flag alt_collocated_indication_flag is not decoded from the encoded data, the collocated picture decoding unit 405a uses, as information corresponding to the alternative collocation indication flag, a reference picture at the time of temporal motion vector prediction as a collocated reference layer indicator that is an alternative means. In order to indicate that it is not specified in, set as the following formula.

alt_collocated_indication_flag = 0
(Modification of Colocate Picture Decoding Unit 405a)
In the modified example of the collocated picture decoding unit 405a, when the alternative collocation indication flag alt_collocated_indication_flag is not decoded from the encoded data, there is no motion dependent layer for the target layer (NumMotionPredRefLayers [i] == 0), the alternative collocation indication flag If the alt_collocated_indication_flag = 0 is set and the motion dependent layer exists (NumMotionPredRefLayers [i]> 0) indicating that it is not specified by the collocated reference layer indicator that is an alternative means as the information corresponding to As information corresponding to the flag, alt_collocated_indication_flag = 1 is set, which indicates that the information is specified by a collocated reference layer indicator that is an alternative means.

(Modification of image decoding device)
Hereinafter, an image decoding device 31b will be described as a modification of the image decoding device. FIG. 7 is a schematic diagram illustrating a configuration of the image decoding device 31b. The image decoding device 31b includes a header decoding unit 400b and an image data decoding unit 300.

  The header decoding unit 400b includes a parameter set decoding unit 401 and a slice segment header decoding unit 402b. The slice segment header decoding unit 402b is configured to include a real dependency layer decoding unit 403, a reference picture set deriving unit 404, and a collocated picture decoding unit 405b.

  The collocated picture decoding unit 405b decodes the alternative collocation indication flag (alt_collocated_indication_flag) and the collocated reference layer indicator (collocated_ref_layer_idx) from the slice segment header extension.

  When the alt_collocated_indication_flag is not decoded from the encoded data, the collocated picture decoding unit 405b sets an alternative collocation indication flag (alt_collocated_indication_flag) according to the reference picture information derived by the reference picture set deriving unit 404.

  Specifically, when the reference picture information derived by the reference picture set deriving unit 404 includes a picture of the same target layer, the collocated picture decoding unit 405b sets the alternative reference layer not to be used (alt_collocated_indication_flag = 0) When the picture of the same target layer is not included in the reference picture information, an alternative reference layer is set to be used (alt_collocated_indication_flag = 1). When alt_collocated_indication_flag = 1 is set, the collocated picture decoding unit 405b sets the actual motion dependent layer identifier of the base layer as information corresponding to the collocated reference layer indicator (collocated_ref_layer_idx = ActiveMotionPredRefLayerId [0]). The collocated picture decoding unit 405b derives a temporal motion prediction reference picture colPic based on the above information.

  As described above, by defining the initial parameters related to inter-layer prediction for non-base layers, even when inter-layer prediction is enabled, the slice segment header extension is omitted to reduce the transmission code amount and to improve the coding efficiency. Can be improved. In addition, regarding slice layer information, by separating the slice segment header into a basic part and an extended part, the influence on the image decoding apparatus and software conforming to the standard of only a single layer is suppressed, and the image decoding apparatus compatible with a single layer, The effect of facilitating the correspondence to a plurality of layers (such as a plurality of viewpoint images) based on software can be obtained.

(Configuration of image encoding device)
Next, the configuration of the image encoding device 11 according to an embodiment of the present invention will be described. FIG. 9 is a schematic diagram illustrating a configuration of the image encoding device 11 according to the present embodiment. The image encoding device 11 includes an image data encoding unit 1000 and a header encoding unit 1400.

  The image data encoding unit 1000 encodes a plurality of layer images T input from the outside of the apparatus to generate encoded image data VD. Also, parameters such as the encoding mode determined in the course of the encoding process are output to the header encoding unit 1400.

  The header encoding unit 1400 includes a parameter set encoding unit 1401 and a slice segment header encoding unit 1402. The parameter set encoding unit 1401 generates a parameter set of a higher layer than the slice layer based on the plurality of layer images T and the parameters input from the image data encoding unit 1000. Upper layer parameters are, for example, a video parameter set VPS, a sequence parameter set SPS, and a picture parameter set PPS. Similarly, the slice segment header encoding unit 1402 generates a slice layer parameter. The parameter of the slice layer is, for example, a slice segment header, or a slice segment header and a slice segment header extension. The header encoding unit 1400 outputs header information HI including the generated parameter set and the like. Details of the header encoding unit 1400 will be described later.

  The image encoding device 11 generates an encoded stream Te obtained by connecting the generated header information HI and the encoded image data VD according to a predetermined rule, and outputs the encoded stream Te to the outside of the device.

(Configuration of image encoding unit)
Next, the configuration of the image data encoding unit 1000 will be described.

  FIG. 10 is a schematic diagram illustrating a configuration of the image data decoding unit 300 according to the present embodiment. The image data encoding unit 1000 includes a prediction image generation unit 101, a subtraction unit 102, a DCT / quantization unit 103, an entropy encoding unit 104, an inverse quantization / inverse DCT unit 105, an addition unit 106, a prediction parameter memory (prediction parameter). A storage unit, a frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determination unit 110, a prediction parameter encoding unit 111, and an inter-layer motion mapping unit 1125 are configured. The prediction parameter encoding unit 111 includes an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113.

  The predicted image generation unit 101 generates a predicted picture block P for each block which is an area obtained by dividing the picture for each viewpoint of the layer image T input from the outside. Here, the predicted image generation unit 101 reads the reference picture block from the reference picture memory 109 based on the prediction parameter input from the prediction parameter encoding unit 111. The prediction parameter input from the prediction parameter encoding unit 111 is, for example, a motion vector or a displacement vector. The predicted image generation unit 101 reads the reference picture block of the block at the position indicated by the motion vector or the displacement vector predicted from the encoding target block. The prediction image generation unit 101 generates a prediction picture block P using one prediction method among a plurality of prediction methods for the read reference picture block. The predicted image generation unit 101 outputs the generated predicted picture block P to the subtraction unit 102. Note that since the predicted image generation unit 101 performs the same operation as the predicted image generation unit 308 already described, details of generation of the predicted picture block P are omitted.

  The subtraction unit 102 subtracts the signal value of the prediction picture block P input from the prediction image generation unit 101 for each pixel from the signal value of the corresponding block of the layer image T input from the outside, and generates a residual signal. Generate. The subtraction unit 102 outputs the generated residual signal to the DCT / quantization unit 103.

  The DCT / quantization unit 103 performs DCT on the residual signal input from the subtraction unit 102 and calculates a DCT coefficient. The DCT / quantization unit 103 quantizes the calculated DCT coefficient to obtain a quantization coefficient. The DCT / quantization unit 103 outputs the obtained quantization coefficient to the entropy encoding unit 104 and the inverse quantization / inverse DCT unit 105.

  The entropy coding unit 104 receives a quantization coefficient from the DCT / quantization unit 103 and receives a coding parameter from the prediction parameter coding unit 111. Input encoding parameters include codes such as a reference picture index refIdxLX, a vector index mvp_LX_idx, a difference vector mvdLX, a prediction mode predMode, and a merge index merge_idx.

  The entropy encoding unit 104 generates and outputs encoded image data VD by entropy encoding the input quantization coefficient and encoding parameter.

  The inverse quantization / inverse DCT unit 105 inversely quantizes the quantization coefficient input from the DCT / quantization unit 103 to obtain a DCT coefficient. The inverse quantization / inverse DCT unit 105 performs inverse DCT on the obtained DCT coefficient to calculate a decoded residual signal. The inverse quantization / inverse DCT unit 105 outputs the calculated decoded residual signal to the addition unit 106.

  The addition unit 106 adds the signal value of the predicted picture block P input from the predicted image generation unit 101 and the signal value of the decoded residual signal input from the inverse quantization / inverse DCT unit 105 for each pixel, and refers to them. Generate a picture block. The adding unit 106 stores the generated reference picture block in the reference picture memory 109.

  The prediction parameter memory 108 stores the prediction parameter generated by the prediction parameter encoding unit 111 at a predetermined position for each picture and block to be encoded.

  The reference picture memory 109 stores the reference picture block generated by the addition unit 106 at a predetermined position for each picture and block to be encoded.

  The encoding parameter determination unit 110 selects one set from among a plurality of sets of encoding parameters. The encoding parameter is a parameter to be encoded that is generated in association with the above-described prediction parameter or the prediction parameter. In addition, among the determined parameters, a parameter common to layers equal to or higher than the slice layer is output to header encoding section 1400. Parameters common to layers above the slice layer will be described later.

  The encoding parameter determination unit 110 calculates a cost value indicating the amount of information and the encoding error for each of the plurality of sets of the encoding parameters. The cost value is, for example, the sum of a code amount and a square error multiplied by a coefficient λ. The code amount is the information amount of the encoded stream Te obtained by entropy encoding the quantization error and the encoding parameter. The square error is the sum between pixels regarding the square value of the residual value of the residual signal calculated by the subtracting unit 102. The coefficient λ is a real number larger than a preset zero. The encoding parameter determination unit 110 selects a set of encoding parameters that minimizes the calculated cost value. As a result, the entropy encoding unit 104 outputs the selected set of encoding parameters to the outside as the encoded stream Te, and does not output the set of unselected encoding parameters.

  The set of coding parameters that are selected depends on the prediction scheme that is selected. The prediction method to be selected is intra prediction, motion prediction, and merge prediction when the picture to be encoded is a base view picture. Motion prediction is prediction between display times among the above-mentioned inter predictions. The merge prediction is a prediction that uses the same reference picture block and prediction parameter as a block that has already been encoded and is within a predetermined range from the encoding target block. When the encoding target picture is a non-base view picture, the prediction methods to be selected are intra prediction, motion prediction, merge prediction, and displacement prediction. The displacement prediction (disparity prediction) is prediction between different layer images (different viewpoint images) in the above-described inter prediction.

  The encoding parameter determination unit 110 outputs the prediction mode predMode corresponding to the selected prediction method to the prediction parameter encoding unit 111.

  When motion prediction is selected as the prediction method, the encoding parameter determination unit 110 also outputs a motion vector mvLX. The motion vector mvLX indicates a vector from the position of the encoding target block to the position of the reference picture block when the predicted picture block P is generated. The information indicating the motion vector mvLX may include information indicating a reference picture (for example, a reference picture index refIdxLX, a picture order number POC), and may represent a prediction parameter.

  When displacement prediction is selected as the prediction method, the encoding parameter determination unit 110 also outputs a displacement vector dvLX. The displacement vector dvLX indicates a vector from the position of the encoding target block to the position of the reference picture block when the predicted picture block P is generated. The information indicating the displacement vector dvLX may include information indicating a reference picture (for example, a reference picture index refIdxLX, a view identifier view_id) and may represent a prediction parameter.

  When merge prediction is selected as the prediction method, the encoding parameter determination unit 110 also outputs a merge index merge_idx.

  The prediction parameter encoding unit 111 derives a prediction parameter to be used when generating a prediction picture based on the parameter input from the encoding parameter determination unit 110, encodes the derived prediction parameter, and sets a set of encoding parameters. Generate. The prediction parameter encoding unit 111 outputs the generated set of encoding parameters to the entropy encoding unit 104.

  The prediction parameter encoding unit 111 stores, in the prediction parameter memory 108, a prediction parameter corresponding to the one selected by the encoding parameter determination unit 110 from the generated set of encoding parameters.

  The prediction parameter encoding unit 111 operates the inter prediction parameter encoding unit 112 when the prediction mode predMode input from the encoding parameter determination unit 110 indicates the inter prediction mode. The prediction parameter encoding unit 111 operates the intra prediction parameter encoding unit 113 when the prediction mode predMode indicates the intra prediction mode.

  The inter prediction parameter encoding unit 112 derives an inter prediction parameter based on the prediction parameter input from the encoding parameter determination unit 110. The inter prediction parameter encoding unit 112 includes the same configuration as the configuration in which the inter prediction parameter decoding unit 303 derives the inter prediction parameter as a configuration for deriving the inter prediction parameter.

  The intra prediction parameter encoding unit 113 determines the intra prediction mode IntraPredMode indicated by the prediction mode predMode input from the encoding parameter determination unit 110 as a set of intra prediction parameters.

  Here, details of parameter determination relating to inter-layer prediction in the encoding parameter determination unit 110 and header information encoding processing in the header encoding unit 1400 based on the determined parameters will be described.

  The encoding parameter determination unit 110 stores information indicating the prediction mode and the reference picture selected in the encoding process of each layer (multi-viewpoint image or the like). If the inter-layer dependency information, that is, the inter-layer dependency relationship regarding sample prediction or temporal motion prediction is determined for all layers, the inter-layer dependency information regarding all layers is output to the header encoding unit 1400. . The inter-layer dependency information includes information indicating whether the layer i refers to the layer j and information indicating whether each reference is based on sample prediction or motion prediction. The header encoding unit 1400 inputs the inter-layer dependency information from the encoding parameter determination unit 110 to the parameter set encoding unit 1401.

  The parameter set encoding unit 1401 encodes the layer dependency flag direct_dependency_flag [i] [j] and the layer dependency type direct_dependency_type [i] [j] based on the input inter-layer dependency information, and has a data structure including them. A certain VPS extension (FIG. 11) is generated and the encoded information is output to the slice segment header encoding unit 1402.

  Also, the coding parameter determination unit 110 outputs slice layer coding parameters including the prediction mode and reference picture information selected in the coding process in units of slice segments to the header coding unit 1400. The header encoding unit 1400 inputs the slice layer encoding parameter from the encoding parameter determining unit 110 to the slice segment header encoding unit 1402.

  The slice segment header encoding unit 1402 is based on the input slice layer encoding parameter and the inter-layer dependency indicating whether or not there is inter-layer prediction based on the encoding parameter related to inter-layer dependency input from the parameter set encoding unit 1401. The flag inter_layer_pred_enabled_flag is encoded. When inter-layer prediction is enabled, the number of inter-layer dependent pictures (num_inter_layer_ref_pics_minus1) and inter-layer dependent identifiers (inter_layer_pred_layer_idc) based on the number of dependent layers (NumDirectRefLayers []) and the actual number of dependent layer pictures (NumActiveRefLayerPics) for the target layer []) Are encoded, and a slice segment header (FIG. 12) including them is generated. Specifically, the value of “NumActiveRefLayerPics-1” is set in num_inter_layer_ref_pics_minus1, and the layer identifier indicating the reference layer j of the target layer i is set in inter_layer_pred_layer_idc [i]. At that time, if there is no actual reference layer picture (NumActiveRefLayerPics == 0), the layer identifier inter_layer_pred_layer_idc [] is not encoded. If there is only one reference layer, the number of inter-layer dependent pictures num_inter_layer_ref_pics_minus1 is also encoded. do not do. In addition, the bit length when encoding these codes is the minimum number of bits that can represent NumDirectRefLayers [] for num_inter_layer_ref_pics_minus1, and the minimum number of bits that can represent NumActiveRefLayerPics for inter_layer_pred_layer_idc []. Are encoded respectively.

  Note that the slice segment header encoding unit 1402 encodes the inter-layer prediction reference picture number (num_inter_layer_ref_pics_minus1) and the inter-layer dependency identifier (inter_layer_pred_layer_idc []) instead of the dependent layer number (NumDirectRefLayers []). Based on the number of sample-dependent layers (NumSamplePredRefLayers), the presence or absence of these codes, the code length, and the value range may be determined and encoded.

  Further, as another configuration of the image encoding device 11, the slice segment header encoding unit 1402 relates the slice segment header to the common part between the base layer and the non-base layer, and the non-base layer as illustrated in FIG. A configuration may be used in which the slice segment header extension is encoded separately. In this configuration, the slice segment header encoding unit 1402 has inter-layer prediction enabled for the target slice segment of the non-base layer (inter_layer_pred_enabled_flag = 1), and the number of inter-layer dependent pictures is equal to the number of dependent layers (num_inter_layer_ref_pics_minus1 = NumDirectRefLayers [nuh_layer_id]? 1) When the identifier indicating the reference layer of the target layer i is equal to the value specified by the VPS extension (inter_layer_pred_layer_idc [i] = 1), the slice segment header extension is not encoded.

  With the configuration as described above, the image encoding apparatus according to the present embodiment has a plurality of parameter group values related to inter-layer prediction for non-base layers when hierarchically encoding image data of a plurality of layers. When the predetermined combination is used, encoding of the extension header information is omitted, so that the amount of parameters required for hierarchical encoding can be reduced and the encoding efficiency can be improved. In addition, regarding slice layer information, by separating the slice segment header into a basic part and an extended part, the influence on the image coding apparatus and software conforming to the standard of only a single layer is suppressed, and image coding corresponding to a single layer is performed. The effect of facilitating multi-layer correspondence (multi-viewpoint image, etc.) based on the device or software can be obtained.

  In addition, a part of the image encoding device 11 and the image decoding device 31 in the above-described embodiment, for example, the header decoding unit 400, the parameter set decoding unit 401, the slice segment header decoding unit 402, the actual dependence layer decoding unit 403, the reference picture Set derivation unit 404, collocated picture decoding unit 405, slice segment header basic decoding unit 4021, slice segment header extended decoding unit 4022, header encoding unit 1400, parameter set encoding unit 1401, slice segment header encoding unit 1402, entropy decoding Unit 301, prediction parameter decoding unit 302, prediction image generation unit 101, DCT / quantization unit 103, entropy encoding unit 104, inverse quantization / inverse DCT unit 105, encoding parameter determination unit 110, prediction parameter encoding unit 111 , D Toropi decoding unit 301, the prediction parameter decoding section 302, predicted image generation unit 308 may be the inverse quantization and inverse DCT unit 311 to be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. Here, the “computer system” is a computer system built in the image encoding device 11 or the image decoding device 31 and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a portable medium such as a flexible disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, or a hard disk built in a computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In this case, it may include a volatile memory inside a computer system serving as a server or a client in such a case that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  Moreover, you may implement | achieve part or all of the image coding apparatus 11 in the embodiment mentioned above and the image decoding apparatus 31 as integrated circuits, such as LSI (Large Scale Integration). Each functional block of the image encoding device 11 and the image decoding device 31 may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where an integrated circuit technology that replaces LSI appears due to progress in semiconductor technology, an integrated circuit based on the technology may be used.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to that described above, and various design changes can be made without departing from the scope of the present invention. Etc. are possible.

  The present invention can be suitably applied to an image decoding apparatus that decodes encoded data obtained by encoding image data and an image encoding apparatus that generates encoded data obtained by encoding image data. Further, the present invention can be suitably applied to the data structure of encoded data generated by an image encoding device and referenced by the image decoding device.

DESCRIPTION OF SYMBOLS 1 ... Image transmission system 11 ... Image coding apparatus 300 ... Image data decoding part 400 ... Header decoding part 401 ... Parameter set decoding part 402 ... Slice segment header decoding part 4021 ... Slice segment header basic decoding part 4022 ... Slice segment header extended decoding Unit 403 ... real dependency layer decoding unit 404 ... reference picture set deriving unit 405 ... collocated picture decoding unit 101 ... predicted image generation unit 102 ... subtraction unit 103 ... DCT / quantization unit 104 ... entropy encoding unit 105 ... inverse quantization / Inverse DCT unit 106 ... addition unit 108 ... prediction parameter memory (frame memory)
109 ... Reference picture memory (frame memory)
DESCRIPTION OF SYMBOLS 110 ... Coding parameter determination part 111 ... Prediction parameter encoding part 112 ... Inter prediction parameter encoding part 113 ... Intra prediction parameter encoding part 1000 ... Image data encoding part 1400 ... Header encoding part 1401 ... Parameter set encoding part 1402 ... Slice segment header encoding unit 21 ... Network 31 ... Image decoding device 301 ... Entropy decoding unit 302 ... Prediction parameter decoding unit 303 ... Inter prediction parameter decoding unit 304 ... Intra prediction parameter decoding unit 306 ... Reference picture memory (frame memory)
307 ... Prediction parameter memory (frame memory)
308 ... Prediction image generation unit 309 ... Inter prediction image generation unit 310 ... Intra prediction image generation unit 311 ... Inverse quantization / inverse DCT unit 312 ... Addition unit 41 ... Image display device

Claims (13)

An image decoding device that decodes hierarchically encoded data,
Dependency type decoding means comprising means for decoding a layer dependency type and determining whether a layer corresponding to the layer dependency type is a layer that exhibits sample dependency;
A dependency layer deriving unit for extracting a sample dependency layer from the dependency layer indicating the sample dependency, a reference picture set decoding unit for deriving a reference picture set using the sample dependency layer,
An image decoding apparatus comprising: an inter prediction image generation unit that generates a prediction image using a reference picture specified by a reference picture list derived using the inter-layer reference picture set. The dependence type decoding means decodes the layer dependence type from the encoded data of the parameter set, derives SamplePredEnableFlag, which is a flag indicating whether the dependence layer is a sample dependence layer, according to the layer dependence type,
The dependency layer deriving unit decodes the inter-layer dependency identifier (inter_layer_pred_layer_idc) of the inter-layer prediction reference picture from the encoded data of the slice segment layer, and derives the sample dependency layer ActiveSamplePredRefLayerId from the dependency layer whose SamplePredEnableFlag is 1 A featured image decoding apparatus. The image decoding apparatus further includes inter-layer prediction limited flag decoding means for decoding an inter-layer prediction limited flag inter_layer_sample_only flag,
The dependency layer derivation unit derives the number NumActiveSamplePredRefLayers of the sample dependency layers according to the layer dependency type,
When the inter-layer prediction only flag inter_layer_sample_only flag indicates only inter-layer prediction, the reference picture set decoding unit does not include pictures in the same layer in the reference picture list,
The inter-layer prediction limited flag decoding means decodes an inter-layer prediction limited flag inter_layer_sample_only flag when the number of motion-dependent layers NumActiveSamplePredRefLayers is 1 or more. An image decoding device that decodes hierarchically encoded data,
Dependency type decoding means comprising means for decoding a layer dependent type and determining whether the layer corresponding to the layer dependent type is a motion dependent limited layer indicating that it is motion dependent and not sample dependent;
An alternative reference picture decoding unit for decoding an alternative reference picture from the motion-dependent limited layer;
An image decoding apparatus comprising: an inter prediction parameter decoding unit that derives a temporal motion vector using the alternative reference picture.   The alternative reference picture decoding unit derives the number of motion-dependent limited layers NumActiveMotionPredRefLayers, which is a dependency layer indicating that it is motion-dependent and not sample-dependent, according to the layer-dependent type, and the number of motion-dependent dependent layers NumActiveMotionPredRefLayers is 1. 4. The image decoding apparatus according to claim 3, wherein a flag alt_collocated_indication_flag indicating whether or not to use an alternative reference picture is decoded from encoded data of a slice segment layer when the above is true. The alternative reference picture decoding unit derives a motion dependent limited layer ActivnMotionPredRefLayerId, which is a dependent layer indicating that it is motion-dependent and not sample-dependent, according to the layer-dependent type,
The image decoding device according to claim 3, wherein an identifier collocated_ref_layer_idx that identifies an alternative reference picture is decoded from the motion-dependent limited layer with reference to MotionPredEnableFlag that is a flag indicating the motion-dependent limited layer. An image encoding apparatus that hierarchically encodes multiple layers of image data,
As a parameter relating to the coding method of the non-base layer picture, it includes header coding means for coding at least actual dependent layer information related to sample prediction between layers and actual dependent layer information related to motion prediction between layers,
Based on the number of sample-dependent layers (NumSamplePredRefLayers) for the target layer, the header encoding means determines the presence / absence of codes of the inter-layer prediction reference pictures (num_inter_layer_ref_pics_minus1) and inter-layer dependency identifiers (inter_layer_pred_layer_idc), code lengths, and value ranges. And encoding the inter-layer prediction reference picture number and the inter-layer dependency identifier, as real dependent layer information related to inter-layer sample prediction, for encoding a non-base layer picture. An image encoding device. An image decoding device that decodes hierarchically encoded data,
Parameter set decoding means for decoding at least the number of dependent layers (NumDirectRefLayers [nuh_layer_id]) related to the layer nuh_layer_id as a parameter related to the encoding method of the non-base layer picture;
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extended decoding means decodes each parameter indicating presence / absence of inter-layer prediction of a non-base layer picture (inter_layer_pred_enable_flag), number of inter-layer prediction reference pictures (num_inter_layer_ref_pics_minus1), and inter-layer dependency identifier (inter_layer_pred_layer_idc [i]) If the encoded data does not include the above syntax, inter_layer_pred_enable_flag is set to 1 indicating that inter-layer prediction is performed, num_inter_layer_ref_pics_minus1 is set to NumDirectRefLayers [nuh_layer_id]-1, and inter_layer_pred_layer_idc [i] is set as a parameter set. An image decoding device, wherein i is set to indicate that the decoded dependency layer is used as it is. An image decoding device that decodes hierarchically encoded data,
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means includes:
Alternative reference picture decoding means for decoding alternative reference picture information (alt_collocated_indication_flag, collocated_ref_layer_idx) indicating a reference picture at the time of temporal motion vector prediction in a non-base layer picture;
A motion vector deriving unit for deriving a temporal motion vector using a picture specified by the decoded alternative reference picture information;
The alternative reference picture decoding means decodes the alternative reference picture information from the encoded data when the encoded data includes a slice segment header extension, and the encoded data does not include the slice segment header extension An image decoding apparatus characterized by decoding alt_collocated_indication_flag = 0 indicating that an alternative reference layer is not used as alternative reference picture information. An image decoding device that decodes hierarchically encoded data,
Parameter set decoding means for decoding at least the number of motion-dependent layers (NumMotionPredRefLayers [i]) as a parameter related to the non-base layer picture encoding method;
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means includes:
Alternative reference picture decoding means for decoding alternative reference picture information (alt_collocated_indication_flag, collocated_ref_layer_idx) indicating a reference picture at the time of temporal motion vector prediction in a non-base layer picture;
A motion vector deriving unit for deriving a temporal motion vector using a picture specified by the decoded alternative reference picture information;
The alternative reference picture decoding means decodes the alternative reference picture information from the encoded data when the encoded data includes a slice segment header extension, and the encoded data does not include the slice segment header extension When the number of motion dependent layers (NumMotionPredRefLayers [i]) derived by the parameter set decoding unit is 0, alt_collocated_indication_flag = 0 indicating that the alternative reference layer is not used is decoded as the alternative reference picture information. An image decoding device characterized by decoding alt_collocated_indication_flag = 1 indicating that an alternative reference layer is used when the number of motion-dependent layers (NumMotionPredRefLayers [i]) is other than 0. An image decoding device that decodes hierarchically encoded data,
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means includes:
Alternative reference picture decoding means for decoding alternative reference picture information (alt_collocated_indication_flag, collocated_ref_layer_idx) indicating a reference picture at the time of temporal motion vector prediction in a non-base layer picture;
Reference picture set deriving means for decoding reference picture information (RPS, RPL);
A motion vector deriving unit for deriving a temporal motion vector using a picture specified by the decoded alternative reference picture information;
The alternative reference picture decoding means decodes the alternative reference picture information from the encoded data when the encoded data includes a slice segment header extension, and the encoded data does not include the slice segment header extension If the reference picture set derived by the reference picture set deriving means includes a picture of the same target layer, alt_collocated_indication_flag = 0 indicating that the alternative reference layer is not used is decoded as alternative reference picture information, An image decoding apparatus characterized by decoding alt_collocated_indication_flag = 1 indicating that an alternative reference layer is used when the reference picture information derived by the alternative reference picture decoding means does not include a picture of the same target layer. An image decoding device that decodes hierarchically encoded data,
Slice segment header basic decoding means for decoding encoding parameters common to the base layer and the non-base layer;
Slice segment header extension decoding means for decoding the encoding parameters for the non-base layer,
The slice segment header extension decoding means, when the slice data header extension is included in the encoded data, as a parameter relating to the decoding method of the non-base layer picture, at least a parameter indicating the reference picture at the time of temporal motion vector prediction (alt_collocated_indication_flag, collocated_ref_layer_idx)
If the encoded data does not include the slice segment header extension, it is based on the encoding parameters included in the slice segment header and indicating the reference picture in temporal motion vector prediction of the base layer (collocated_from_0_flag, collocated_ref_idx) An image decoding apparatus for determining each parameter in a non-base layer picture. An image encoding apparatus that hierarchically encodes multiple layers of image data,
A slice segment header basic encoding means for encoding a common encoding parameter in the base layer and the non-base layer;
Slice segment header extension encoding means for encoding the encoding parameters for the non-base layer,
The slice segment header extension encoding means omits encoding of the slice segment header extension when the values of a plurality of parameter groups related to inter-layer prediction among the encoding parameters related to the non-base layer are a predetermined combination. An image encoding device.