JP2015002495A - Image decoder, and image encoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the memory in coding or decoding.SOLUTION: Prior to decode picture derivation processing of a target picture in an upper layer, a hierarchy moving picture decoder adds to a decode picture buffer an interlayer reference picture that is generated on the basis of a decode picture of a lower layer, sets the reference mark of the interlayer reference picture to reference use, and sets the output mark of the interlayer reference picture to output unnecessary. With reference to the output mark and reference mark of each picture of the decode picture buffer, a picture management section performs bumping for specifying a picture not requiring decoding and output after the target picture, and discarding the picture thus specified from the decode picture buffer.

Description

  The present invention relates to an image decoding apparatus that decodes hierarchically encoded data in which an image is hierarchically encoded, and an image encoding apparatus that generates hierarchically encoded data by hierarchically encoding an image.

  One of information transmitted in the communication system or information recorded in the storage device is an image or a moving image. 2. Description of the Related Art Conventionally, a technique for encoding an image for transmitting and storing these images (hereinafter including moving images) is known.

  As a moving picture coding system, AVC (H.264 / MPEG-4 Advanced Video Coding) and its successor codec HEVC (High-Efficiency Video Coding) are known (Non-patent Document 1).

  In these moving image encoding methods, a predicted image is usually generated based on a local decoded image obtained by encoding / decoding an input image, and obtained by subtracting the predicted image from the input image (original image). Prediction residuals (sometimes referred to as “difference images” or “residual images”) are encoded. In addition, examples of the method for generating a predicted image include inter-screen prediction (inter prediction) and intra-screen prediction (intra prediction).

  In intra prediction, predicted images in a picture are sequentially generated based on a locally decoded image in the same picture.

  In inter prediction, a predicted image is generated by motion compensation between pictures. A decoded picture used for predictive image generation in inter prediction is called a reference picture.

  In HEVC, decoded pictures are recorded in a decoded picture buffer (DPB). Each picture on the DPB is associated with information on whether or not there is a possibility of being used as a reference picture (reference mark) and information on whether or not there is a need to output as an output picture (output mark). By removing unnecessary pictures from the DPB based on the reference mark and output mark information, the number of pictures recorded in the DPB can be controlled so as not to be excessive.

  In recent years, a hierarchical encoding technique for hierarchically encoding an image according to a necessary data rate has been proposed. SHVC (Scalable HEVC) is known as one of representative hierarchical coding methods (Non-patent Document 2).

  SHVC supports spatial scalability, temporal scalability, and SNR scalability. For example, in the case of spatial scalability, an image downsampled from an original image to a desired resolution is encoded as a lower layer. Next, the upper layer performs inter-layer prediction in order to remove redundancy between layers.

  Inter-layer prediction includes inter-layer image prediction and inter-layer motion prediction. In inter-layer image prediction, a predicted image is generated using a decoded image of a lower layer. In inter-layer motion prediction, motion information prediction values are derived using motion information of lower layers.

  In SHVC, any one of inter prediction, intra prediction, and inter-layer image prediction can be used to generate a predicted image.

  The decoded image of the lower layer used in the inter-layer image prediction of SHVC is recorded in the buffer as an inter-layer reference picture after applying a predetermined transformation (for example, scaling). In SHVC, a mechanism for recording and managing a decoded picture in the DPB is applied to the decoded picture of the target layer, as in HEVC.

"Recommendation H.265 (04/13)", ITU-T (released June 7, 2013) JCTVC-M1008_v1 `` SHVC Working Draft 2 '', Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 13th Meeting: Incheon, KR, 18- 26 Apr. 2013 (May 21, 2013)

  However, the SHVC cited as the prior art does not provide a mechanism for correctly managing lower layer decoded pictures used for inter-layer prediction image generation by DPB in the enhancement layer. More specifically, the timing at which the decoded picture of the lower layer referenced in the inter-layer prediction becomes unnecessary is not clear. For this reason, there is a problem that the decoded picture of the lower layer cannot be discarded at an appropriate timing during the encoding or decoding process of the enhancement layer.

  The present invention has been made in view of the above problems, and its purpose is to eliminate the need for decoded pictures in the lower layer on the buffer by managing the decoded pictures in the lower layer in the DPB in the hierarchical coding scheme. It is to provide a mechanism that can be discarded when it becomes. It is also intended to realize an image encoding device and an image decoding device that encode / decode encoded data with less memory.

  In order to solve the above problem, an image decoding apparatus according to the present invention is an image decoding apparatus that decodes higher layer encoded data included in hierarchically encoded data and restores a decoded picture of the upper layer. A picture management unit for managing a decoded picture buffer of an upper layer, wherein the picture management unit is a layer generated based on a decoded picture of a lower layer prior to a decoded picture derivation process of a target picture in the upper layer An inter-reference picture is added to the decoded picture buffer, the reference mark of the inter-layer reference picture is set to be used for reference, and the output mark of the inter-layer reference picture is set to be output unnecessary.

  In the image decoding device, the picture management unit may perform the reference mark of the inter-layer reference picture at a timing after the decoded picture derivation process is completed and before the target picture moves to the next target picture in decoding order. Is preferably set to non-reference use.

  Further, in the image decoding device, the picture management unit refers to an output mark and a reference mark of each picture in the decoded picture buffer, identifies a picture that does not need to be decoded and output after the target picture, It is preferable to execute a bumping process for discarding the identified picture from the decoded picture buffer.

  Further, in the image decoding apparatus, a base decoding unit that generates a base decoded picture that is a decoded picture of a reference layer picture, and a base decoding unit that manages a base decoded picture buffer that is a decoded picture buffer of a reference layer Preferably, the base picture management unit determines a picture to be discarded from the decoded picture buffer by the same bumping process as the bumping process executed by the picture management unit.

  In order to solve the above problem, an image decoding apparatus according to the present invention is an image decoding apparatus that decodes higher layer encoded data included in hierarchically encoded data and restores a decoded picture of the upper layer. A picture management unit for managing a decoded picture buffer common to a plurality of layers, wherein the picture management unit determines whether or not inter prediction is applied between pictures in the same layer in a sublayer to which the target picture belongs, and the determination result is When inter prediction between the pictures in the same layer is not applicable, the reference mark of the target picture is set to non-reference use.

  In the image decoding apparatus, it is preferable that the determination of whether to apply inter prediction between pictures in the same layer is applied only to a picture belonging to the highest sublayer.

  In order to solve the above problem, an image decoding apparatus according to the present invention is an image decoding apparatus that decodes higher layer encoded data included in hierarchically encoded data and restores a decoded picture of the upper layer. A variable length decoding unit that decodes a syntax value from hierarchically encoded data, and the variable length decoding unit can be discarded as a syntax value. The flag is decoded, and the discardable flag takes a value of 1 when the target picture is not referred to in inter prediction of the subsequent picture or inter-layer image prediction in the decoding order, and the target picture is interlaced of the subsequent picture in the decoding order. It takes a value of 0 when referred to in prediction or inter-layer image prediction, and the picture management unit sets the pair according to the value of the discardable flag. Updating a reference mark of the picture, it is preferable.

  In order to solve the above problems, an image encoding device according to the present invention is an image encoding device that generates encoded data of an upper layer from an input image, and a picture management unit that manages a decoded picture buffer of an upper layer The picture management unit adds an inter-layer reference picture generated based on the decoded picture of the lower layer to the decoded picture buffer prior to the decoding picture derivation process of the target picture in the upper layer, and The reference mark of the reference picture is set to be used for reference, and the output mark of the inter-layer reference picture is set to be output unnecessary.

  The image decoding apparatus according to the present invention includes a picture management unit, and the picture management unit adds an inter-layer reference picture to the DPB, and subsequently sets a reference mark of the inter-layer reference picture to “use reference”. In addition, a base reference picture control unit for setting the output mark of the inter-layer reference picture to “output unnecessary” is provided. Therefore, the hierarchical video decoding device 1 can remove the inter-layer reference picture used for picture decoding from the DPB when it becomes unnecessary by using a bumping process that does not distinguish between the inter-layer reference picture and the inter-prediction reference picture. Therefore, input encoded data can be decoded with a small average memory usage.

It is a flowchart which illustrates the picture management process in the decoding picture management part contained in the hierarchy moving image decoding apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the layer structure of the hierarchy coding data which concerns on embodiment of this invention, Comprising: (a) has shown about the hierarchy moving image encoder side, (b) is a hierarchy moving image decoding. The device side is shown. It is a figure for demonstrating the structure of the hierarchy coding data which concerns on embodiment of this invention, Comprising: (a) has shown the sequence layer which prescribes | regulates sequence SEQ, (b) has prescribed | regulated picture PICT. (C) shows the slice layer that defines the slice S, (d) shows the CTU layer that defines the coding tree unit CTU, and (e) shows the code layer 3 illustrates a CU layer that defines a coding unit (CU) included in a coding tree unit CTU. It is a functional block diagram which shows the schematic structure of the said hierarchy moving image decoding apparatus. It is the functional block diagram which illustrated the structure of the decoded picture management part contained in the said hierarchy moving image decoding apparatus. It is the figure which illustrated the state of DPB in the specific timing of a decoding process. It is a part of the syntax table utilized at the time of SPS decoding from the variable length decoding part of the said hierarchy moving image decoding apparatus, Comprising: It is a figure which shows the part which concerns on a reference picture set and a reference picture list. It is a figure which shows the syntax table utilized at the time of decoding of short-term reference picture set information from the variable-length decoding part of the said hierarchy moving image decoding apparatus. It is a figure which is a part of syntax tables used at the time of slice header decoding from the variable length decoding part of the said hierarchy moving image decoding apparatus, Comprising: It is a figure which shows the part which concerns on a reference picture set. It is a part of syntax table referred at the time of slice decoding, and is a figure which shows the part corresponded to ILRP information. It is a figure which is a part of syntax tables used at the time of slice header decoding from the variable length decoding part of the said hierarchy moving image decoding apparatus, Comprising: It is a figure which shows the part which concerns on a reference picture list. It is a figure which shows the syntax table utilized at the time of the decoding of reference picture list correction information from the variable-length decoding part of the said hierarchy moving image decoding apparatus. It is a figure which shows the specific example of the picture management process in two pictures (a picture M and a picture N in order) with which the decoding order continues on the object layer. It is a functional block diagram which illustrates the structure of the base decoding part contained in the said hierarchy moving image decoding apparatus. It is a flowchart showing a part of process in an example of the picture management process at the time of common DPM model utilization. It is a flowchart showing a part of process in another example of the picture management process at the time of common DPM model use. It is a functional block diagram which shows schematic structure of the hierarchy moving image encoder which concerns on one Embodiment of this invention. It is the figure which showed the structure of the transmitter which mounts the said hierarchy moving image encoder, and the receiver which mounts the said hierarchy moving image decoder. (A) shows a transmission device equipped with a hierarchical video encoding device, and (b) shows a reception device equipped with a hierarchical video decoding device. It is the figure which showed the structure of the recording device carrying the said hierarchy moving image encoder, and the reproducing | regenerating apparatus carrying the said hierarchy moving image decoding apparatus. (A) shows a recording device equipped with a hierarchical video encoding device, and (b) shows a playback device equipped with a hierarchical video decoding device.

  The hierarchical moving picture decoding apparatus 1 and the hierarchical moving picture encoding apparatus 2 according to an embodiment of the present invention will be described as follows based on FIGS.

〔Overview〕
A hierarchical video decoding device (image decoding device) 1 according to the present embodiment decodes encoded data that has been hierarchically encoded by a hierarchical video encoding device (image encoding device) 2. Hierarchical coding is a coding scheme that hierarchically encodes moving images from low quality to high quality. Hierarchical coding is standardized in SVC and SHVC, for example. Note that the quality of a moving image here widely means an element that affects the appearance of a subjective and objective moving image. The quality of the moving image includes, for example, “resolution”, “frame rate”, “image quality”, and “pixel representation accuracy”. Therefore, hereinafter, if the quality of the moving image is different, it means that, for example, “resolution” is different, but it is not limited thereto. For example, in the case of moving images quantized in different quantization steps (that is, moving images encoded with different encoding noises), it can be said that the quality of moving images is different from each other.

  In addition, the hierarchical coding technology is (1) spatial scalability, (2) temporal scalability, (3) SNR (Signal to Noise Ratio) scalability, and (4) view scalability from the viewpoint of the type of information to be layered. May be classified. Spatial scalability is a technique for hierarchizing resolution and image size. Time scalability is a technique for layering at a frame rate (number of frames per unit time). SNR scalability is a technique for layering in coding noise. Also, view scalability is a technique for hierarchizing at the viewpoint position associated with each image.

  Prior to detailed description of the hierarchical video encoding device 2 and the hierarchical video decoding device 1 according to the present embodiment, first, (1) the hierarchical video encoding device 2 generates and the hierarchical video decoding device 1 performs decoding. The layer structure of the hierarchically encoded data to be performed will be described, and then (2) a specific example of the data structure that can be adopted in each layer will be described.

[Layer structure of hierarchically encoded data]
Here, encoding and decoding of hierarchically encoded data will be described with reference to FIG. FIG. 2 is a diagram schematically illustrating a case where a moving image is hierarchically encoded / decoded by three layers of a lower layer L3, a middle layer L2, and an upper layer L1. That is, in the example shown in FIGS. 2A and 2B, of the three layers, the upper layer L1 is the highest layer and the lower layer L3 is the lowest layer.

  In the following, a decoded image corresponding to a specific quality that can be decoded from hierarchically encoded data is referred to as a decoded image of a specific hierarchy (or a decoded image corresponding to a specific hierarchy) (for example, an upper layer L1). Decoded image POUT # A).

  FIG. 2A shows a hierarchical video encoding device 2 # A-2 # C that generates encoded data DATA # A-DATA # C by hierarchically encoding input images PIN # A-PIN # C, respectively. Is shown. FIG. 2B illustrates a hierarchical video decoding device 1 # A that generates decoded images POUT # A to POUT # C by decoding the hierarchically encoded data DATA # A to DATA # C, respectively. 1 # C is shown.

  First, the encoding device side will be described with reference to FIG. The input images PIN # A, PIN # B, and PIN # C that are input on the encoding device side have the same original image but different image quality (resolution, frame rate, image quality, and the like). The image quality decreases in the order of the input images PIN # A, PIN # B, and PIN # C.

  The hierarchical video encoding apparatus 2 # C of the lower hierarchy L3 encodes the input image PIN # C of the lower hierarchy L3 to generate encoded data DATA # C of the lower hierarchy L3. Basic information necessary for decoding the decoded image POUT # C of the lower layer L3 is included (indicated by “C” in FIG. 2). Since the lower layer L3 is the lowest layer, the encoded data DATA # C of the lower layer L3 is also referred to as basic encoded data.

  Further, the hierarchical video encoding apparatus 2 # B of the middle hierarchy L2 encodes the input image PIN # B of the middle hierarchy L2 with reference to the encoded data DATA # C of the lower hierarchy, and performs the middle hierarchy L2 Encoded data DATA # B is generated. In addition to the basic information “C” included in the encoded data DATA # C, additional data necessary for decoding the decoded image POUT # B of the intermediate hierarchy is added to the encoded data DATA # B of the intermediate hierarchy L2. Information (indicated by “B” in FIG. 2) is included.

  Further, the hierarchical video encoding apparatus 2 # A of the upper hierarchy L1 encodes the input image PIN # A of the upper hierarchy L1 with reference to the encoded data DATA # B of the intermediate hierarchy L2 to Encoded data DATA # A is generated. The encoded data DATA # A of the upper layer L1 is used to decode the basic information “C” necessary for decoding the decoded image POUT # C of the lower layer L3 and the decoded image POUT # B of the middle layer L2. In addition to the necessary additional information “B”, additional information (indicated by “A” in FIG. 2) necessary for decoding the decoded image POUT # A of the upper layer is included.

  As described above, the encoded data DATA # A of the upper layer L1 includes information related to decoded images having a plurality of different qualities.

  Next, the decoding device side will be described with reference to FIG. On the decoding device side, the decoding devices 1 # A, 1 # B, and 1 # C corresponding to the layers of the upper layer L1, the middle layer L2, and the lower layer L3 are encoded data DATA # A and DATA # B, respectively. , And DATA # C are decoded to output decoded images POUT # A, POUT # B, and POUT # C.

  It is also possible to reproduce a moving image having a specific quality by extracting a part of the information of the upper layer encoded data and decoding the extracted information in a lower specific decoding device.

  For example, the hierarchy decoding apparatus 1 # B of the middle hierarchy L2 receives information necessary for decoding the decoded image POUT # B from the hierarchy encoded data DATA # A of the upper hierarchy L1 (that is, the hierarchy encoded data DATA # A decoded image POUT # B may be decoded by extracting “B” and “C”) included in A. In other words, on the decoding device side, the decoded images POUT # A, POUT # B, and POUT # C can be decoded based on information included in the hierarchically encoded data DATA # A of the upper hierarchy L1.

  The hierarchical encoded data is not limited to the above three-layer hierarchical encoded data, and the hierarchical encoded data may be hierarchically encoded with two layers or may be hierarchically encoded with a number of layers larger than three. Good.

  Also, a part or all of the encoded data related to the decoded image of a specific hierarchy is encoded independently of the other hierarchy, and it is not necessary to refer to information of the other hierarchy when decoding the specific hierarchy. Hierarchically encoded data may be configured as described above. For example, in the example described above with reference to FIGS. 2A and 2B, it has been described that “C” and “B” are referred to for decoding the decoded image POUT # B, but the present invention is not limited thereto. It is also possible to configure the hierarchically encoded data so that the decoded image POUT # B can be decoded using only “B”. For example, it is possible to configure a hierarchical video decoding apparatus that receives the hierarchically encoded data composed only of “B” and the decoded image POUT # C for decoding the decoded image POUT # B.

  When SNR scalability is realized, the same original image is used as the input images PIN # A, PIN # B, and PIN # C, and the decoded images POUT # A, POUT # B, and POUT # C have different image quality. Hierarchically encoded data can also be generated so that In that case, the lower layer hierarchical video encoding device generates hierarchical encoded data by quantizing the prediction residual using a larger quantization width than the upper layer hierarchical video encoding device. To do.

  In this document, the following terms are defined for convenience of explanation. The following terms are used to indicate the following technical matters unless otherwise specified.

  Upper layer: A layer positioned higher than a certain layer is referred to as an upper layer. For example, in FIG. 2, the upper layers of the lower layer L3 are the middle layer L2 and the upper layer L1. The decoded image of the upper layer means a decoded image with higher quality (for example, high resolution, high frame rate, high image quality, etc.).

  Lower layer: A layer located lower than a certain layer is referred to as a lower layer. For example, in FIG. 2, the lower layers of the upper layer L1 are the middle layer L2 and the lower layer L3. Further, the decoded image of the lower layer refers to a decoded image with lower quality.

  Target layer: A layer that is the target of decoding or encoding.

  Reference layer: A specific lower layer referred to for decoding a decoded image corresponding to a target layer is referred to as a reference layer.

  In the example shown in FIGS. 2A and 2B, the reference layers of the upper hierarchy L1 are the middle hierarchy L2 and the lower hierarchy L3. However, the present invention is not limited to this, and the hierarchically encoded data can be configured so that it is not necessary to refer to all of the lower layers in decoding of the specific layer. For example, the hierarchical encoded data can be configured such that the reference layer of the upper hierarchy L1 is either the middle hierarchy L2 or the lower hierarchy L3.

  Base layer: A layer located at the lowest layer is referred to as a base layer. The decoded image of the base layer is the lowest quality decoded image that can be decoded from the encoded data, and is referred to as a basic decoded image. In other words, the basic decoded image is a decoded image corresponding to the lowest layer. The partially encoded data of the hierarchically encoded data necessary for decoding the basic decoded image is referred to as basic encoded data. For example, the basic information “C” included in the hierarchically encoded data DATA # A of the upper hierarchy L1 is the basic encoded data.

  Enhancement layer: The upper layer of the base layer is referred to as an enhancement layer.

  Layer identifier: The layer identifier is for identifying a hierarchy, and corresponds to the hierarchy one-to-one. The hierarchically encoded data includes a hierarchical identifier used for selecting partial encoded data necessary for decoding a decoded image of a specific hierarchy. A subset of hierarchically encoded data associated with a layer identifier corresponding to a specific layer is also referred to as a layer representation.

  In general, for decoding a decoded image of a specific hierarchy, a layer expression of the hierarchy and / or a layer expression corresponding to a lower layer of the hierarchy is used. That is, in decoding the decoded image of the target layer, layer representation of the target layer and / or layer representation of one or more layers included in a lower layer of the target layer are used.

  Inter-layer prediction: Inter-layer prediction is based on a syntax element value included in a layer expression of a layer (reference layer) different from the layer expression of the target layer, a value derived from the syntax element value, and a decoded image. It is to predict the syntax element value of the target layer, the encoding parameter used for decoding of the target layer, and the like. Inter-layer prediction that predicts information related to motion prediction from reference layer information is sometimes referred to as motion information prediction. In addition, inter-layer prediction predicted from a lower layer decoded image may be referred to as inter-layer image prediction (or inter-layer texture prediction). Note that the hierarchy used for inter-layer prediction is, for example, a lower layer of the target layer. In addition, performing prediction within a target layer without using a reference layer may be referred to as intra-layer prediction.

  Note that the above terms are for convenience of explanation until they are tired, and the above technical matters may be expressed by other terms.

[Data structure of hierarchically encoded data]
Hereinafter, a case where HEVC and its extension method are used as an encoding method for generating encoded data of each layer will be exemplified. However, the present invention is not limited to this, and the encoded data of each layer may be generated by an encoding method such as MPEG-2 or H.264 / AVC.

  Further, the lower layer and the upper layer may be encoded by different encoding methods. Also, the encoded data of each layer may be supplied to the hierarchical video decoding device 1 via different transmission paths, or may be supplied to the hierarchical video decoding device 1 via the same transmission path. .

  For example, when transmitting ultra-high-definition video (moving image, 4K video data) with a base layer and one extended layer in a scalable encoding, the base layer downscales 4K video data, and interlaced video data. It may be encoded by MPEG-2 or H.264 / AVC and transmitted over a television broadcast network, and the enhancement layer may encode 4K video (progressive) with HEVC and transmit over the Internet.

(Basic layer)
FIG. 3 is a diagram illustrating a data structure of encoded data (hierarchically encoded data DATA # C in the example of FIG. 2) that can be employed in the base layer. Hierarchically encoded data DATA # C illustratively includes a sequence and a plurality of pictures constituting the sequence.

  FIG. 3 shows a hierarchical structure of data in the hierarchically encoded data DATA # C. 3A to 3E respectively show a sequence layer that defines the sequence SEQ, a picture layer that defines the picture PICT, a slice layer that defines the slice S, and a coding tree unit (CTU). It is a figure which shows the CU layer which prescribes | regulates the coding unit (Coding Unit; CU) contained in the CTU layer and coding tree unit CTU to prescribe | regulate.

(Sequence layer)
In the sequence layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a sequence SEQ to be processed (hereinafter also referred to as a target sequence) is defined. As shown in FIG. 3A, the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), and pictures PICT _{1 to} PICT. _{It includes NP} (NP is the total number of pictures included in the sequence SEQ) and supplemental enhancement information (SEI).

  In the video parameter set VPS, the number of layers included in the encoded data and the dependency relationship between the layers are defined.

  In the sequence parameter set SPS, a set of encoding parameters referred to by the hierarchical video decoding device 1 for decoding the target sequence is defined. A plurality of SPSs may exist in the encoded data. In that case, an SPS used for decoding is selected from a plurality of candidates for each target sequence. An SPS used for decoding a specific sequence is also called an active SPS. In the following, unless otherwise specified, it means an active SPS for the target sequence.

  In the picture parameter set PPS, a set of encoding parameters referred to by the hierarchical video decoding device 1 for decoding each picture in the target sequence is defined. A plurality of PPS may exist in the encoded data. In that case, one of a plurality of PPSs is selected from each picture in the target sequence. A PPS used for decoding a specific picture is also called an active PPS. In the following, unless otherwise specified, PPS means active PPS for the current picture.

(Picture layer)
In the picture layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a picture PICT to be processed (hereinafter also referred to as a target picture) is defined. As shown in FIG. 3B, the picture PICT includes slice headers SH _{1 to} SH _NS and slices S _{1 to} S _NS (NS is the total number of slices included in the picture PICT).

Hereinafter, when it is not necessary to distinguish each of the slice headers SH _{1 to} SH _NS and the slices S _{1 to} S _NS , the reference numerals may be omitted. The same applies to other data with subscripts included in hierarchically encoded data DATA # C described below.

The slice header SH _k includes a coding parameter group referred to by the hierarchical video decoding device 1 in order to determine a decoding method for the corresponding slice S _k . For example, an SPS identifier (seq_parameter_set_id) that specifies SPS and a PPS identifier (pic_parameter_set_id) that specifies PPS are included. The slice type designation information (slice_type) for designating the slice type is an example of an encoding parameter included in the slice header SH.

  As slice types that can be specified by the slice type specification information, (1) I slice that uses only intra prediction at the time of encoding, (2) P slice that uses unidirectional prediction or intra prediction at the time of encoding, (3) B-slice using unidirectional prediction, bidirectional prediction, or intra prediction at the time of encoding may be used.

(Slice layer)
In the slice layer, a set of data that is referred to by the hierarchical video decoding device 1 in order to decode a slice S (also referred to as a target slice) to be processed is defined. As shown in FIG. 3C, the slice S includes coding tree units CTU _{1 to} CTU _NC (NC is the total number of CTUs included in the slice S).

(CTU layer)
In the CTU layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a coding tree unit CTU to be processed (hereinafter also referred to as a target CTU) is defined. Note that the coding tree unit may be referred to as a coding tree block (CTB) or a maximum coding unit (LCU).

The coding tree unit CTU includes a CTU header CTUH and coding unit information CU _{1 to} CU _NL (NL is the total number of coding unit information included in the CTU). Here, first, the relationship between the coding tree unit CTU and the coding unit information CU will be described as follows.

  The coding tree unit CTU is divided into units for specifying a block size for each process of intra prediction or inter prediction and transformation.

  The unit of the coding tree unit CTU is divided by recursive quadtree partitioning. The tree structure obtained by this recursive quadtree partitioning is hereinafter referred to as a coding tree.

  Hereinafter, a unit corresponding to a leaf, which is a node at the end of the coding tree, is referred to as a coding node. In addition, since the encoding node is a basic unit of the encoding process, hereinafter, the encoding node is also referred to as an encoding unit (CU).

That is, coding unit information (hereinafter referred to as CU information) CU _{1 to} CU _NL corresponds to each coding node (coding unit) obtained by recursively dividing the coding tree unit CTU into quadtrees. Information.

  The root of the coding tree is associated with the coding tree unit CTU. In other words, the coding tree unit CTU is associated with the highest node of the tree structure of the quadtree partition that recursively includes a plurality of coding nodes.

  Note that the size of each coding node is half of the size of the coding node that is the parent node of the coding node (that is, the node one layer higher than the coding node).

  Also, the size of the coding tree unit CTU and the size that each coding unit can take are the size designation information of the minimum coding node and the maximum coding node and the minimum coding node included in the sequence parameter set SPS. Depends on hierarchy depth difference. For example, when the size of the minimum coding node is 8 × 8 pixels and the difference in the layer depth between the maximum coding node and the minimum coding node is 3, the size of the coding tree unit CTU is 64 × 64 pixels. Thus, the size of the encoding node can take any of four sizes, that is, 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, and 8 × 8 pixels.

(CTU header)
The CTU header CTUH includes an encoding parameter referred to by the hierarchical video decoding device 1 in order to determine a decoding method of the target CTU. Specifically, as shown in FIG. 3 (d), CTU division information SP_CTU for designating a division pattern of the target CTU into each CU, and a quantization parameter difference Δqp (for designating the quantization step size) qp_delta).

  The CTU division information SP_CTU is information representing a coding tree for dividing the CTU, and specifically, information specifying the shape and size of each CU included in the target CTU and the position in the target CTU. It is.

  Note that the CTU partition information SP_CTU may not explicitly include the shape or size of the CU. For example, the CTU division information SP_CTU may be a set of flags indicating whether or not the entire target CTU or a partial region of the CTU is to be divided into four. In that case, the shape and size of each CU can be specified by using the shape and size of the CTU together.

  The quantization parameter difference Δqp is a difference qp−qp ′ between the quantization parameter qp in the target CTU and the quantization parameter qp ′ in the CTU encoded immediately before the target CTU.

(CU layer)
In the CU layer, a set of data referred to by the hierarchical video decoding device 1 for decoding a CU to be processed (hereinafter also referred to as a target CU) is defined.

  Here, before describing specific contents of data included in the CU information CU, a tree structure of data included in the CU will be described. The encoding node is a node at the root of a prediction tree (PT) and a transform tree (TT). The prediction tree and the conversion tree are described as follows.

  In the prediction tree, the encoding node is divided into one or a plurality of prediction blocks, and the position and size of each prediction block are defined. In other words, the prediction block is one or a plurality of non-overlapping areas constituting the encoding node. The prediction tree includes one or a plurality of prediction blocks obtained by the above division.

  Prediction processing is performed for each prediction block. Hereinafter, a prediction block that is a unit of prediction is also referred to as a prediction unit (PU).

  Broadly speaking, there are two types of partitioning in the prediction tree (hereinafter abbreviated as PU partitioning): intra prediction and inter prediction.

  In the case of intra prediction, there are 2N × 2N (the same size as the encoding node) and N × N division methods.

  In the case of inter prediction, 2N × 2N (the same size as the encoding node), 2N × N, 2N × nU, 2N × nD, N × 2N, nL × 2N, nR × 2N, and the like are used as division methods. is there.

  In the transform tree, the encoding node is divided into one or a plurality of transform blocks, and the position and size of each transform block are defined. In other words, the transform block is one or a plurality of non-overlapping areas constituting the encoding node. The conversion tree includes one or a plurality of conversion blocks obtained by the above division.

  There are two types of division in the transformation tree: one in which an area having the same size as that of a coding node is assigned as a transformation block, and the other in division by recursive quadtree division as in the above-described division of a tree block.

  The conversion process is performed for each conversion block. Hereinafter, a transform block that is a unit of transform is also referred to as a transform unit (TU).

(Data structure of CU information)
Next, specific contents of data included in the CU information CU will be described with reference to FIG. As shown in FIG. 3E, the CU information CU specifically includes a skip flag SKIP, prediction tree information (hereinafter abbreviated as PT information) PTI, and conversion tree information (hereinafter abbreviated as TT information). Include TTI).

  The skip flag SKIP is a flag indicating whether or not the skip mode is applied to the target PU. When the value of the skip flag SKIP is 1, that is, when the skip mode is applied to the target CU, A part of the PT information PTI and the TT information TTI in the CU information CU are omitted. Note that the skip flag SKIP is omitted for the I slice.

[PT information]
The PT information PTI is information related to a prediction tree (hereinafter abbreviated as PT) included in the CU. In other words, the PT information PTI is a set of information related to each of one or a plurality of PUs included in the PT, and is referred to when a predicted image is generated by the hierarchical video decoding device 1. As shown in FIG. 3E, the PT information PTI includes prediction type information PType and prediction information PInfo.

  The prediction type information PType is information that specifies a predicted image generation method for the target PU. In the base layer, it is information that specifies whether intra prediction or inter prediction is used.

  The prediction information PInfo is prediction information used in the prediction method specified by the prediction type information PType. In the base layer, intra prediction information PP_Intra is included in the case of intra prediction. In the case of inter prediction, inter prediction information PP_Inter is included.

  The inter prediction information PP_Inter includes prediction information that is referred to when the hierarchical video decoding device 1 generates an inter prediction image by inter prediction. More specifically, the inter prediction information PP_Inter includes inter PU division information that specifies a division pattern of the target CU into each inter PU, and inter prediction parameters (motion compensation parameters) for each inter PU. Examples of inter prediction parameters include a merge flag (merge_flag), a merge index (merge_idx), an estimated motion vector index (mvp_idx), a reference picture index (ref_idx), an inter prediction flag (inter_pred_flag), and a motion vector residual (mvd) including.

  The intra prediction information PP_Intra includes an encoding parameter that is referred to when the hierarchical video decoding device 1 generates an intra predicted image by intra prediction. More specifically, the intra prediction information PP_Intra includes intra PU division information that specifies a division pattern of the target CU into each intra PU, and intra prediction parameters for each intra PU. The intra prediction parameter is a parameter for designating an intra prediction method (prediction mode) for each intra PU.

  Here, the intra prediction parameter is a parameter for restoring intra prediction (prediction mode) for each intra PU. The parameters for restoring the prediction mode include mpm_flag which is a flag related to MPM (Most Probable Mode, the same applies hereinafter), mpm_idx which is an index for selecting the MPM, and an index for specifying a prediction mode other than MPM. Rem_idx is included. Here, MPM is an estimated prediction mode that is highly likely to be selected in the target partition. For example, the MPM may include an estimated prediction mode estimated based on prediction modes assigned to partitions around the target partition, and a DC mode or Planar mode that generally has a high probability of occurrence.

  In the following description, when simply described as “prediction mode”, it means the luminance prediction mode unless otherwise specified. The color difference prediction mode is described as “color difference prediction mode” and is distinguished from the luminance prediction mode. The parameter for restoring the prediction mode includes chroma_mode that is a parameter for designating the color difference prediction mode.

[TT information]
The TT information TTI is information regarding a conversion tree (hereinafter abbreviated as TT) included in the CU. In other words, the TT information TTI is a set of information regarding each of one or a plurality of transform blocks included in the TT, and is referred to when the hierarchical video decoding device 1 decodes residual data.

As shown in FIG. 3 (e), the TT information TTI includes TT division information SP_TT that specifies a division pattern of the target CU into each transform block, and quantized prediction residuals QD _{1 to} QD _NT (NT is the target The total number of blocks included in the CU).

  Specifically, the TT division information SP_TT is information for determining the shape of each transform block included in the target CU and the position in the target CU. For example, the TT division information SP_TT can be realized from information (split_transform_unit_flag) indicating whether or not the target node is divided and information (trafoDepth) indicating the division depth.

  For example, when the CU size is 64 × 64, each transform block obtained by the division can take a size from 32 × 32 pixels to 4 × 4 pixels.

  Each quantization prediction residual QD is encoded data generated by the hierarchical video encoding device 2 performing the following processes 1 to 3 on a target block that is a conversion block to be processed.

Process 1: The prediction residual obtained by subtracting the prediction image from the encoding target image is subjected to frequency conversion (for example, DCT conversion (Discrete Cosine Transform) and DST conversion (Discrete Sine Transform));
Process 2: Quantize the transform coefficient obtained in Process 1;
Process 3: Variable length coding is performed on the transform coefficient quantized in Process 2;
Note that the quantization parameter qp described above represents the size of the quantization step QP used when the hierarchical moving image encoding apparatus 2 quantizes the transform coefficient (QP = 2 ^{qp / 6} ).

(PU partition information)
The PU partition type specified by the PU partition information includes the following eight patterns in total, assuming that the size of the target CU is 2N × 2N pixels. That is, 4 symmetric splittings of 2N × 2N pixels, 2N × N pixels, N × 2N pixels, and N × N pixels, and 2N × nU pixels, 2N × nD pixels, nL × 2N pixels, And four asymmetric splittings of nR × 2N pixels. N = 2 ^m (m is an arbitrary integer of 1 or more). Hereinafter, a prediction unit obtained by dividing the target CU is referred to as a prediction block or a partition.

(Enhancement layer)
For encoded data included in the layer representation of the enhancement layer (hereinafter, enhancement layer encoded data), for example, a data structure substantially similar to the data structure shown in FIG. 3 can be adopted. However, in the enhancement layer encoded data, additional information can be added or parameters can be omitted as follows.

  In the slice layer, spatial scalability, temporal scalability, SNR scalability, and view scalability hierarchy identification information (dependency_id, temporal_id, quality_id, and view_id, respectively) may be encoded.

  Further, the prediction type information PType included in the CU information CU is information that specifies whether the prediction image generation method for the target CU is intra prediction, inter prediction, or inter-layer image prediction. The prediction type information PType includes a flag (inter-layer image prediction flag) that specifies whether or not to apply the inter-layer image prediction mode. Note that the inter-layer image prediction flag may be referred to as texture_rl_flag, inter_layer_pred_flag, or base_mode_flag.

  In the enhancement layer, it may be specified whether the CU type of the target CU is an intra CU, an inter-layer CU, an inter CU, or a skip CU.

  An intra CU can be defined similarly to an intra CU in the base layer. In the intra CU, the inter-layer image prediction flag is set to “0”, and the prediction mode flag is set to “0”.

  An inter-layer CU can be defined as a CU that uses a decoded image of a picture of a reference layer to generate a predicted image. In the inter-layer CU, the inter-layer image prediction flag is set to “1” and the prediction mode flag is set to “0”.

  The skip CU can be defined in the same manner as in the HEVC scheme described above. For example, in the skip CU, “1” is set in the skip flag.

  The inter CU may be defined as a CU that applies non-skip and motion compensation (MC). In the inter CU, for example, “0” is set in the skip flag and “1” is set in the prediction mode flag.

  Further, as described above, the encoded data of the enhancement layer may be generated by an encoding method different from the encoding method of the lower layer. That is, the encoding / decoding process of the enhancement layer does not depend on the type of the lower layer codec.

  The lower layer may be encoded by, for example, MPEG-2 or H.264 / AVC format.

  In the enhancement layer encoded data, the VPS may be extended to include a parameter representing a reference structure between layers.

  Also, in the enhancement layer encoded data, SPS, PPS, and slice header are extended, and information related to a decoded image of a reference layer used for inter-layer image prediction (for example, an inter-layer reference picture set, an inter-layer reference picture list described later) , Syntax for deriving base control information or the like directly or indirectly).

  Note that the parameters described above may be encoded independently, or a plurality of parameters may be encoded in combination. When a plurality of parameters are encoded in combination, an index is assigned to the combination of parameter values, and the assigned index is encoded. Also, if the parameter can be derived from another parameter or decoded information, the encoding of the parameter can be omitted.

[Hierarchical video decoding device]
Below, the structure of the hierarchy moving image decoding apparatus 1 which concerns on this embodiment is demonstrated with reference to FIGS.

(Configuration of Hierarchical Video Decoding Device)
The schematic configuration of the hierarchical video decoding device 1 will be described with reference to FIG. FIG. 4 is a functional block diagram showing a schematic configuration of the hierarchical video decoding device 1. The hierarchical video decoding device 1 decodes the hierarchical encoded data DATA supplied from the hierarchical video encoding device 2 to generate a target layer decoded image POUT # T. In the following description, it is assumed that the target layer is an enhancement layer. Therefore, the target layer is also an upper layer with respect to the reference layer. Conversely, the reference layer is also a lower layer with respect to the target layer.

  As shown in FIG. 4, the hierarchical video decoding device 1 includes a NAL demultiplexing unit 11, a variable length decoding unit 12, a picture decoding unit 13, a decoded picture management unit 14, and a base decoding unit 15.

  The NAL demultiplexing unit 11 demultiplexes hierarchically encoded data DATA transmitted in units of NAL units in NAL (Network Abstraction Layer).

  The NAL is a layer provided to abstract communication between a VCL (Video Coding Layer) and a lower system that transmits and stores encoded data.

  VCL is a layer that performs moving image encoding processing, and encoding is performed in VCL. On the other hand, the lower system here corresponds to the H.264 / AVC and HEVC file formats and the MPEG-2 system.

  In NAL, a bit stream generated by VCL is divided into units called NAL units and transmitted to a destination lower system. The NAL unit includes encoded data encoded by the VCL and a header for appropriately delivering the encoded data to the destination lower system. Also, the encoded data in each layer is stored in the NAL unit, is NAL multiplexed, and is transmitted to the hierarchical moving image decoding apparatus 1.

  The NAL demultiplexing unit 11 demultiplexes the hierarchical encoded data DATA, and extracts the target layer encoded data DATA # T and the reference layer encoded data DATA # R. Further, the NAL demultiplexing unit 11 supplies the target layer encoded data DATA # T to the variable length decoding unit 12 and also supplies the reference layer encoded data DATA # R to the base decoding unit 15.

  The variable length decoding unit 12 decodes and outputs various syntax values from the binary included in the target layer encoded data DATA # T.

  The syntax value decoded by the variable length decoding unit 12 includes a syntax value related to transform coefficients and prediction information.

  The syntax value decoded by the variable length decoding unit 12 includes a parameter set used for decoding of the target layer, that is, a syntax value included in VPS, SPS, and PPS. Further, the syntax value included in the slice header is also included.

  The variable length decoding unit 12 supplies a slice header, prediction information, and a syntax value related to the transform coefficient to the picture decoding unit 13. Further, the variable length decoding unit 12 supplies the parameter set and the syntax value related to the slice header to the decoded picture management unit 14.

  The picture decoding unit 13 decodes the decoded picture of the target picture based on the slice header related to the target picture, the prediction information, and the syntax value related to the transform coefficient. The decoded picture is output to the decoded picture management unit 14.

  Note that, in the prediction picture generation by inter prediction in the picture decoding unit 13, a decoded picture recorded in the DPB in the decoded picture management unit 14 and included in the input reference picture list is used as a reference picture. Is done. In addition, in the prediction image generation by the inter-layer image prediction in the picture decoding unit 13, an inter-layer reference picture recorded in the DPB in the decoded picture management unit 14 and included in the input reference picture list is used. Used as a reference picture.

  The decoded picture management unit 14 records the input decoded picture and the base decoded picture in the internal DPB, and performs reference picture list generation and output picture determination based on the input syntax value. The decoded picture management unit 14 is an important component in this embodiment, and will be described in detail later.

  The base decoding unit 15 decodes a base decoded picture from the reference layer encoded data DATA # R. The base decoded picture is a decoded picture of the reference layer used when decoding the decoded picture of the target layer. Note that the base decoded picture is not necessarily a one-layer decoded picture. That is, when the target layer refers to a plurality of reference layers, the base decoding unit 15 decodes base decoded pictures corresponding to the plurality of layers to be referenced. The base decoding unit 15 supplies the decoded base decoded picture to the decoded picture management unit 14. Details of the base decoding unit 15 will be described later.

  Below, the detail of the decoded picture management part 14 and the base decoding part 15 is demonstrated.

(Decoded picture management unit)
The detailed configuration of the decoded picture management unit 14 will be described with reference to FIG. FIG. 5 is a functional block diagram illustrating the configuration of the decoded picture management unit 14.

  As shown in FIG. 5, the decoded picture management unit 14 includes a DPB 141, an RPS deriving unit 142, a reference picture control unit 143, a base reference picture control unit 144, an RPL deriving unit 145, and an output control unit 146.

<DPB141>
The DPB 141 is also called a decoded picture buffer, and records the decoded picture of each picture of the target layer decoded by the picture decoding unit 13. In the DPB, a decoded picture corresponding to each picture of the target layer is recorded in association with an output order (POC: Picture Order Count). In addition, a reference mark and an output mark can be set for each DPB picture.

  The reference mark is information indicating whether or not the picture on the DPB can be used for prediction image generation processing (for example, inter prediction or inter-layer image prediction) in decoding processing after the target picture. Specifically, the reference marks are "short-term reference use" ("used for short-term reference"), "long-term reference use" ("used for long-term reference"), "not used" ("not used for reference ").

  In addition, although the value which a reference mark can take is set as above, it is not restricted to it. For example, a reference mark may be set to the value of “used for inter-layer reference”. Further, without distinguishing between “short-term reference use” and “long-term reference use”, the union of both may be defined as “used for reference”.

  The output mark is information indicating whether or not a picture on the DPB needs to be output to the outside. Specifically, the output mark takes one of the values “needed for output” and “not needed for output”.

  Note that the reference mark and the output mark may not be explicitly set at a specific timing of the decoding process or the encoding process. In this case, the reference mark or the output mark is determined as “undefined”.

  Here, an example of the state of the DPB will be described with reference to FIG. FIG. 6 illustrates the state of the DPB at a specific timing of the decoding process. In FIG. 6, four pictures are recorded in the DPB, and a POC, a reference mark (RM in the figure), and an output mark (OM in the figure) are associated with each other. If a picture with POC a, reference mark b, and output mark c is written as “abc”, DPB has “2-LR-NNfO”, “10-NR-NNfO”, “14-SR-NfO”, “ The picture “12-unk-NfO” is included. Note that the value of the reference mark is abbreviated. “Long-term reference use” is “LR”, “Long-term reference use” is “SR”, “Long-term reference use” is “NR”, “Undefined” is “unk” ". In addition, the value of the output mark is abbreviated to indicate “output required” as “NfO” and “output unnecessary” as “NNfO”. In the following description regarding the present invention, unless otherwise specified, the above-described notation method is used to represent the state of a picture in the DPB.

  In this DPB state example, the picture of “10-NR-NNfO” is “reference not used” and “output unnecessary”, so it is not necessary to keep it in the DPB, and it is deleted from the DPB as early as possible. By doing so, you can save DPB memory usage.

<RPS deriving unit 142>
The RPS deriving unit 142 derives an RPS (reference picture set) used for decoding processing of the target picture based on the syntax value related to the input RPS, and a base reference picture control unit 144, a reference picture control unit 143, and The data is output to the RPL deriving unit 145.

  Hereinafter, a syntax value related to RPS and details of RPS derivation processing based on the syntax value will be described.

(RPS information)
Syntax values related to RPS (hereinafter referred to as RPS information) include the following.
1. 1. SPS short-term RPS information: short-term reference picture set information included in the SPS 2. SPS long-term RP information: long-term reference picture information included in SPS SH short-term RPS information: short-term reference picture set information included in the slice header SH long-term RP information: long-term reference picture information included in the slice header ILRP information: Inter-layer reference picture information (1. SPS short-term RPS information)
The SPS short-term RPS information includes information on a plurality of short-term RPSs that can be used from each picture referring to the SPS. Note that the short-term RPS is a set of pictures that can be a reference picture (short-term reference picture) specified by a relative position with respect to the target picture (for example, a POC difference from the target picture).

  The SPS short-term RPS information will be described with reference to FIG. FIG. 7 illustrates a part of the SPS syntax table used at the time of SPS decoding. The portion (A) in FIG. 7 corresponds to SPS short-term RPS information. The SPS short-term RPS information includes the number of short-term RPS included in the SPS (num_short_term_ref_pic_sets) and the definition of each short-term RPS (short_term_ref_pic_set (i)).

  The short-term RPS information will be described with reference to FIG. FIG. 8 illustrates a short-term RPS syntax table used at the time of SPS decoding and slice header decoding.

  The short-term RPS information includes the number of short-term reference pictures (num_negative_pics) whose display order is earlier than that of the target picture and the number of short-term reference pictures (num_positive_pics) whose display order is later than that of the target picture. In the following, a short-term reference picture whose display order is earlier than the target picture is referred to as a front short-term reference picture, and a short-term reference picture whose display order is later than the target picture is referred to as a rear short-term reference picture.

  The short-term RPS information includes, for each forward short-term reference picture, the absolute value of the POC difference with respect to the target picture (delta_poc_s0_minus1 [i]), and the presence / absence of use as a reference picture of the target picture (used_by_curr_pic_s0_flag [ i]). In addition, for each backward short-term reference picture, the absolute value of the POC difference with respect to the target picture (delta_poc_s1_minus1 [i]) and the possibility of being used as the reference picture of the target picture (used_by_curr_pic_s1_flag [i]) are included It is.

(2. SPS long-term RP information)
The SPS long-term RP information includes information on a plurality of long-term reference pictures that can be used from each picture that references the SPS. A long-term reference picture is a picture specified by an absolute position (for example, POC) in a sequence.

  Decoding of SPS long-term RP information will be described with reference to FIG. 7 again. 7B corresponds to the SPS long-term RP information. The SPS long-term RP information includes information (long_term_ref_pics_present_flag) indicating the presence or absence of a long-term reference picture transmitted in the SPS, the number of long-term reference pictures included in the SPS (num_long_term_ref_pics_sps), and information on each long-term reference picture. The long-term reference picture information includes the POC (lt_ref_pic_poc_lsb_sps [i]) of the reference picture and the possibility of being used as the reference picture of the target picture (used_by_curr_pic_lt_sps_flag [i]).

  The POC of the reference picture may be the POC value associated with the reference picture, or the LSB (Least Significant Bit) of the POC, that is, the POC divided by a predetermined number of powers of 2. The remainder value may be used.

(3. SH short-term RPS information)
The SH short-term RPS information includes information of a single short-term RPS that can be used from a picture that references a slice header.

  Decoding of SPS short-term RPS information will be described with reference to FIG. FIG. 9 illustrates a part of a slice header syntax table used at the time of decoding the slice header. 9A corresponds to the SH short-term RPS information. The SH short-term RPS information includes a flag (short_term_ref_pic_set_sps_flag) indicating whether the short-term RPS is selected from the short-term RPS decoded by the SPS or explicitly included in the slice header. When selecting from SPS decoded, an identifier (short_term_ref_pic_set_idx) for selecting one decoded short-term RPS is included. When explicitly included in the slice header, information corresponding to the syntax table (short_term_ref_pic_set (idx)) described with reference to FIG. 8 is included in the SPS short-term RPS information.

(4. SH long-term RP information)
The SH long-term RP information includes information on a long-term reference picture that can be used from a picture that references a slice header.

  Decoding of SH long-term RP information will be described with reference to FIG. 9 again. 9B corresponds to the SH long-term RP information. The SH long-term RP information is included in the slice header only when a long-term reference picture is available in the target picture (long_term_ref_pic_present_flag). When one or more long-term reference pictures have been decoded by SPS (num_long_term_ref_pics_sps> 0), the number of reference pictures (num_long_term_sps) that can be referred to by the target picture among the long-term reference pictures decoded by SPS is the SH long-term RP information included. Further, the number of long-term reference pictures (num_long_term_pics) explicitly transmitted in the slice header is included in the SH long-term RP information. In addition, information (lt_idx_sps [i]) for selecting the num_long_term_sps number of long-term reference pictures from among the long-term reference pictures transmitted by the SPS is included in the SH long-term RP information. Furthermore, as information on long-term reference pictures to be explicitly included in the slice header, the number of reference pictures POC (poc_lsb_lt [i]) and the presence / absence of use as a reference picture of the target picture (used_by_curr_pic_lt_flag) [i]) is included.

(5. ILRP information)
The ILRP information includes information on an inter-layer reference picture that can be referred to by inter-layer prediction from a picture including a slice header.

  ILRP information will be described with reference to FIG. FIG. 10 shows a part of a syntax table that is referred to at the time of slice decoding, and corresponds to ILRP information.

  The ILRP information includes an inter-layer prediction enabled flag (inter_layer_pred_enabled_flag). Furthermore, when the inter-layer prediction valid flag is 1 (inter-layer prediction is valid) and the number of reference layers that can be referred from the target picture (NumDirectRefLayers [nuh_layer_id]) is greater than 1, the number of inter-layer reference pictures is set. The representing syntax (num_inter_layr_ref_pics_minus1) is included in the ILRP information. The number of inter-layer reference pictures (NumActiveRefLayerPics) in the target picture is set to a value of “num_inter_layer_ref_pics_minus1 + 1”. In addition, a layer identifier (inter_layer_pred_layer_idc [i]) indicating a layer to which each inter-layer reference picture belongs is included in the ILRP information.

  Note that each syntax included in the ILRP information may be omitted if it is obvious. For example, when the number of inter-layer reference pictures that can be referenced from one picture is limited to one, the syntax related to the number of inter-layer reference pictures is not necessary.

(RPS derivation process)
Next, details of the RPS deriving process based on the RPS information will be described.

  The RPS derivation process can divide a plurality of RPS subsets into derivation processes. A subset of RPS is defined as follows:

The RPS is divided into the following two subsets depending on the type of the referenceable picture.
-Current picture referenceable list ListCurr: List of referenceable pictures in the target picture-Subsequent picture referenceable list ListFoll: List of pictures that are not referenced in the target picture but can be referred to in subsequent pictures in the decoding order of the target picture The number of pictures included in the picture referable list is referred to as the current picture referable picture number NumCurrList.

The current picture referable list further includes four partial lists.
Current picture long-term referable list ListLtCurr: Current picture referenceable picture specified by SPS long-term RP information or SH long-term RP information Current picture short-term forward-referenceable list ListStCurrBefore: Specified by SPS short-term RPS information or SH short-term RPS information Current picture referenceable picture whose display order is earlier than the current picture. Current picture short-term backward referenceable list ListStCurrAfter: current picture referenceable picture specified by SPS short-term RPS information or SH short-term RPS information, and displayed Current picture layer referenceable list ListIlCurr: Current picture referenceable picture specified by ILRP information The subsequent picture referenceable list is further composed of two partial lists.
Subsequent picture long-term referable list ListLtFoll: Subsequent picture referenceable picture specified by SPS long-term RP information or SH long-term RP information.
Subsequent picture short-term referable list ListStFoll: current picture referable picture specified by SPS short-term RPS information or SH short-term RPS information.

The current picture short-term forward referenceable list ListStCurrBefore, current picture short-term backward referenceable list ListStCurrAfter, current picture long-term referenceable list ListLtCurr, current picture inter-layer referenceable list ListIlCurr, subsequent picture short-term referenceable list ListStFoll, and RPS The subsequent picture long-term referable list ListLtFoll is generated by the following procedure. In addition, the current picture referenceable picture number NumPocTotalCurr is derived. Each of the referable lists is set to be empty before starting the following process.
(S101) Based on the SPS short-term RPS information and the SH short-term RPS information, a single short-term reference picture set used for decoding the current picture is specified. Specifically, when the value of short_term_ref_pic_set_sps included in the SH short-term RPS information is 0, the short-term RPS explicitly transmitted in the slice header included in the SH short-term RPS information is selected. Other than that (when the value of short_term_ref_pic_set_sps is 1, the short-term RPS indicated by short_term_ref_pic_set_idx included in the SH short-term RPS information is selected from a plurality of short-term RPSs included in the SPS short-term RPS information.
(S102) The POC of each reference picture included in the selected short-term RPS is derived. When the reference picture is a forward short-term reference picture, the reference picture POC is derived by subtracting the value of “delta_poc_s0_minus1 [i] +1” from the POC of the target picture. On the other hand, when the reference picture is a backward short-term reference picture, it is derived by adding the value of “delta_poc_s1_minus1 [i] +1” to the POC of the target picture.
(S103) Confirm the forward reference pictures included in the short-term RPS in the order of transmission, and if the associated used_by_curr_pic_s0_flag [i] value is 1, add the forward reference picture to the current picture short-term forward-referenceable list ListStCurrBefore To do. Otherwise (used_by_curr_pic_s0_flag [i] value is 0), the forward reference picture is added to the subsequent picture short-term referable list ListStFoll.
(S104) The backward reference pictures included in the short-term RPS are confirmed in the order of transmission, and if the associated used_by_curr_pic_s1_flag [i] value is 1, the backward reference picture is added to the current picture short-term backward-referenceable list ListStCurrAfter To do. Other than that (when the value of used_by_curr_pic_s1_flag [i] is 0, the forward reference picture is added to the subsequent picture short-term referable list ListStFoll.
(S105) Based on the SPS long-term RP information and the SH long-term RP information, a long-term reference picture used for decoding the current picture is specified. Specifically, num_long_term_sps reference pictures are selected from the reference pictures included in the SPS long-term RP information, and are sequentially added to the long-term RPS. The selected reference picture is the reference picture indicated by lt_idx_sps [i]. Subsequently, reference pictures included in the SH long-term RP information are sequentially added to the long-term RPS as many reference pictures as num_long_term_pics.
(S106) The POC of each reference picture included in the long-term RPS is derived. The POC of the long-term reference picture is directly derived from the value of poc_lst_lt [i] or lt_ref_pic_poc_lsb_sps [i] decoded in association with each other.
(S107) Check the reference pictures included in the long-term RPS in order, and if the value of associated used_by_curr_pic_lt_flag [i] or used_by_curr_pic_lt_sps_flag [i] is 1, the long-term reference picture is listed as the current picture long-term referenceable list ListLtCurr Add to In other cases (used_by_curr_pic_lt_flag [i] or used_by_curr_pic_lt_sps_flag [i] has a value of 0), the long-term reference picture is added to the subsequent picture long-term referable list ListLtFoll.
(S108) The reference picture specified by the ILRP information is added to the current picture layer referenceable list ListIlCurr. The addition is executed in the following order when inter-layer reference prediction is enabled (inter_layer_pred_enabled_flag is 1). First, reference layers of several inter-layer reference pictures included in ILRP information are selected based on a reference layer identifier (inter_layer_pred_layer_idc [i]) and set as an active reference layer. Next, a picture associated with the same POC as the target picture on the active reference layer is added to the inter-picture-layer referenceable list ListIlCurr.
(S109) The value of the variable NumPocTotalCurr is set to the sum of reference pictures that can be referenced from the current picture. That is, the value of the variable NumPocTotalCurr is set to each of the four lists of the current picture short-term forward reference list ListStCurrBefore, the current picture short-term backward reference list ListStCurrAfter, the current picture long-term reference list ListLtCurr, and the current picture inter-layer referenceable list ListIlCurr. Set to the sum of the number of elements.

<Reference picture control unit 143>
The reference picture control unit 143 updates the DPB 141 based on the input RPS. Schematically, the reference picture control unit 143 sets a reference mark of a picture that is indicated as being referenceable by the inter prediction of the target picture (current picture) in the input RPS as “reference use” (“short-term reference use” or Set to “Long term reference use”). In addition, a decoded picture of the target layer recorded in the DPB, which is not marked as “reference use” in the above process, is set to “reference nonuse”. The reference mark of the inter-layer reference picture on the DPB is not changed by the reference picture control unit 143. In other words, the reference mark change of the picture on the DPB derived from the base decoded picture is not performed by the reference picture control unit 143 but is performed by the base reference picture control unit 144 described later.

The reference mark setting process in the reference picture control unit 143 is executed in the following procedure.
(S201) The reference marks of all the pictures on the DPB are set to “not use reference”.
(S202) For each picture that satisfies the following condition (a) or (b), a corresponding picture on the DPB, that is, a picture to which the same POC is assigned, is specified and the short-term reference is used as a reference mark. Set to.

  (A) Pictures included in the RPS current picture short-term forward referenceable list ListStCurrBefore.

(B) Each picture included in the list StListCurrAfter that can be referred to long-term backward reference of the current picture of RPS.
(S203) For each picture that satisfies the following condition (c), the corresponding picture on the DPB is identified and the reference mark is set to “long-term reference use”.

  (C) Pictures included in the RPS current picture long-term referable list ListLtCurr.

  The reference mark setting procedure described above is an example, and the reference mark may be set by another procedure as long as the reference marks of the pictures on the DPB are the same. For example, the processing of S202 and S203 of the above procedure is executed first, and the mark of each picture on the DPB that is a picture whose reference mark has not been changed in both procedures and is a decoded picture of the target layer is “referenced”. It may be set to “unused”.

<Base Reference Picture Control Unit 144>
The base reference picture control unit 144 updates the DPB 141 based on the input base decoded picture and RPS. Schematically, the base reference picture control unit 144 uses, as an inter-layer reference picture, a base decoded picture corresponding to a picture that can be referred to in inter-layer inter prediction of the target picture (current picture) in the input RPS. Record in DPB. In addition, the reference mark of the inter-layer reference picture recorded in the DPB is set to “reference use” (“short-term reference use” or “long-term reference use”). In addition, the output mark of the inter-layer reference picture is set to “output unnecessary” on the DPB.

  More specifically, the reference mark setting process in the base reference picture control unit 144 is executed in the following procedure.

  (S301) For each picture that satisfies the following condition (d), a picture buffer on the DPB is secured, and the input base decoded picture is recorded in the picture buffer.

  (D) Pictures included in the RPS current picture layer referenceable list ListIlCurr.

  When recording the base decoded picture, it may be recorded in the picture buffer after applying scaling and filtering as necessary. In particular, when the resolutions of the output pictures of the reference layer and the target layer are different (in the case of spatial scalability), it is necessary to scale the base decoded picture that is the decoded picture of the reference layer according to the resolution of the output picture of the target layer.

  (S302) The reference mark of each picture recorded in the DPB in S301 is set to “use reference”.

  (S303) The output mark of each picture recorded in the DPB in S301 is set to “no output required”.

<RPL deriving unit 145>
The RPL deriving unit 145 derives and outputs a reference picture list used for inter prediction or inter-layer image prediction of the target slice of the target picture based on the input RPS and the RPL information included in the input syntax value.

(RPL information)
The RPL information is a syntax value decoded from the SPS or the slice header in order to construct the reference picture list RPL. The RPL information includes SPS list correction information and SH list correction information.

  The SPS list modification information is information included in the SPS, and is information related to restrictions on modification of the reference picture list. The SPS list correction information will be described with reference to FIG. 7 again. 7C corresponds to the SPS list correction information. In the SPS list correction information, a flag (restricted_ref_pic_lists_flag) indicating whether or not the reference picture list is common in the previous slice included in the picture, and a flag (whether or not information related to list rearrangement exists in the slice header ( lists_modification_present_flag).

  The SH list correction information is information included in the slice header, and the update information of the length of the reference picture list (reference list length) applied to the target picture, and the reordering information of the reference picture list (reference list reordering information) ) Is included. The SH list correction information will be described with reference to FIG. FIG. 11 illustrates a part of a slice header syntax table used when decoding a slice header. A portion (C) in FIG. 11 corresponds to SH list correction information.

  The reference list length update information includes a flag (num_ref_idx_active_override_flag) indicating whether or not the list length is updated. In addition, information (num_ref_idx_l0_active_minus1) indicating the reference list length after the change of the L0 reference list and information (num_ref_idx_l1_active_minus1) indicating the reference list length after the change of the L1 reference list are included.

  Information included in the slice header as reference list rearrangement information will be described with reference to FIG. FIG. 12 exemplifies a syntax table of reference list rearrangement information used at the time of decoding the slice header.

  The reference list rearrangement information includes an L0 reference list rearrangement presence / absence flag (ref_pic_list_modification_flag_l0). When the value of the flag is 1 (when the L0 reference list is rearranged) and NumPocTotalCurr is larger than 2, the L0 reference list rearrangement order (list_entry_l0 [i]) is included in the reference list rearrangement information. Here, NumPocTotalCurr is a variable representing the number of reference pictures that can be used in the current picture. Therefore, the L0 reference list rearrangement order is included in the slice header only when the L0 reference list is rearranged and the number of reference pictures available in the current picture is larger than two.

  Similarly, when the reference picture is a B slice, that is, when the L1 reference list is available in the current picture, the L1 reference list rearrangement presence / absence flag (ref_pic_list_modification_flag_l1) is included in the reference list rearrangement information. When the value of the flag is 1 and NumPocTotalCurr is greater than 2, the L1 reference list rearrangement order (list_entry_l1 [i]) is included in the reference list rearrangement information. In other words, the L1 reference list rearrangement order is included in the slice header only when the L1 reference list is rearranged and the number of reference pictures available in the current picture is larger than two.

(RPL derivation process)
Details of the reference picture list construction process will be described. The reference picture list deriving unit generates a reference picture list RPL based on the reference picture set RPS and the RPL modification information.

The reference picture list is composed of two lists, an L0 reference list and an L1 reference list. First, the construction procedure of the L0 reference list will be described. The L0 reference list is constructed according to the procedure shown in S301 to S307 below.
(S301) A temporary L0 reference list is generated and initialized to an empty list.
(S302) The reference pictures included in the current picture short-term forward referenceable list are sequentially added to the provisional L0 reference list.
(S303) Reference pictures included in the current picture short-term backward referenceable list are sequentially added to the provisional L0 reference list.
(S304) Reference pictures included in the current picture long-term referable list are sequentially added to the provisional L0 reference list.
(S305) When the reference picture list is modified (when the value of lists_modification_present_flag included in the RPL modification information is 1), the following processes of S306a to S306b are executed. Otherwise (when the value of lists_modification_present_flag is 0), the process of S307 is executed.
(S306a) When modification of the L0 reference picture is valid (when the value of ref_pic_list_modification_flag_l0 included in the RPL modification information is 1) and the current picture referenceable picture number NumCurrList is equal to 2, S306b is executed. . Otherwise, S306c is executed.
(S306b) The value of the list rearrangement order list_entry_l0 [i] included in the RPL correction information is set by the following equation, and then S306c is executed.

list_entry_l0 [0] = 1
list_entry_l0 [1] = 0
(S306c) Based on the value of the reference list rearrangement order list_entry_l0 [i], the elements of the provisional L0 reference list are rearranged to form the L0 reference list. The element RefPicList0 [rIdx] of the L0 reference list corresponding to the reference picture index rIdx is derived by the following equation. Here, RefListTemp0 [i] represents the i-th element of the provisional L0 reference list.

RefPicList0 [rIdx] = RefPicListTemp0 [list_entry_l0 [rIdx]]
According to the above formula, in the reference list rearrangement order list_entry_l0 [i], the value recorded at the position indicated by the reference picture index rIdx is referred to, and the reference recorded at the position of the value in the provisional L0 reference list The picture is stored as a reference picture at the position of rIdx in the L0 reference list.
(S307) The provisional L0 reference list is set as the L0 reference list.

  Next, an L1 reference list is constructed. Note that the L1 reference list can also be constructed in the same procedure as the L0 reference list. In the L0 reference list construction procedure (S301 to S307), the L0 reference picture, the L0 reference list, the provisional L0 reference list, and list_entry_l0 may be replaced with the L1 reference picture, the L1 reference list, the provisional L1 reference list, and list_entry_l1, respectively.

<Output control unit 146>
In general, the output control unit 146 outputs the picture of the DPB 141 to the outside at a predetermined timing and updates the output mark. Specifically, the picture output process by the output control unit 146 is executed in the following procedure.

  (S401) Among the pictures on the DPB, the picture having the smallest POC among the pictures whose output mark is “output required” is output.

  (S402) The output mark of the picture output in S401 is set to “no output required”.

  (S403) From the pictures on the DPB, select a picture whose reference mark is “reference not used” and whose output mark is “output unnecessary”, and delete the picture from the DPB.

  The above process can be referred to as a general unnecessary picture deletion process (general bumping process) on the DPB. In general bumping processing, each picture on the DPB is a decoded picture of the target layer or an inter-layer reference picture (a picture derived from the decoded picture of the reference layer) without determining whether the picture on the DPB is a decoded picture of the target layer. Unnecessary pictures on the output and DPB can be deleted.

(Picture management processing procedure)
Next, picture management processing in the decoded picture management unit 14 will be described with reference to FIG. FIG. 1 is a flowchart showing an outline of a picture management process at the time of decoding a target picture in the decoded picture management unit 14. The picture management process at the time of decoding the target picture is executed according to the following steps S501 to S510.

  (S501) The RPS deriving unit 142 derives the RPS of the current picture from the input syntax value. The derived RPS is output to the reference picture control unit 143, the base reference picture control unit 144, and the RPL deriving unit 145. The process proceeds to S502.

  (S502) The reference picture control unit 143 updates the reference mark of the decoded picture of the target layer on the DPB based on the input RPS. The process proceeds to S503.

  (S503) The base reference picture control unit 144 determines whether the input RPS includes an inter-layer reference picture. If included (YES in S503), the process proceeds to S504. On the other hand, if not included (NO in S503), the process proceeds to S506.

  (S504) The base reference picture control unit 144 records the input base decoded picture in the picture buffer on the DPB 141 as an inter-layer reference picture, and sets a reference mark. The process proceeds to S505.

  (S505) The base reference picture control unit 144 sets the output mark of the inter-layer reference picture recorded in S504 to “output unnecessary”. The process proceeds to S506.

  (S506) The output control unit 146 outputs the picture on the DPB 141 to the outside as necessary, and updates the output mark of the picture. The process proceeds to S507.

  (S507) Based on the input RPS and syntax value, the RPL deriving unit 145 generates a reference picture list used at the time of decoding the target picture and outputs the reference picture list to the outside. The process proceeds to S508. At this timing, the target picture is decoded using the reference picture list and the reference picture on the DPB 141 to generate a decoded picture of the target picture.

  (S508) The decoded picture of the input target picture is recorded in the picture buffer on the DPB 141. The process proceeds to S509.

  (S509) The reference mark of the inter-layer reference picture is set to “not use reference”. The process proceeds to S510.

  (S510) The reference mark of the target picture is set to “use short-term reference”. The process ends.

(Specific example of picture management processing)
Next, a specific example of the picture management process described with reference to FIG. 1 will be described with reference to FIG. FIG. 13 is a diagram for describing a specific example of picture management processing for two pictures (in order, picture M and picture N) in decoding order on the target layer. FIG. 13A shows a flow of picture management processing. FIG. 13B shows the state of the reference mark and output mark of the inter-layer reference picture regarding the picture M on the DPB in each step shown in FIG.

  Hereinafter, the description will be made according to the flow shown in FIG. In the description, the reference mark of the inter-layer reference picture related to the picture M on the DPB is “a” and the output mark is “b”, and the state of the inter-layer reference picture related to the picture M is “ab”. . The POC of picture M is m, and the POC of picture N is n. Further, the symbols for each step indicate the corresponding step in FIG. For example, “Step S504M” corresponds to “Step S504 for picture M”.

  (S500M) The picture M is set as the target picture. The inter-layer reference picture related to the picture M does not exist on the DPB of the target layer at this stage, and the state is “unk-unk”.

  (S504M) The base decoded picture whose POC is m is recorded in the DPB as an inter-layer reference picture (inter-layer reference picture related to the picture M). In addition, the reference mark is set to “Long Term Reference Use” (“LR”). The state of the inter-layer reference picture related to the picture M is “LR-unk”.

  (S505M) The output mark of the inter-layer reference picture related to the picture M is set to “output unnecessary” (“NNfO”). The state of the inter-layer reference picture related to the picture M is “LR-NNfO”.

  (S506M) The DPB picture is output and the output mark is updated. Here, since the reference mark of the inter-layer reference picture related to the picture M is “long-term reference use”, the picture is not removed from the DPB, and the output mark is not updated. Therefore, the state of the inter-layer reference picture related to the picture M remains “LR-NNfO”.

  (S508M) The decoded picture of picture M is added to the DPB. The state of the inter-layer reference picture related to the picture M remains “LR-NNfO”.

  (S509M) The reference mark of the inter-layer reference picture is updated to “reference not used” (“NR”). The state of the inter-layer reference picture related to the picture M is “NR-NNfO”.

  (S500N) The picture N is set as the target picture. The state of the inter-layer reference picture related to the picture M remains “NR-NNfO”.

  (S506N) The DPB picture is output and the output mark is updated. Here, since the inter-layer reference picture related to the picture M on the DPB is “reference not used” and “output unnecessary”, the inter-layer reference picture is removed from the DPB.

(Picture management process summary)
As described above, the picture management processing in the decoded picture management unit 14 included in the hierarchical video decoding device 1 described with reference to FIGS. 1 and 13 has the following characteristics A1 to A4.

  A1. Includes a process of adding an inter-layer reference picture to the DPB.

  A2. When the inter-layer reference picture is added to the DPB (immediately after the processing of A1), the processing includes setting the reference mark of the inter-layer reference picture of the target picture to “long-term reference use”.

  A3. When the inter-layer reference picture is added to the DPB (immediately after the processing of A1), the processing includes setting the output mark of the inter-layer reference picture to “output unnecessary”.

  A4. After completion of the decoded picture generation process of the target picture, the process includes a process of updating the reference mark of the inter-layer reference picture to “unused reference”.

  From the above characteristics, according to the picture management processing by the decoded picture management unit 14, a picture used as an inter-layer reference picture in decoding of the target picture can be added to the DPB and used for generating a decoded picture of the target picture. In addition, when the decoded picture generation is completed, the state of the reference mark and output mark of the inter-layer reference picture that are not required for subsequent picture decoding can be set to a state that can be discarded from the DPB by a general bumping process. .

  Therefore, according to the picture management process performed by the decoded picture management unit 14, the inter-layer reference picture used for picture decoding can be removed from the DPB when it is no longer needed. Can be decrypted. In addition, since it is possible to discard unnecessary pictures on the DPB without distinguishing whether the pictures on the DPB are inter-layer reference pictures or inter-reference reference pictures, an inter-layer reference picture is used for decoding the input encoded data in this process. By using the same algorithm as when not used, the circuit scale can be reduced.

  In other words, the decoded picture management unit 14 can determine a picture to be discarded from the decoded picture buffer (DPB 141) by the same bumping process as that performed by the decoded picture management unit included in the base decoding unit 15. The circuit scale can be reduced.

  In addition, although it has been described that the reference mark is set to “long-term reference use” in A2 of the above feature, the reference mark is not limited to this, but another reference mark (for example, “short-term reference use” indicating that it may be used for decoding of the target picture. Or “inter-layer reference use”) may be set. When a reference mark indicating that there is a possibility of being used for decoding the target picture is expressed as “reference use”, the feature of A2 may be replaced with the feature of A2 ′ below.

  A2 '. When the inter-layer reference picture is added to the DPB (immediately after the process of A1), the process includes a process of setting the reference mark of the inter-layer reference picture of the target picture to “use reference”.

  Further, the processing order described in the above A1 to A4 and FIG. 1 is merely an example, and is not limited thereto.

  Specifically, the processing of A1, A2, and A3 may be executed at a timing before the start of the target picture derivation processing (picture decoding processing by the picture decoding unit 13).

  In other words, the picture management unit 14 adds an inter-layer reference picture generated based on the decoded picture of the lower layer to the decoded picture buffer prior to the decoded picture derivation process of the target picture in the upper layer (the process of A1 is performed). The reference mark of the inter-layer reference picture may be set to be used for reference (execution of the process A2), and the output mark may be set so as not to be output (the process of A3 is executed).

  The process A4 may be executed after the target picture derivation process (the picture decoding process by the picture decoding unit 13) is completed and before the decoding process of the subsequent picture in the same layer is started in the decoding order.

  In other words, the picture management unit 14 sets the reference mark of the inter-layer reference picture at a timing after the decoding of the decoded picture of the target picture is completed and before the target picture starts decoding the next picture of the target picture in decoding order. The reference may not be used (the process of A4 is executed).

(Base decoding unit)
The detailed structure of the base decoding part 15 is demonstrated using FIG. FIG. 14 is a functional block diagram illustrating the configuration of the base decoding unit 15.

  As illustrated in FIG. 14, the base decoding unit 15 includes a variable length decoding unit 151, a base picture decoding unit 153, and a base decoded picture management unit 154.

  The variable length decoding unit 151 decodes various syntax values related to various coding parameters (typically transform coefficients, prediction information, RPS information, and RPL information) from the binary included in the reference layer encoded data DATA # R. . The syntax of the prediction information and the transform coefficient decoded by the variable length decoding unit 151 is the same as that of the variable length decoding unit 12 in FIG.

  The variable length decoding unit 151 supplies the decoded syntax value to the base picture decoding unit 153 and the base decoded picture management unit 154.

  The base picture decoding unit 153 decodes the decoded picture (base decoded picture) of the target picture on the reference layer based on the slice header, the prediction information, and the syntax value related to the transform coefficient regarding the target picture on the reference layer. Is output. The base decoded picture decoding process in the base picture decoding unit 153 is the same as the picture decoding process in the picture decoding unit 13 in FIG.

  The base decoded picture management unit 154 records the input base decoded picture in the internal DPB, and generates a reference picture list and determines a base decoded picture to be output based on the input syntax value. The generated reference picture list is output to the base picture decoding unit 153, and the base decoded picture is output to the outside. Note that the processing in the base decoded picture management unit 154 is the same as the processing in the decoded picture management unit 14 in FIG.

(Effect of moving image decoding apparatus 1)
The hierarchical moving picture decoding apparatus 1 according to the present embodiment described above includes the picture management unit 14, and the picture management unit adds the inter-layer reference picture to the DPB, and subsequently the reference mark of the inter-layer reference picture. Is set to “reference use”, and a base reference picture control unit 144 is provided for setting the output mark of the inter-layer reference picture to “output unnecessary”. Therefore, the hierarchical video decoding device 1 can remove the inter-layer reference picture used for picture decoding from the DPB when it becomes unnecessary by using a bumping process that does not distinguish between the inter-layer reference picture and the inter-prediction reference picture. Therefore, input encoded data can be decoded with a small average memory usage.

[Modification 1: Reference mark for common DPB]
In the picture management processing of the hierarchical moving picture decoding apparatus 1 described above, it is assumed that each layer has a decoded picture of each layer and a DPB for managing the reference picture. However, the present invention is not limited to this. For example, the decoded picture management unit 14 includes a DPB 141 that manages the decoded picture of the enhancement layer that is the target layer, and the base decoded picture management unit 154 in the base decoding unit 15 includes a DPB that manages the base decoded picture. Although it has been assumed, the hierarchical moving picture decoding apparatus can be configured so that both use a common DPB. Hereinafter, a mechanism for managing decoded pictures of a plurality of layers with a common DPB is referred to as a common DPB model.

  When the common DPB model is used, it is necessary to change at least the following points in the configuration described in the hierarchical video decoding device 1.

1. Bumping processing method Updating method of reference mark and output mark processing of picture on DPB Each change will be described below.

(1. Change of bumping method)
It changes so that the bumping process in the output control part 146 may be performed in the following procedure.

  (S401r) Of the pictures on the DPB, the picture with the smallest POC is output among the pictures of the target layer whose output mark is “output required”.

  (S402r) The output mark of the picture output in S401r is set to “no output required”.

  (S403r) From the pictures on the DPB, select a picture whose reference mark is “reference not used” and whose output mark is “output unnecessary”, and delete the picture from the DPB.

(2. Change of update method of reference mark and output mark processing of picture on DPB)
A part of the process of FIG. 1 is replaced by the process shown in FIG. FIG. 15 is a flowchart showing a part of the picture management process when using the common DPM model. Processes S509r1 to S510r1 shown in FIG. 15 described below replace the processes of S509 and S510 of FIG.

  (S509r1) The reference mark of the target picture is updated according to the value of the discardable flag (discardable_flag). The discardable flag is a flag indicating whether or not there is a possibility that the target picture is used as a reference picture or an inter-layer reference picture during subsequent picture decoding. If the value of the discardable flag is 1, the reference mark is updated to “not use reference”. If the value of the discardable flag is 0, the reference mark is updated to “use reference”.

  (S510r1) Of the pictures recorded on the DPB, the reference marks of the pictures that satisfy all of the following conditions D1, D2, and D3 are set to “not use reference”.

  D1: A picture having the same POC as the POC of the target picture.

  D2: A picture that is not used as an inter-layer reference picture when decoding a picture having the same POC as the POC of the target picture that is later in decoding order than the target picture.

  D3: A picture that is not used by prediction (for example, inter prediction) in the same layer.

  With the picture management process with the above changes, even when a common DPB model is used, the reference mark of a decoded picture that is no longer referred to in inter-layer image prediction or inter prediction can be set to “not use reference”. Therefore, since the picture can be deleted from the DPB at the time when the output mark of the picture on the DPB is changed to “output unnecessary”, the average memory usage can be reduced.

  Note that a configuration in which S510r1 is omitted may be used. In that case, it is preferable to provide a constraint that the value of the discardable flag is 1 for a picture that is not referenced in inter-layer image prediction or inter prediction in a subsequent picture that is later in decoding order than the target picture. While the degree of freedom in selecting the discardable flag is reduced, there is an advantage that the processing amount can be reduced because the processing of S510r1 can be omitted.

[Modification 1-1: Consideration of Reference Mark and Sublayer in Common DPB]
In the picture management process applied to the common DPB model described in the first modification, determination using information on a sublayer (temporal layer) to which a picture belongs may be included.

  The sublayer is used to express temporal scalability. Generally, a time identifier (temporal_id) is assigned to a picture belonging to each sublayer. The time identifier is assigned a smaller value in lower order sublayers, in other words, in sublayers corresponding to lower frame rates. When an output image having a frame rate corresponding to a specific sublayer is to be obtained, encoded data to which a time identifier having a value less than or equal to the time identifier assigned to the sublayer is extracted from hierarchically encoded data. What is necessary is just to decode.

  For example, suppose that the value of the time identifier is a as Tid = a, Tid = 0 corresponds to 15 fps (frame per second), Tid = 1 corresponds to 30 fps, and Tid = 2 corresponds to 60 fps. In this case, in order to obtain an output image of 15 fps, encoded data of Tid = 0 may be extracted and decoded. In order to obtain an output image of 60 fps, encoded data of Tid = 0, Tid = 1, and Tid = 2 may be extracted and decoded. The maximum time identifier included in the hierarchically encoded data is also called the highest time identifier (Highest Temporal Id).

  In order to realize the above characteristics, a picture with Tid = k has a feature that only a picture having a time identifier of k or less can be referred to by inter prediction. Therefore, when the picture having the highest time identifier is not referred to in inter prediction, it can be used only as a reference picture (inter-layer reference picture) by inter-layer image prediction.

  In this modification, a part of the process of FIG. 1 is replaced by the process shown in FIG. FIG. 16 is a flowchart showing a part of picture management processing when the common DPM model is used. Processes S509r1 and S510r2 to S515r2 shown in FIG. 16 described below replace the processes of S509 and S510 in FIG. Note that the processing of S509 is the same as the processing of the same reference numerals described in the first modification, and description thereof is omitted.

  (S510r2) It is determined whether the current picture is referred to in inter-layer image prediction in the subsequent picture in decoding order. When referred to (YES in S510r2), the process proceeds to S514r2. If not referenced (NO in S510r2), the process proceeds to S511r2.

  (S511r2) It is determined whether inter prediction is permitted between pictures having the highest time identifier (HighestTid). If permitted (YES in S511r2), the process proceeds to S513r2. If not permitted (NO in S511r2), the process proceeds to S512r2.

  (S512r2) It is determined whether the time identifier of the target picture is the highest time identifier (HighestTid). When it is the highest time identifier (YES in S512r2), the process proceeds to S515r2. If it is not the highest order identifier (NO in S512r2), the process proceeds to S513r2.

  (S513r2) It is determined whether the current picture is referred to in inter prediction. When referred to by inter prediction (YES in S513r2), the process proceeds to S515r2. When not referred by inter prediction (it is NO at S513r2), it progresses to S514r2.

  (S514r2) The reference mark of the target picture is set to “use reference”, and the process ends.

  (S515r2) The reference mark of the target picture is set to “not use reference”, and the process ends.

  The above-described processing described with reference to FIG. 16 can be roughly expressed as follows. That is, in order to set a reference mark for a decoded picture, first, the possibility of reference in the subsequent inter-layer image prediction is checked. Then, when not used in inter-layer image prediction, a determination using a temporal identifier is performed prior to the presence or absence of reference possibility in inter prediction. When the determination using the temporal identifier satisfies a predetermined condition, the determination of the possibility of reference in inter prediction is omitted, and the reference mark of the target picture is set to “reference not used”. Here, the predetermined condition is whether or not the target picture belongs to the highest order sublayer (has the highest order time identifier) and inter prediction is permitted between pictures belonging to the highest order sublayer.

  Judgment of reference possibility in inter prediction is not easy when using RPS mechanism. This is because it is not known whether the target picture is used for inter prediction until the RPS of the next picture is decoded in the decoding order and the reference mark of the DPB is updated.

  According to the picture management process described above, when inter prediction between pictures in the highest order sublayer is limited, determination of inter prediction availability can be omitted by using the fact. Accordingly, since the target picture can be marked as “reference not used” at an earlier stage, the output target picture can be deleted from the DPB earlier, and the average memory usage of the DPB can be reduced.

  In particular, it is judged whether or not inter prediction is applied between pictures belonging to the highest order sublayer. Pictures belonging to the highest order sublayer generally have low image quality, and often refer to pictures belonging to lower order sublayers. Because. Whether or not to apply inter prediction may be determined between pictures belonging to sublayers lower than the highest order by the same determination as in S511r2. Since the number of pictures belonging to the highest order sublayer is larger than that of pictures belonging to the lower order sublayer and the effectiveness of inter prediction between pictures belonging to the highest order sublayer is low, pictures for any of the sublayers If it is determined whether or not inter prediction is applied, it is preferably performed on the highest order sublayer.

  When application of inter prediction is prohibited between pictures having the same temporal identifier, the determination in S511r2 and S512r2 is redundant and is preferably omitted.

  The configuration of the modification 1-1 can be expressed as follows. That is, in the image decoding apparatus that restores the decoded picture of the upper layer, the picture management unit that manages the decoded picture buffer (DPB) of the upper layer prior to the decoded picture derivation process of the target picture in the upper layer, When the inter-layer prediction application in the sublayer to which it belongs is determined and the determination result is applicable to inter-layer prediction, the inter-layer reference picture generated based on the decoded picture of the lower layer is added to the decoded picture buffer, The reference mark of the inter-layer reference picture is set to be used for reference, and the output mark is set to be unnecessary for output.

[Hierarchical video encoding device]
Below, the structure of the hierarchy moving image encoder 2 which concerns on this embodiment is demonstrated with reference to FIG.

(Configuration of Hierarchical Video Encoding Device)
A schematic configuration of the hierarchical video encoding device 2 will be described with reference to FIG. FIG. 17 is a functional block diagram showing a schematic configuration of the hierarchical video encoding device 2. The hierarchical video encoding device 2 encodes the input image PIN # T of the target layer with reference to the reference layer encoded data DATA # R to generate hierarchical encoded data DATA of the target layer. It is assumed that the reference layer encoded data DATA # R has been encoded in the hierarchical video encoding apparatus corresponding to the reference layer.

  As shown in FIG. 17, the hierarchical video encoding device 2 includes a picture encoding unit 21, a variable length encoding unit 22, a NAL multiplexing unit 23, a decoded picture management unit 14, and a base decoding unit 15.

  The picture encoding unit 21 encodes the input image PIN # T to generate and output a corresponding syntax value. Also, a decoded picture is generated and output by performing decoding processing using the simultaneously output syntax value. The picture encoding unit 21 uses inter prediction or inter-layer image prediction using an input reference picture list to generate a syntax value or a decoded picture.

  The variable length coding unit performs variable length coding on the input syntax value, and outputs the result as target layer coded data DATA # T.

  The NAL multiplexing unit 23 generates NAL-multiplexed hierarchical moving image encoded data DATA by storing the input target layer encoded data DATA # T and reference layer encoded data DATA # R in the NAL unit. And output to the outside.

  The decoded picture management unit 14 is the same component as the decoded picture management unit 14 included in the hierarchical moving image decoding apparatus 1 described above. However, since the decoded picture management unit 14 included in the hierarchical video encoding device 2 does not need to output the picture recorded in the DPB as an output picture, the output can be omitted. Note that the description described as “decoding” in the description of the decoded picture management unit 14 of the hierarchical video decoding device 1 is also applied to the decoded picture management unit 14 of the hierarchical video encoding device 2 by replacing “encoding”. it can.

  The base decoding unit 15 is the same component as the base decoding unit 15 included in the hierarchical video decoding device 1 described above, and detailed description thereof is omitted.

(Effect of moving picture coding apparatus 2)
The hierarchical video encoding apparatus 2 according to the present embodiment described above includes the picture management unit 14, which adds the inter-layer reference picture to the DPB, and subsequently refers to the inter-layer reference picture. The base reference picture control unit 144 sets the mark to “use reference” and sets the output mark of the inter-layer reference picture to “output unnecessary”. Therefore, the hierarchical video encoding device 2 uses the bumping process that does not distinguish the inter-layer reference picture and the inter-prediction reference picture from the DPB when the inter-layer reference picture used for the picture encoding becomes unnecessary. Since it can be removed, the input image can be encoded with a small average memory usage.

(Application example to other hierarchical video encoding / decoding systems)
The above-described hierarchical video encoding device 2 and hierarchical video decoding device 1 can be used by being mounted on various devices that perform transmission, reception, recording, and reproduction of moving images. The moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.

  Based on FIG. 18, it will be described that the above-described hierarchical video encoding device 2 and hierarchical video decoding device 1 can be used for transmission and reception of video. FIG. 18A is a block diagram illustrating a configuration of a transmission device PROD_A in which the hierarchical video encoding device 2 is mounted.

  As illustrated in (a) of FIG. 18, the transmission device PROD_A modulates a carrier wave with an encoding unit PROD_A1 that obtains encoded data by encoding a moving image and the encoded data obtained by the encoding unit PROD_A1. Thus, a modulation unit PROD_A2 that obtains a modulation signal and a transmission unit PROD_A3 that transmits the modulation signal obtained by the modulation unit PROD_A2 are provided. The hierarchical moving image encoding apparatus 2 described above is used as the encoding unit PROD_A1.

  The transmission device PROD_A is a camera PROD_A4 that captures a moving image, a recording medium PROD_A5 that records the moving image, an input terminal PROD_A6 that inputs the moving image from the outside, as a supply source of the moving image input to the encoding unit PROD_A1. An image processing unit A7 that generates or processes an image may be further provided. FIG. 18A illustrates a configuration in which the transmission apparatus PROD_A includes all of these, but a part of the configuration may be omitted.

  The recording medium PROD_A5 may be a recording of a non-encoded moving image, or a recording of a moving image encoded by a recording encoding scheme different from the transmission encoding scheme. It may be a thing. In the latter case, a decoding unit (not shown) for decoding the encoded data read from the recording medium PROD_A5 according to the recording encoding method may be interposed between the recording medium PROD_A5 and the encoding unit PROD_A1.

  FIG. 18B is a block diagram illustrating a configuration of a receiving device PROD_B in which the hierarchical video decoding device 1 is mounted. As illustrated in FIG. 18B, the reception device PROD_B includes a reception unit PROD_B1 that receives a modulation signal, a demodulation unit PROD_B2 that obtains encoded data by demodulating the modulation signal received by the reception unit PROD_B1, and a demodulation A decoding unit PROD_B3 that obtains a moving image by decoding the encoded data obtained by the unit PROD_B2. The above-described hierarchical video decoding device 1 is used as the decoding unit PROD_B3.

  The receiving device PROD_B has a display PROD_B4 for displaying a moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside as a supply destination of the moving image output by the decoding unit PROD_B3. PROD_B6 may be further provided. FIG. 18B illustrates a configuration in which the reception apparatus PROD_B includes all of these, but a part may be omitted.

  The recording medium PROD_B5 may be used for recording a non-encoded moving image, or may be encoded using a recording encoding method different from the transmission encoding method. May be. In the latter case, an encoding unit (not shown) for encoding the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5.

  Note that the transmission medium for transmitting the modulation signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, a transmission mode in which the transmission destination is not specified in advance) or communication (here, transmission in which the transmission destination is specified in advance). Refers to the embodiment). That is, the transmission of the modulation signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

  For example, a terrestrial digital broadcast broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by wireless broadcasting. Further, a broadcasting station (such as broadcasting equipment) / receiving station (such as a television receiver) of cable television broadcasting is an example of a transmitting device PROD_A / receiving device PROD_B that transmits and receives a modulated signal by cable broadcasting.

  Also, a server (workstation or the like) / client (television receiver, personal computer, smartphone, etc.) such as a VOD (Video On Demand) service or a video sharing service using the Internet transmits and receives a modulated signal by communication. This is an example of PROD_A / reception device PROD_B (usually, either a wireless or wired transmission medium is used in a LAN, and a wired transmission medium is used in a WAN). Here, the personal computer includes a desktop PC, a laptop PC, and a tablet PC. The smartphone also includes a multi-function mobile phone terminal.

  Note that the client of the video sharing service has a function of encoding a moving image captured by a camera and uploading it to the server in addition to a function of decoding the encoded data downloaded from the server and displaying it on the display. That is, the client of the video sharing service functions as both the transmission device PROD_A and the reception device PROD_B.

  Based on FIG. 19, it will be described that the above-described hierarchical moving image encoding device 2 and hierarchical moving image decoding device 1 can be used for recording and reproduction of moving images. FIG. 19A is a block diagram illustrating a configuration of a recording apparatus PROD_C in which the above-described hierarchical video encoding apparatus 2 is mounted.

  As shown in FIG. 19A, the recording apparatus PROD_C has an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and the encoded data obtained by the encoding unit PROD_C1 on the recording medium PROD_M. A writing unit PROD_C2 for writing. The hierarchical moving image encoding device 2 described above is used as the encoding unit PROD_C1.

  The recording medium PROD_M may be of a type built in the recording device PROD_C, such as (1) HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) SD memory. It may be of the type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory, or (3) DVD (Digital Versatile Disc) or BD (Blu-ray Disc: registration) For example, it may be loaded into a drive device (not shown) built in the recording device PROD_C.

  The recording device PROD_C is a camera PROD_C3 that captures moving images as a supply source of moving images to be input to the encoding unit PROD_C1, an input terminal PROD_C4 for inputting moving images from the outside, and reception for receiving moving images. The unit PROD_C5 and an image processing unit C6 that generates or processes an image may be further provided. FIG. 19A illustrates a configuration in which the recording apparatus PROD_C includes all of these, but a part of the configuration may be omitted.

  The receiving unit PROD_C5 may receive a non-encoded moving image, or may receive encoded data encoded by a transmission encoding scheme different from the recording encoding scheme. You may do. In the latter case, a transmission decoding unit (not shown) that decodes encoded data encoded by the transmission encoding method may be interposed between the reception unit PROD_C5 and the encoding unit PROD_C1.

  Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is a main supply source of moving images). . In addition, a camcorder (in this case, the camera PROD_C3 is a main source of moving images), a personal computer (in this case, the receiving unit PROD_C5 or the image processing unit C6 is a main source of moving images), a smartphone (in this case In this case, the camera PROD_C3 or the receiving unit PROD_C5 is a main supply source of moving images) is also an example of such a recording device PROD_C.

  FIG. 19B is a block diagram illustrating a configuration of a playback device PROD_D in which the above-described hierarchical video decoding device 1 is mounted. As shown in (b) of FIG. 19, the playback device PROD_D reads a moving image by decoding a read unit PROD_D1 that reads encoded data written on the recording medium PROD_M and a coded data read by the read unit PROD_D1. And a decoding unit PROD_D2 to be obtained. The hierarchical moving image decoding apparatus 1 described above is used as the decoding unit PROD_D2.

  Note that the recording medium PROD_M may be of the type built into the playback device PROD_D, such as (1) HDD or SSD, or (2) such as an SD memory card or USB flash memory, It may be of a type connected to the playback device PROD_D, or (3) may be loaded into a drive device (not shown) built in the playback device PROD_D, such as DVD or BD. Good.

  In addition, the playback device PROD_D has a display PROD_D3 that displays a moving image, an output terminal PROD_D4 that outputs the moving image to the outside, and a transmission unit that transmits the moving image as a supply destination of the moving image output by the decoding unit PROD_D2. PROD_D5 may be further provided. FIG. 19B illustrates a configuration in which the playback apparatus PROD_D includes all of these, but some of the configurations may be omitted.

  The transmission unit PROD_D5 may transmit an unencoded moving image, or transmits encoded data encoded by a transmission encoding method different from the recording encoding method. You may do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image with an encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

  Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, an output terminal PROD_D4 to which a television receiver or the like is connected is a main supply destination of moving images). . In addition, a television receiver (in this case, the display PROD_D3 is a main supply destination of moving images), a digital signage (also referred to as an electronic signboard or an electronic bulletin board), and the display PROD_D3 or the transmission unit PROD_D5 is the main supply of moving images. Desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main video image supply destination), laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a moving image) A smartphone (which is a main image supply destination), a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is a main moving image supply destination), and the like are also examples of such a playback device PROD_D.

(About hardware implementation and software implementation)
Finally, each block of the hierarchical video decoding device 1 and the hierarchical video encoding device 2 may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central It may be realized by software using a Processing Unit).

  In the latter case, each of the devices includes a CPU that executes instructions of a control program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program, the program, and A storage device (recording medium) such as a memory for storing various data is provided. An object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a control program for each of the above devices, which is software that realizes the above-described functions, is recorded in a computer-readable manner. This can also be achieved by supplying to each of the above devices and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU (Micro Processing Unit)).

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, CD-ROM (Compact Disc Read-Only Memory) / MO (Magneto-Optical) / Disks including optical disks such as MD (Mini Disc) / DVD (Digital Versatile Disk) / CD-R (CD Recordable), cards such as IC cards (including memory cards) / optical cards, mask ROM / EPROM (Erasable Programmable Read-only Memory (EEPROM) (Electrically Erasable and Programmable Read-only Memory) / Semiconductor memories such as flash ROM, or logic circuits such as PLD (Programmable Logic Device) and FPGA (Field Programmable Gate Array) Etc. can be used.

  Further, each of the above devices may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, an intranet, an extranet, a LAN (Local Area Network), an ISDN (Integrated Services Digital Network), a VAN (Value-Added Network), a CATV (Community Antenna Television) communication network, a virtual private network (Virtual Private Network), A telephone line network, a mobile communication network, a satellite communication network, etc. can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, infra-red such as IrDA (Infrared Data Association) or remote control even with wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, etc. , Bluetooth (registered trademark), IEEE 802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance), mobile phone network, satellite line, terrestrial digital network, etc. Is possible. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The present invention is suitable for a hierarchical image decoding device that decodes encoded data in which image data is hierarchically encoded, and a hierarchical image encoding device that generates encoded data in which image data is hierarchically encoded. Applicable to.

1. Hierarchical video decoding device (image decoding device)
11 NAL demultiplexing unit 12 Variable length decoding unit (variable length decoding means)
13 Picture decoding unit (picture decoding means)
14 Decoded picture management unit (decoded picture management means)
141 DPB
142 RPS derivation unit 143 Reference picture control unit 144 Base reference picture control unit (base reference picture control means)
145 RPL deriving unit 146 Output control unit (output control means)
15 Base decoding unit 151 Variable length decoding unit (variable length decoding means)
153 Base picture decoding unit 154 Base decoded picture management unit (base decoded picture management means)
2. Hierarchical video encoding device (image encoding device)
21 picture encoding unit 22 variable length encoding unit 23 NAL multiplexing unit

Claims (8)

An image decoding apparatus that decodes higher layer encoded data included in hierarchically encoded data and restores a decoded picture of an upper layer,
A picture management unit for managing a decoded picture buffer of an upper layer;
Prior to the decoded picture derivation process of the target picture in the upper layer, the picture management unit adds an inter-layer reference picture generated based on the decoded picture of the lower layer to the decoded picture buffer, and An image decoding apparatus, wherein the reference mark is set to be used for reference, and the output mark of the inter-layer reference picture is set to be unnecessary.   The picture management unit sets the reference mark of the inter-layer reference picture to non-reference at a timing after the completion of the decoded picture derivation process and before the target picture moves to the next target picture in decoding order. The image decoding apparatus according to claim 1, wherein:   The picture management unit refers to an output mark and a reference mark of each picture in the decoded picture buffer, identifies a picture that does not need to be decoded and output after the target picture, and identifies the identified picture as the decoded picture The image decoding apparatus according to claim 1, wherein a bumping process for discarding from the buffer is executed. A base decoding unit that generates a base decoded picture that is a decoded picture of a reference layer picture;
The base decoding unit includes a base decoded picture management unit that manages a base decoded picture buffer that is a decoded picture buffer of the reference layer,
4. The image decoding apparatus according to claim 3, wherein the base picture management unit determines a picture to be discarded from the decoded picture buffer by the same bumping process as the bumping process executed by the picture management unit. An image decoding apparatus that decodes higher layer encoded data included in hierarchically encoded data and restores a decoded picture of an upper layer,
A picture management unit for managing a decoded picture buffer common to a plurality of layers;
The picture management unit determines whether to apply inter prediction between pictures in the same layer in a sublayer to which the target picture belongs, and when the determination result indicates that inter prediction application between pictures in the same layer is not applicable, the target picture An image decoding apparatus characterized in that the reference mark is set not to be used.   6. The image decoding apparatus according to claim 5, wherein the determination of whether or not to apply inter prediction between pictures in the same layer is applied only to pictures belonging to the highest order sublayer. An image decoding apparatus that decodes higher layer encoded data included in hierarchically encoded data and restores a decoded picture of an upper layer,
A picture management unit for managing a decoded picture buffer common to a plurality of layers;
A variable length decoding unit that decodes a syntax value from hierarchically encoded data,
The variable length decoding unit decodes the discardable flag as a syntax value,
The discardable flag takes a value of 1 when the target picture is not referenced in inter prediction or inter-layer image prediction of the subsequent picture in decoding order, and the inter-prediction or inter-layer image of the subsequent picture in the decoding order. Takes a value of 0 when referenced in the prediction,
The picture decoding apparatus, wherein the picture management unit updates a reference mark of the target picture according to a value of the discardable flag. An image encoding device that generates encoded data of an upper layer from an input image,
A picture management unit for managing a decoded picture buffer of an upper layer;
Prior to the decoded picture derivation process of the target picture in the upper layer, the picture management unit adds an inter-layer reference picture generated based on the decoded picture of the lower layer to the decoded picture buffer, and An image encoding apparatus, wherein a reference mark is set to be used for reference, and an output mark of the inter-layer reference picture is set so as not to be output.