JP6656335B2 - Image encoding device, image encoding method, image decoding device, image decoding method, and program - Google Patents

Description

  The present invention relates to encoding and decoding of layers having different spatial resolutions and image qualities. In particular, the present invention relates to an image encoding and decoding technique for dividing an image into a plurality of regions in a moving image and performing encoding and decoding for each of the divided regions.

  As an encoding method of compression recording of a moving image, H.264 is used. H.264 / MPEG-4 AVC (hereinafter H.264) is known.

  In recent years, H.S. As a successor to H.264, activities for international standardization of a more efficient coding method have started, and JCT-VC (Joint Collaborative Team on Video Coding) has been established between ISO / IEC and ITU-T. In this JCT-VC, the standardization of High Efficiency Video Coding (hereinafter referred to as HEVC) is being advanced (Non-Patent Document 1).

  In the HEVC, a technique called a tile division method is adopted in which an image is divided into rectangular regions (tiles), and each region is independently encoded and decoded. Furthermore, in the tile division method, it is proposed to use a Motion Constrained Tile Sets (hereinafter, MCTS) composed of one or more tiles to independently encode and decode the MCTS without depending on other tiles. (Non-Patent Document 2). The proposal described in Non-Patent Document 2 defines that the MCTS can be set in sequence units. That is, if the sequence is the same, the position of the MCTS in each frame is relatively equal. Then, in the above proposal, when encoding and decoding the MCTS in the frame to be processed, a group of pixels in another frame that is relatively equal to the MCTS is set as a target of inter-frame prediction. That is, pixels other than the pixel group are not set as reference pixels to be referred to in the motion vector search. As a result, independence of encoding and decoding in the MCTS can be ensured. Note that the position of the tile included in the MCTS in the image is encoded by being included in an SEI (Supplemental Enhancement Information) message.

  On the other hand, in standardization of HEVC, extension to hierarchical coding is also being studied. In hierarchical coding, tiles to be coded are respectively coded in a base layer and an enhancement layer. Then, the tiles encoded in each layer are multiplexed to generate a bit stream. In the above-described hierarchical coding, the boundary positions of the tiles of the base layer and the boundary positions of the tiles of the enhancement layer can be set independently. Then, when encoding a tile to be encoded in the enhancement layer, it is necessary to refer to the corresponding tile in the base layer, and thus it is necessary to specify the position of the tile in the base layer. Therefore, it has been proposed to use a tile_boundaries_aligned_flag code as a VUI (Video Usability Information) parameter (vui_parameters) of the enhancement layer (Non-Patent Document 3). The code is obtained by encoding match information indicating whether or not the relative positions of the tiles match in each layer. When the code is 1, it is guaranteed that the position of the boundary of the tile of the enhancement layer matches the position of the boundary of the corresponding tile of the base layer. Thereby, since the position of the tile of the base layer called in encoding and decoding of the tile of the enhancement layer can be specified, the tile of the enhancement layer can be encoded and decoded independently, and high-speed encoding and decoding can be performed. To Note that the base layer is the highest layer, and the subsequent enhancement layers are sequentially lower layers.

ITU-TH. 265 (04/2013) High efficiency video coding JCT-VC Contributions JCTVC-M0235 Internet <http: // phenix. int-every. fr / jct / doc_end_user / documents / 13_Incheon / wg11 / >> JCT-VC Contributions JCTVC-M0202 Internet <http: // phenix. int-every. fr / jct / doc_end_user / documents / 13_Incheon / wg11 / >>

  However, the MCTS described in Non-Patent Document 2 does not consider hierarchical coding. That is, when the boundaries of the tiles and the position of the MCTS can be set for each layer, the relative positions of the tiles in each layer may not match. For example, when a predetermined tile of the enhancement layer is included in the MCTS and a tile at a position corresponding to the predetermined tile of the base layer is not included in the MCTS, the base layer corresponds to the predetermined tile. The surrounding tiles other than the position also need to be decoded.

  Here, a specific description will be given with reference to FIG. FIG. 13 shows a state of tile division. 1301 to 1310 each represent a frame. Each of the frames 1301 to 1310 is configured by twelve tiles having tile numbers 0 to 11, respectively. Hereinafter, the tile of tile number 1 is referred to as tile 1. The same applies even if the number changes. Further, for the sake of explanation, it is assumed that the tile division of each frame is divided into two in the horizontal direction and not divided in the vertical direction in the base layer. Further, in the enhancement layer, the tile division of each frame is such that the frame is divided into four in the horizontal direction and three in the vertical direction. The thin frame in the figure represents the boundary of the tile.

  Each of frames 1301, 1303, 1305, and 1307 represents a frame of each layer at time t. Frame 1301 represents the frame of the base layer at time t. A frame 1305 represents a frame of the first extended layer (first extended layer) at time t. A frame 1303 represents a frame obtained by enlarging the reconstructed image obtained by locally decoding the frame 1301 to the resolution of the first enhancement layer. A frame 1309 represents a frame of the second extended layer (second extended layer) at time t. A frame 1307 represents a frame obtained by enlarging the decoded image of the frame 1305 to the resolution of the second enhancement layer.

  Furthermore, frames 1302, 1304, 1306, and 1308 represent frames of each layer at time t + δ. Frame 1302 represents the frame of the base layer at time t + δ. Frame 1306 represents the frame of the first enhancement layer at time t + δ. A frame 1304 represents a frame obtained by enlarging the decoded image of the frame 1302 to the resolution of the first enhancement layer. Frame 1310 represents the frame of the second enhancement layer at time t + δ. A frame 1308 represents a frame obtained by enlarging the decoded image of the frame 1306 to the resolution of the second enhancement layer.

  Hereinafter, the tile 5 of each frame (frames 1305, 1306, 1309, 1310) of the enhancement layer will be described as an MCTS tile. In FIG. 13, the thick frames indicate tiles belonging to MCTS or their corresponding positions.

  In FIG. 13, to decode the MCTS (tile 5) of the frame 1310 of the second enhancement layer, the tile 5 of the frame 1306 of the first enhancement layer needs to be decoded. Furthermore, in order to decode tile 5 of frame 1306 of the first enhancement layer, tile 0 of frame 1302 of the base layer needs to be decoded. Furthermore, in order to decode the tile 0 of the frame 1302 of the base layer, it is necessary to perform inter-frame prediction with reference to the frame 1301, and it is necessary to decode all the tiles of the frame 1301.

  That is, in the related art, when decoding the MCTS of the second enhancement layer at time t + δ, an area other than the area indicating the position of the tile 5 of the frame 1302 of the base layer at time t (the dotted line part of the frame 1302) is decoded. There is a need. Therefore, when encoding and decoding a predetermined tile using MCTS or the like in hierarchical encoding, it is not possible to independently encode and decode only the tile corresponding to the position of the MCTS. There is.

  The present invention has been made in order to solve the above-described problem, and has an object to enable a predetermined tile to be independently coded and decoded without depending on other tiles in hierarchical coding. I have.

The image encoding device of the present invention has, for example, the following configuration. That is, an image encoding apparatus that hierarchically encodes an image forming a moving image in a plurality of layers, wherein the layer is different from the first image corresponding to the first layer and corresponds to a second layer corresponding to a second layer. Generating means for generating two images, a first tile set including one or a plurality of tiles in the first image, and a first tile set including one or a plurality of tiles in the second image. Encoding means for encoding the second tile set, information encoding means, and flag setting means , wherein the second tile set corresponds to the first tile set in the second image. At a corresponding position, the encoding unit encodes the first tile set without referring to anything other than the first tile set in the first image, and encodes the first tile set in the second image. The second tile Encoding the first tile set by referring to at least a partial area of the second image, wherein the encoding unit encodes the second tile set without referring to other than the first tile set. the in the second image by encoding the first tile set limits to see only the second set of tiles, the information encoding means, said first set of tiles and the second The SEI message indicating the restriction on the decoding process of the tile set is encoded, and the flag setting unit uses the information encoding unit to execute the SEI message indicating the restriction on the decoding process of the first tile set and the second tile set. If the message is encoded, at least the position of the first tileset in the first image and the second tileset in the second image Position and sets the tile_boundaries_aligned_flag indicating that corresponding to 1.

The image decoding device of the present invention has, for example, the following configuration. That is, an image decoding apparatus that decodes encoded data generated by hierarchically encoding images forming a moving image in a plurality of layers, and restricts a decoding process of a tile set including one or a plurality of tiles. information decoding means for decoding the SEI message indicating, according to the SEI message, a first set of tiles in the first image that corresponds to the first layer, corresponding to the different second layer between the first layer and a decoding means for decoding the second set of tiles in the second image you, if the SEI message is decoded by the information decoding device, the second tile set, the second image At a position corresponding to the first tile set, and the decoding unit does not refer to anything other than the first tile set in the first image. Decoding the first tile set and decoding the second tile set without referring to anything other than the second tile set in the second image, wherein the decoding means determines that the SEI message is When the first tile set is decoded by referring to at least a part of the area of the second image, the second tile is used in the second image. When the first tile set is decoded with reference to only the set, and the information decoding means decodes the SEI message, at least the position of the first tile set in the first image And tile_boundaries_aligned_ indicating that the position of the second tile set in the second image corresponds to lag is 1.

  According to the present invention, it is possible to set tiles that can be independently coded and decoded in hierarchical coding.

1 is a block diagram illustrating a configuration of an image encoding device 100 according to a first embodiment. Diagram showing an example of the configuration of a tile 4 is a flowchart illustrating image encoding processing in the image encoding device 100 according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration of an image encoding device 400 according to the first embodiment. 5 is a flowchart illustrating an image encoding process in the image encoding device 400 according to the first embodiment. FIG. 4 is a block diagram illustrating a configuration of an image display device 600 according to a second embodiment. FIG. 14 is a block diagram illustrating a configuration of an image decoding unit 605 according to the second embodiment. 11 is a flowchart illustrating image decoding processing in the image decoding unit 605 according to the second embodiment. FIG. 14 is a block diagram illustrating another configuration of the image decoding unit 605 according to the second embodiment. 17 is a flowchart illustrating image decoding processing in another mode of the image decoding unit 605 according to the second embodiment. FIG. 1 is a block diagram illustrating a configuration example of hardware of a computer applicable to an image encoding device or an image decoding device of the present invention. FIG. 6 is a diagram illustrating syntax relating to vui_parameters of a bitstream. Diagram showing an example of a conventional example of a tile configuration

Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.
Hereinafter, a tile that can be independently encoded and decoded, such as each tile included in the MCTS, is referred to as an independent tile, and a group of independent tiles such as the MCTS is referred to as an independent tile set.

<First embodiment>
Hereinafter, an outline of each processing unit included in the image encoding device according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating an image encoding device 100 according to the present embodiment.

  Reference numeral 101 in FIG. 1 denotes a terminal (input means) for inputting an image (input image). It is assumed that the input image is input one frame at a time. A tile setting unit 102 determines the number of tile divisions in one frame in the vertical direction and the horizontal direction, and determines the position of each tile. Further, the tile setting unit 102 determines whether any of the divided tiles is to be encoded as an independent tile. Hereinafter, information indicating the number of divisions of the horizontal direction tile, the number of divisions of the vertical direction tile, and the position of division set by the tile setting unit 102 is referred to as tile division information. The tile division information is described in Non-Patent Literature 1 in the description part of Picture Parameter Set (PPS) which is header data of a picture, and thus the description thereof is omitted here.

  Here, an example of tile division in the present embodiment will be described with reference to FIG. In FIG. 2 of the present embodiment, one frame is 4K2K (4096 pixels in the horizontal direction × 2160 pixels in the vertical direction). Hereinafter, in this embodiment, 4096 pixels in the horizontal direction × 2160 pixels in the vertical direction are referred to as 4096 × 2160 pixels. The same is true even if the number of pixels changes. Further, reference numerals 201 to 206 in FIG. 2 each indicate a frame. Each of the frames 201 to 206 is configured by twelve tiles having tile numbers 0 to 11 by dividing the frame into four in the horizontal direction and into three in the vertical direction. That is, the size of one tile is 1024 × 720 pixels. However, the number of divisions is not limited to this. Further, the tiles 5 and 6 indicated by thick frames in the frames 201 to 206 shown in FIG. 2 are each set as an independent tile, and an area including the tiles 5 and 6 is set as an independent tile set. In addition, the thin frames in the frames 201 to 206 shown in FIG. 2 represent boundaries of each tile. The thick frame in the enlarged image shown in FIG. 2 indicates a position corresponding to these independent tile sets. Further, as is apparent from FIG. 2, the number of divisions of the tiles in the horizontal direction and the vertical direction and the relative positions of the tiles are the same in each layer.

  A frame 201 in FIG. 2 represents a frame of the base layer input at time t. Frame 202 represents the frame of the base layer input at time t + δ. At time t + δ, the frame 201 has been coded and locally decoded (inverse quantization and inverse transform), and when the frame 202 is coded, the locally decoded frame 201 can be used as a reference frame.

  The frame 203 is an enlarged image obtained by generating a reconstructed image by performing local decoding after encoding the frame 201 and further enlarging the reconstructed image to the same size as the enhancement layer. The frame 204 is an enlarged image obtained by generating a reconstructed image by performing local decoding after encoding the frame 202, and further enlarging the reconstructed image to the same size as the enhancement layer.

  Frame 205 represents the frame of the enhancement layer input at time t. Frame 206 represents the frame of the enhancement layer input at time t + δ.

  Returning to the description of each processing unit in FIG. 1 again. Hereinafter, the frame at time t + δ will be described as a frame to be encoded.

  The tile setting unit 102 generates an independent tile flag indicating information on whether or not an independent tile is included for each sequence. The tile setting unit 102 sets the value of the independent tile flag to 1 when the encoding target frame includes the independent tile, and sets the value of the independent tile flag to 0 when the encoding target frame does not include the independent tile. I do. Further, when the frame to be encoded includes an independent tile (the value of the independent tile flag is 1), the tile setting unit 102 generates independent tile position information indicating the position of the independent tile. Generally, independent tile position information is represented by a tile number in an image, but the present invention is not limited to this. Then, the tile setting unit 102 outputs the generated independent tile flag and the independent tile position information to the subsequent stage as tile division information. In the present embodiment, the tile division information output from the tile setting unit 102 is input to the enhancement layer division unit 104, the base layer division unit 105, the independent tile determination unit 106, and the header encoding unit 114.

  103 is a reduction unit. The reduction unit 103 reduces the input image input from the terminal 101 using a predetermined filter or the like, and generates a reduced image (basic layer image) with reduced resolution.

  104 is an enhancement layer division unit. The enhancement layer division unit 104 sets an input image input from the terminal 101 as an image of an enhancement layer (enhancement layer image), and based on the tile division information output by the tile setting unit 102, converts the enhancement layer image into one or more Divide into tiles. Here, the enhancement layer division unit 104 divides the input frame 206 into twelve tiles 0 to 11 as shown in FIG. Further, the enhancement layer division unit 104 outputs each of the divided tiles to the subsequent stage in the order of the tile numbers (0, 1, 2,..., 11).

  Reference numeral 114 denotes a header encoding unit. It generates header code data in sequence units and picture units. In particular, the header encoding unit 114 receives the independent tile flag and the independent tile position information generated by the tile setting unit 102, generates an MCTS SEI (SEI message), and encodes VUI parameters (vui_parameters).

  Reference numeral 105 denotes a basic layer division unit. The base layer division unit 105 divides the base layer image generated by the reduction unit 103 into one or more tiles based on the tile division information output by the tile setting unit 102. That is, the basic layer dividing unit 105 divides the basic layer image into tiles such that the position of each tile based on the tile division information is relatively equal in the basic layer image generated by the reducing unit 103. . In the present embodiment, the base layer division unit 105 divides the input frame 202 into twelve tiles 0 to 11 as shown in FIG. Further, the base layer division unit 105 outputs the divided tiles to the subsequent stages in the order of the tile numbers. Further, the base layer division unit 105 notifies the independent tile determination unit 106 of the number of the tile (tile to be encoded) to be output.

  Reference numeral 106 denotes an independent tile determination unit that determines whether a tile to be coded (a tile to be coded) is an independent tile. The independent tile determination unit 106 determines the tile to be encoded based on the independent tile flag and the independent tile position information generated by the tile setting unit 102 and the number of the tile to be encoded input from the base layer division unit 105. It is determined whether the tile is an independent tile. Here, when the independent tile flag is 1, the position of the independent tile is tile 5 according to the independent tile position information, and the tile to be encoded is tile 5, the independent tile determination unit 106 determines that the tile to be encoded is It can be determined that the tile is an independent tile. Further, the independent tile determination unit 106 outputs the determination result as an independent tile encoding flag to a subsequent stage. Here, the independent tile determination unit 106 sets the value of the independent tile encoding flag to 1 when the encoding target tile is an independent tile, and sets the value of the independent tile encoding flag when the encoding target tile is not an independent tile. Is set to 0.

  Reference numeral 107 denotes a basic layer encoding unit that encodes the image of the encoding target tile of the basic layer image input from the basic layer dividing unit 105. Base layer coding section 107 codes the tile to be coded based on the independent tile coding flag input from independent tile determination section 106 and generates base layer code data.

  Here, the encoding process in the base layer encoding unit 107 when the independent tile encoding flag indicates that the encoding target tile is an independent tile will be described. In this case, the base layer coding unit 107 performs prediction and coding with reference to only pixels at positions relatively equal to the position of the independent tile set including the coding target tile in the locally decoded reconstructed image of the base layer. Perform the conversion. Further, referring to FIG. 2 as an example, when tile 5 of frame 202 is to be encoded, base layer encoding section 107 performs prediction with reference to only tile 5 and tile 6 in the independent tile set of frame 201. And encoding. On the other hand, when the independent tile encoding flag indicates that the tile to be encoded is not an independent tile, the base layer coding unit 107 performs prediction and prediction with reference to all the pixels of the locally decoded reconstructed image of the base layer. Encode errors and the like. Referring to FIG. 2, when tile 2 of frame 202 is to be encoded, base layer encoding section 107 performs prediction and encoding with reference to all tiles (tiles 0 to 11) of frame 201. Do.

  Further, base layer coding section 107 outputs, to the subsequent stage, a prediction mode used for prediction, a prediction error generated by prediction, base layer coded data generated by coding the prediction error, and the like.

  Reference numeral 108 denotes a base layer reconstructing unit that inputs coefficients (prediction mode and prediction error) generated by the base layer encoding unit 107 and locally decodes the prediction error to generate a reconstructed image of the base layer. Further, the base layer reconstruction unit 108 holds the generated reconstructed image. This is because the base layer coding section 107 and the enhancement layer coding section 112 perform prediction using the reconstructed image.

  An enlargement unit 109 enlarges the reconstructed image of the base layer to the size of the enhancement layer. In FIG. 2, an enlargement unit 109 enlarges each reconstructed image of a frame 201 and a frame 202 to generate a frame 203 and a frame 204.

  Reference numeral 112 denotes an enhancement layer encoding unit that encodes an image of a tile input from the enhancement layer division unit 104. The enhancement layer encoding unit 112 selects a reference image based on the independent tile encoding flag input from the independent tile determination unit 106, encodes the encoding target tile, and generates enhancement layer encoded data.

  Here, when the independent tile encoding flag is 1 (the encoding target tile is an independent tile), the enhancement layer encoding unit 112 enlarges the locally decoded reconstructed image of the base layer with the enlarged image. And the reconstructed image of the enhancement layer that has already been used. Then, the enhancement layer coding unit 112 performs prediction and coding with reference to the images included in the independent tile set of each of the enlarged image and the reconstructed image. Further, referring to FIG. 2 as an example, when tile 5 of frame 206 is to be encoded, enhancement layer encoding section 112 performs local decoding of tile 5 and tile 6 of frame 204 and tile 5 of frame 206. The prediction and the encoding are performed with reference to the already-constructed reconstructed image. On the other hand, when the independent tile encoding flag is 0 (the encoding target tile is not an independent tile), the enhancement layer encoding unit 112 enlarges the locally decoded base layer enlarged image and the locally decoded enhancement layer reconstructed image. And making predictions without limiting to independent tiles. Then, enhancement layer encoding section 112 encodes a prediction error or the like generated by prediction.

  Further, enhancement layer coding section 112 performs, similarly to base layer coding section 107, a prediction mode used for prediction, a prediction error generated by prediction, and an enhancement layer code generated by coding the prediction error. Data and the like are output to the subsequent stage.

  Reference numeral 113 denotes an enhancement layer reconstructing unit that performs local decoding using a coefficient (prediction mode and prediction error) generated during the encoding by the enhancement layer encoding unit 112 and generates a reconstructed image of the enhancement layer. . Further, enhancement layer reconstructing section 113 holds the generated reconstructed image for use in the encoding process in enhancement layer encoding section 112.

  Numeral 110 integrates the base layer code data generated by the base layer coding unit 107, the enhancement layer code data generated by the enhancement layer coding unit 112, and the header code data generated by the header coding unit 114, and forms a bit stream. Is an integration unit that generates Reference numeral 111 denotes a terminal that outputs the bit stream generated by the integration unit 110 to the outside.

  The overall control unit 115 controls each processing unit in the image encoding device and transmits parameters between the processing units. In FIG. 1, the connection between the overall control unit 115 and each processing unit in the image encoding device is omitted. The overall control unit 115 can control each processing unit in the image encoding device and read and write parameters between the processing units through either the parameter signal line or the register bus. Further, in the present embodiment, the overall control unit 115 in FIG. 1 is installed in the image encoding device, but the present invention is not limited to this. That is, the overall control unit 115 is installed outside the image encoding apparatus, and controls each processing unit in the image encoding apparatus and reads / writes parameters between the processing units by using any one of the parameter signal line and the register bus. You may go through or.

  The image encoding operation in the image encoding device 100 described above will be described below with reference to the flowchart shown in FIG.

  In step S301, the image coding device 100 acquires the number of layers of the layer coding specified by the user. In the present embodiment, it is assumed that the enhancement layer is one layer, and the layer coding of two layers (a base layer and one enhancement layer) is performed as a whole.

  In step S302, the tile setting unit 102 determines the number of tile divisions and the position of division in the encoding target frame, and further determines whether any tile in the encoding target frame is an independent tile. To determine. In this embodiment, the tiles 5 and 6 are independent tiles, and the tiles 5 and 6 are combined to form one independent tile set. Therefore, in the present embodiment, the independent tile determination unit 106 sets the independent tile flag to 1. Incidentally, when the independent tile is not included in the encoding target frame, the independent tile determination unit 106 sets the independent tile flag to 0. Further, the independent tile determination unit 106 inputs the determined independent tile flag to the enhancement layer division unit 104, the base layer division unit 105, the independent tile determination unit 106, and the header encoding unit 114.

  In step S303, header encoding section 114 determines an independent tile flag input from independent tile determination section 106. If the header encoding unit 114 determines that the independent tile flag is 1, the process proceeds to step S304. If the header encoding unit 114 determines that the independent tile flag is 0, the process proceeds to step S305.

  In step S304, the header encoding unit 114 sets the tile_boundaries_aligned_flag code of vui_parameters representing the coincidence information of the position of each tile to 1. Note that the tile_boundaries_aligned_flag code of the vui_parameters is obtained by encoding match information indicating whether or not the relative positions of tiles match between the layers.

  In step S305, header encoding section 114 encodes video_parameter_set, which is one of the sequence headers. The video_parameter_set code includes a vps_max_layers_minus1 code indicating the number of layers of the hierarchical coding. In this embodiment, vps_max_layers_minus1 is 1. Subsequently, the header encoding unit 114 encodes the Sequence parameter set (described in 7.3.2.2 in Non-Patent Document 1). The Sequence parameter set code also includes vui_parameters. vui_parameters includes the tile_boundaries_aligned_flag code set in step S304. The integration unit 110 receives the code data (video_parameter_set code and Sequence parameter set code) and generates a bit stream. Further, the integrating unit 110 outputs the generated bit stream to the outside of the image encoding device 100 via the terminal 111.

  In step S306, the header encoding unit 114 encodes a picture parameter set (described in 7.4.3.3 in Non-Patent Document 1), which is a picture header. The integrating unit 110 receives the code data (Picture parameter set code) of the picture header and generates a bit stream. Further, the integrating unit 110 outputs the generated bit stream to the outside of the image encoding device 100 via the terminal 111.

  In step S307, header encoding section 114 determines an independent tile flag input from independent tile determination section 106. If the header encoding unit 114 determines that the independent tile flag is 1, the process proceeds to step S308. If the header encoding unit 114 determines that the independent tile flag is 0, the process proceeds to step S309.

  In step S308, since the encoding target sequence includes the independent tile, the header encoding unit 114 encodes the MCTS SEI. The MCTS SEI code is as described in Chapter 2 of Non-Patent Document 2. In the present embodiment, since one frame includes one independent tile set, the num_sets_in_message_minus1 code is 0. The mcts_id code is 0. Further, the code of num_tile_rects_in_set_minus1 is 1. The num_tile_rects_in_set_minus1 code represents the number of independent tiles belonging to the MCTS. In the present embodiment, since two tiles, tile 5 and tile 6, are included as independent tiles in the independent tile set, the value of the num_tile_rects_in_set_minus1 code is 1. Further, the top_left_tile_index code and the bottom_right_tile_index code represent the position of the independent tile. In the present embodiment, the former value is 5 and the latter value is 6. The header encoding unit 114 encodes each piece of header information as described above to generate an MCTS SEI code. Further, integrating section 110 receives the MCTS SEI code generated by header coding section 114 to generate a bit stream, and outputs the bit stream to outside of image coding apparatus 100 via terminal 111.

  In step S309, reduction section 103 reduces the input image to generate a base layer image. In the present embodiment, the base layer is generated by the reduction unit 103 because the enhancement layer is one layer, but the present invention is not limited to this. In the case of hierarchical coding with two or more enhancement layers (three or more layers in total), a plurality of reduction units 103 may be provided, or one reduction unit 103 may generate an image of the required number of layers.

  In step S310, base layer dividing section 105 extracts a base layer tile image to be encoded in order of tile number from the upper left of the image. The base layer division unit 105 outputs the extracted base layer tile image to the base layer encoding unit 107.

  In step S311, the independent tile determination unit 106 inputs the tile number of the encoding target tile from the base layer division unit 105. Further, the independent tile determination unit 106 inputs the independent tile position information of the encoding target tile from the tile setting unit 102. In this embodiment, the independent tile position information is 5 and 6. The independent tile determination unit 106 compares the input tile number of the encoding target tile with the tile number of the independent tile position information. When the tile number of the encoding target tile matches the tile number of the independent tile position information, the independent tile determination unit 106 determines that the encoding target tile is an independent tile, sets the independent tile encoding flag to 1, Proceed to step S312. On the other hand, if the tile number of the tile to be encoded does not match the tile number of the independent tile position information, the independent tile determination unit 106 determines that the tile to be encoded is not an independent tile, and sets the independent tile encoding flag to 0. And the process proceeds to step S313.

  In step S312, the encoding target tile is an independent tile in the encoding target frame of the base layer. For this reason, the base layer encoding unit 107 refers to the reconstructed image included in the independent tile set at a position relatively equal to the position of the encoding target tile in another frame of the base layer that has been locally decoded. Perform inter-frame prediction and encoding. In addition, the base layer encoding unit 107 performs intra prediction and encoding with reference to the locally decoded reconstructed image in the encoding target tile of the encoding target frame. In FIG. 2, the case where the tile 5 of the frame 202 is encoded will be described. The base layer coding unit 107 performs prediction and coding with reference to the locally decoded reconstructed images of the tiles 5 and 6 of the frame 201 and the tile 5 of the frame 202 stored in the base layer reconstructing unit 108. I do. Further, base layer coding section 107 outputs the code data of the coding target tile of the base layer obtained by coding to base section coding data as base layer code data. The integration unit 110 integrates the base layer code data output from the base layer coding unit 107 with other code data output from the header coding unit 114 and the enhancement layer coding unit 112 to generate a bit stream. . Then, the integrating unit 110 outputs the generated bit stream via the terminal 111. Further, the base layer reconstructing unit 108 sequentially generates reconstructed images of the base layer using the coefficients (prediction mode and prediction residual) generated during the coding by the base layer coding unit 107, and the like. Hold.

  In step S313, the encoding target tile is not an independent tile in the encoding target frame of the base layer. Therefore, the base layer coding unit 107 performs inter-frame prediction and coding of the tile to be coded with reference to the entire image of another frame of the base layer that has been locally decoded. In FIG. 2, when encoding tile 5 of frame 202, base layer encoding section 107 locally decodes all tiles of frame 201 and tile 5 of frame 202 stored in base layer reconstructing section 108. Prediction and encoding are performed with reference to the already-constructed reconstructed image. Further, base layer coding section 107 outputs the generated base layer code data to integrating section 110. The integration unit 110 integrates the base layer code data and other code data to generate a bit stream, and outputs the bit stream via the terminal 111, as described in step S312. Further, the base layer reconstructing unit 108 sequentially generates and holds reconstructed images of the base layer using the coefficients and the like generated during the coding by the base layer coding unit 107.

  In step S314, overall control section 115 determines whether or not all tiles of the base layer have been encoded. If it is determined that the encoding process for all the tiles of the base layer has not been completed (NO in step S314), the process returns to step S310, and the base layer division unit 105 extracts and outputs the tile of the next tile number, and performs processing. To continue. On the other hand, when it is determined that the encoding processing of the images of all the tiles of the base layer has been completed (YES in step S314), the process proceeds to step S315.

  In step S315, enhancement layer division section 104 extracts an image of the tile of the enhancement layer to be coded in order of tile number from the upper left of the image. The enhancement layer dividing unit 104 outputs the extracted image of the tile of the enhancement layer to the enhancement layer encoding unit 112.

  In step S316, the independent tile determination unit 106 compares the input tile number of the encoding target tile with the tile number of the independent tile position information, as in the process in step S311. When the tile number of the encoding target tile matches the tile number of the independent tile position information, the independent tile determination unit 106 determines that the encoding target tile is an independent tile, sets the independent tile encoding flag to 1, Proceed to step S317. On the other hand, if the tile number of the tile to be encoded does not match the tile number of the independent tile position information, the independent tile determination unit 106 determines that the tile to be encoded is not an independent tile, and sets the independent tile encoding flag to 0. Then, the process proceeds to step S319.

  In step S317, the encoding target tile is an independent tile in the encoding target frame of the enhancement layer. For this reason, the enlargement unit 109 includes, from the locally decoded base layer reconstructed image stored in the base layer reconstructing unit 108, an independent tile set at a position relatively equal to the position of the encoding target tile. The reconstructed image to be input. The enlarging unit 109 generates an enlarged image by enlarging by filtering or the like using only the input reconstructed image, and outputs the enlarged image to the enhancement layer encoding unit 112.

  In step S318, enhancement layer encoding section 112 predicts and encodes the image of the encoding target tile input from enhancement layer division section 104, using the locally decoded reconstructed image of the base layer as a reference. That is, the enhancement layer encoding unit 112 performs inter-layer prediction with reference to the enlarged image generated in step S317. Further, enhancement layer encoding section 112 is a reconstructed image of an independent tile set at a position relatively equal to the position of the encoding target tile among the locally decoded enhancement layers stored in enhancement layer reconstruction section 113. , The inter-frame prediction of the encoding target tile is performed. Further, enhancement layer coding section 112 performs intra prediction with reference to a locally decoded reconstructed image in the current tile. The enhancement layer encoding unit 112 encodes information (such as a motion vector obtained by inter-frame prediction) related to prediction obtained by these predictions and a prediction error. Further, the enhancement layer reconstructing unit 113 sequentially generates reconstructed images of the enhancement layer using the coefficients (prediction mode and prediction residual) generated during the encoding by the enhancement layer encoding unit 112, and the like. Hold.

  In step S319, the encoding target tile is not an independent tile in the encoding target frame of the enhancement layer. For this reason, the enlarging unit 109 generates an enlarged image by performing filtering or the like on the entire reconstructed image of the base layer stored in the base layer reconstructing unit 108, and generates the enlarged image using the enhanced layer code. Output to the conversion unit 112.

  In step S320, enhancement layer encoding section 112 encodes the image of the encoding target tile input from enhancement layer division section 104 with reference to the locally decoded reconstructed image of the base layer. That is, the enhancement layer encoding unit 112 performs inter-layer prediction with reference to the enlarged image generated in step S319. In addition, the enhancement layer encoding unit 112 performs inter-frame prediction of the encoding target tile with reference to the reconstructed image of the locally decoded enhancement layer stored in the enhancement layer reconstructing unit 113. Furthermore, the enhancement layer encoding unit 112 performs intra prediction of the tile to be encoded with reference to the reconstructed image that has been locally decoded in the tile to be encoded. The enhancement layer encoding unit 112 encodes information and prediction errors related to prediction obtained by these predictions. Further, the enhancement layer reconstructing unit 113 sequentially generates and retains enhancement layer reconstructed images using the coefficients and the like generated during the encoding by the enhancement layer encoding unit 112.

  In step S321, overall control section 115 determines whether or not encoding of all tiles in the enhancement layer has been completed. If it is determined that the encoding processing for all tiles in the enhancement layer has not been completed (NO in step S321), the process returns to step S315, and the enhancement layer division unit 104 extracts and outputs the tile with the next tile number, and performs the processing. continue. On the other hand, when it is determined that the encoding processing of the images of all the tiles in the enhancement layer has been completed (YES in step S321), the process proceeds to step S322.

  In step S322, overall control unit 115 determines whether or not the encoding processing of the images of all the frames included in the sequence input from terminal 101 has been completed. If there is a frame that has not been subjected to the encoding process (NO in step S322), the process advances to step S309 to process the next frame. If there is no frame that has not been subjected to the encoding process (YES in step S322), the encoding process ends.

  With the above configuration and operation, when using the independent tile and the independent tile set, the relative positions of the tiles of the enhancement layer and the base layer can be matched. That is, the tiles included in the independent tile set set in the base layer are set so as to be included in the independent tile set at a position relatively equal to the independent tile set in each enhancement layer. This makes it possible to limit the pixels to be referred to for prediction and decoding of an independent tile in any layer of the hierarchical coding, and to speed up the prediction processing. In particular, by setting a region of interest or the like as an independent tile, the independent tile can be independently coded from the base layer to the enhancement layer without referring to other tiles, so that necessary parts are processed faster than before. It becomes possible.

  Note that, in the present embodiment, as shown in FIG. 2, an example has been described in which only a frame temporally earlier than the encoding target frame is predicted and encoded as a reference frame, but the present invention is not limited to this. That is, it is apparent from the above description that the same reference is made in the case of performing prediction and encoding with reference to a plurality of frames.

  Further, in the present embodiment, the image encoding device 100 using the reduction unit 103 and the enlargement unit 109 has been described, but the present invention is not limited to this. That is, the reduction unit 103 and the enlargement unit 109 may be omitted. Alternatively, the quantization parameter set by enhancement layer coding section 112 may be made smaller than the quantization parameter set by base layer coding section 107, where the reduction rate and the enlargement rate are set to 1. This makes it possible to perform SNR hierarchical coding.

  Further, in the present embodiment, the enlarged image referred to when predicting the tile of the independent tile set of the enhancement layer is generated only by the image of the tile of the base layer at a position relatively equal to the independent tile set. However, the present invention is not limited to this. That is, as in step S319, pixels around the independent tile in the base layer may be referred to.

  Further, in the present embodiment, it has been described that the base layer and the one-layer enhancement layer are subjected to hierarchical coding (two-layer hierarchical coding as a whole). However, the present invention is not limited to this, and three-layer coding The above hierarchical coding may be used. In this case, reduction section 103, enhancement layer division section 104, enhancement layer coding section 112, enhancement layer reconstruction section 113, and expansion section 109 are provided as one set, and the set is provided by the number of layers of the enhancement layer. , And can correspond to more layers. Also, as shown in FIG. 4, it has one enhancement layer coding unit 112, one enhancement layer reconstructing unit 413, one expansion unit 409, and one reduction unit 403. It does not matter.

  FIG. 4 is an image encoding device capable of encoding a plurality of enhancement layers, and includes one enhancement layer encoding unit 112, one enhancement layer reconstructing unit 413, one enlargement unit 409, and one reduction unit 403. It is a block diagram of an image encoding device. 4, components having the same functions as those of the processing units of the image encoding device 100 of FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. Reference numeral 401 denotes a layer number setting unit that sets the number of layers for layer coding. 403 is a reduction unit. While the reducing unit 103 in FIG. 1 reduces the input image input from the terminal 101 to generate one reduced image, the reducing unit 403 determines the input image based on the number of layers input from the number-of-layers setting unit 401. Is reduced to generate reduced images of a plurality of layers. Reference numeral 402 denotes a frame memory, which stores reduced images of each layer generated by the reduction unit 403. Reference numeral 409 denotes an enlarged portion. The enlargement unit 109 in FIG. 1 enlarges the reconstructed image of the base layer to the size of the enhancement layer to generate one enlarged image, whereas the enlargement unit 109 generates a single enlarged image based on the number of layers input from the layer number setting unit 401. Then, the reconstructed image is enlarged to generate enlarged images of a plurality of hierarchies having different resolutions. 413 is an enhancement layer reconfiguration unit. The enhancement layer reconstructing unit 413 receives the number of layers from the number-of-layers setting unit 401, generates a reconstructed image of the enhancement layer using the coefficients and the like generated by the enhancement layer encoding unit 112, and generates the reconstructed image. Output to expansion section 409 and enhancement layer coding section 112. Reference numeral 410 denotes an integration unit that inputs the number of layers from the number-of-layers setting unit 401 and integrates the code data for the number of layers into a bit stream.

  The operation of each processing unit when encoding is performed using the image encoding device 400 illustrated in FIG. 4 will be described below with reference to the flowchart illustrated in FIG. FIG. 5 shows only the part that has been changed from step S309 to step S320 in FIG. 5, steps that perform the same functions as those in FIG. 3 are assigned the same numbers as in FIG. 3, and descriptions thereof will be omitted. Also, the following description will be given assuming that the number-of-layers setting unit 401 sets the number of layers to 3 in step S301 in FIG. In the present invention, the number of layers is not particularly limited. Also, it is assumed that in step S305, header code data is generated with the vps_max_layers_minus1 code set to 2.

  In step S501, the reduction unit 403 generates reduced images for the number of layers of one frame. In the present embodiment, since the number of layers is set to 3 in step S301, the reduction unit 403 generates one base layer image and two enhancement layer images. That is, the reduction unit 403 generates an enhanced first hierarchical (first enhanced layer) image in which the input image is reduced in height and width by １ ／, and a base layer image in which the first enhanced layer image is further reduced in height and width by １ ／. Here, the reduction unit 403 sets the input resolution image as an extended second hierarchical (second extended layer) image. Further, the reduction unit 403 outputs the base layer image, the first enhancement layer image, and the second enhancement layer image to the frame memory 402, respectively.

  As described above, in steps S312 to S314, the overall control unit 115 encodes the base layer image output from the frame memory 402. The base layer reconstruction unit 108 locally decodes the encoded image to generate a reconstructed image, which is stored.

  In step S502, the number-of-layers setting unit 401 sets the base layer encoded in steps S312 to S313 or the enhancement layer of the layer encoded in steps S518 to S520 described later as an upper layer. Further, the number-of-layers setting unit 401 sets an enhancement layer to be encoded following that layer as a lower layer. Here, first, the base layer encoded in steps S312 to S313 is set as an upper layer, and the first enhancement layer is set as a lower layer.

  In step S515, the enhancement layer dividing unit 104 extracts tile images of the enhancement layer to be coded in order of tile number from the upper left of the image of the layer to be coded. The enhancement layer dividing unit 104 outputs the extracted image of the tile of the enhancement layer to the enhancement layer encoding unit 112. Here, an image of the tile to be encoded in the first enhancement layer image is extracted and input to the enhancement layer encoding unit 112.

  In step S517, the encoding target tile is an independent tile in the encoding target frame. For this reason, the enlargement unit 409 obtains an independent tile set at a position relatively equal to the position of the encoding target tile from the reconstructed image of the upper layer stored in the base layer reconstruction unit 108 or the enhancement layer reconstruction unit 413. Is input. The enlargement unit 409 enlarges the image by filtering or the like using only the input reconstructed image to generate an enlarged image, and inputs the enlarged image to the enhancement layer encoding unit 112. Here, enlargement section 409 generates an enlarged image from the reconstructed image stored in base layer reconstruction section 108, and inputs the enlarged image to enhancement layer encoding section 112.

  In step S518, enhancement layer encoding section 112 predicts and encodes the image of the encoding target tile input from enhancement layer division section 104 with reference to the locally decoded reconstructed image. That is, the enhancement layer encoding unit 112 performs inter-layer prediction with reference to the enlarged image generated in step S517. Further, enhancement layer coding section 112 reconstructs an independent tile set at a position relatively equal to the position of the encoding target tile in another frame of the locally decoded enhancement layer stored in enhancement layer reconstructing section 413. Inter-frame prediction is performed with reference to the constituent images. Further, enhancement layer coding section 112 performs intra prediction with reference to the locally decoded reconstructed image in the tile to be coded. The enhancement layer encoding unit 112 encodes information (such as a motion vector obtained by inter-frame prediction) related to prediction obtained by these predictions and a prediction error. Further, the enhancement layer reconstructing unit 413 sequentially generates reconstructed images of the enhancement layer using the coefficients (prediction mode and prediction residual) generated during the encoding by the enhancement layer encoding unit 112, and the like. Hold.

  In step S519, the encoding target tile is not an independent tile in the encoding target frame. For this reason, the enlargement unit 409 transmits the entire reconstructed image of the base layer stored in the base layer reconstructing unit 108 or the entire reconstructed image of the higher enhancement layer stored in the enhancement layer reconstructing unit 413. The image is enlarged by filtering or the like to generate an enlarged image. Further, enlargement section 409 outputs the generated enlarged image to enhancement layer encoding section 112. Here, the enlargement unit 409 generates an enlarged image from the reconstructed image stored in the base layer reconstruction unit 108.

  In step S520, enhancement layer encoding section 112 encodes the image of the encoding target tile input from enhancement layer division section 104, with reference to the locally decoded reconstructed image. That is, the enhancement layer encoding unit 112 performs inter-layer prediction with reference to the enlarged image generated in step S519. Also, the enhancement layer encoding unit 112 performs inter-frame prediction of the encoding target tile with reference to the reconstructed image of the locally decoded enhancement layer stored in the enhancement layer reconstruction unit 413. Furthermore, the enhancement layer encoding unit 112 performs intra prediction of the tile to be encoded with reference to the reconstructed image that has been locally decoded in the tile to be encoded. The enhancement layer encoding unit 112 encodes information and prediction errors related to prediction obtained by these predictions. Further, the enhancement layer reconstructing unit 413 sequentially generates and retains a reconstructed image of the enhancement layer using the coefficients and the like generated during the encoding by the enhancement layer encoding unit 112.

  In step S503, overall control section 115 determines whether or not encoding has been completed for all layers set by number-of-layers setting section 401. If it is determined that the encoding process for all the tiles has not been completed (NO in step S521), the process returns to step S502, and the number-of-layers setting unit 401 sets the next layer as a lower layer, and continues the process. On the other hand, if it is determined that the encoding processing of the images of all tiles in the enhancement layer has been completed (YES in step S521), the process proceeds to step S523. Here, it is determined that the encoding of the second enhancement layer has not been completed, and the process returns to step S502.

  In step S522, overall control unit 115 determines whether or not the encoding processing of the images of all the frames included in the sequence input from terminal 101 has been completed. If there is a frame that has not been subjected to the encoding process (NO in step S522), the process advances to step S501 to process the next frame. If there is no frame that has not been subjected to the encoding process (YES in step S522), the encoding process ends.

  Hereinafter, the encoding process of the second enhancement layer image will be described. That is, in step S502, the layer number setting unit 401 sets the first enhancement layer encoded in steps S518 to S520 as an upper layer and sets the second enhancement layer as a lower layer. In step S515, enhancement layer division section 104 extracts an image of the encoding target tile in the second enhancement layer image, and inputs the image to enhancement layer encoding section 112.

  In step S517, the encoding target tile is an independent tile in the encoding target frame. For this reason, the enlargement unit 409 converts the reconstructed image of the upper layer (first enhancement layer) stored in the enhancement layer reconstruction unit 413 into an independent tile set at a position relatively equal to the position of the encoding target tile. Enter the included reconstructed image. The enlargement unit 409 generates an enlarged image of an upper layer (first enhancement layer) by performing enlargement by filtering or the like using only the input reconstructed image of the independent tile set, and encodes the enlarged image into an enhancement layer encoding unit. Input to 112. In step S518, enhancement layer coding section 112 refers to the locally decoded reconstructed image of the tile image of the encoding target lower layer (second enhancement layer) input from enhancement layer division section 104. Prediction and encoding. That is, the enhancement layer encoding unit 112 performs inter-layer prediction with reference to the enlarged image of the upper layer (first enhancement layer) generated in step S517. Enhancement layer encoding section 112 also generates an independent tile set at a position relatively equal to the position of the encoding target tile in the locally decoded lower layer (second enhancement layer) stored in enhancement layer reconstructing section 413. Is performed with reference to the image of FIG. Further, enhancement layer encoding section 112 performs intra prediction with reference to a locally decoded reconstructed image of a lower layer (second enhancement layer) in the tile to be encoded. The enhancement layer encoding unit 112 encodes information (such as a motion vector obtained by inter-frame prediction) related to prediction obtained by these predictions and a prediction error. Further, enhancement layer reconstructing section 413 sequentially generates and retains a reconstructed image of a lower layer (second enhancement layer) by using a coefficient or the like generated in the course of encoding by enhancement layer encoding section 112. .

  On the other hand, in step S519, the encoding target tile is not an independent tile in the encoding target frame. For this reason, the enlargement unit 409 enlarges the reconstructed image of the upper enhancement layer (first enhancement layer) stored in the enhancement layer reconstructing unit 413 by filtering or the like, and enlarges the upper layer (first enhancement layer). Generates an enlarged image of. Further, enlargement section 409 outputs the generated enlarged image to enhancement layer encoding section 112.

  In step S520, enhancement layer encoding section 112 encodes the image of the encoding target tile input from enhancement layer division section 104 with reference to the locally decoded reconstructed image. That is, the enhancement layer coding unit 112 performs inter-layer prediction with reference to the enlarged image of the upper layer (first enhancement layer) generated in step S519. Further, enhancement layer coding section 112 performs inter-frame prediction with reference to the locally decoded reconstructed image of the lower layer (second enhancement layer layer) stored in enhancement layer reconstructing section 413. Further, enhancement layer encoding section 112 performs intra prediction with reference to a locally decoded reconstructed image in the encoding target tile of the lower layer (second enhancement layer). The enhancement layer encoding unit 112 encodes information and prediction errors related to prediction obtained by these predictions. Further, the enhancement layer reconstructing unit 413 sequentially generates and stores a reconstructed raw image of a lower layer (second enhancement layer) using the coefficients and the like generated during the encoding by the enhancement layer encoding unit 112. I do.

  In step S503, the overall control unit 115 determines whether or not encoding has been completed for all layers set by the number-of-layers setting unit 401. If it is determined that coding has been completed, the process proceeds to step S522. If it is determined that the processing has not been completed, the process returns to step S502. Here, since the encoding up to the second enhancement layer has been completed, the process proceeds to step S522. If encoding of all frames is completed in step S522, the encoding process ends.

  With the above operation, even when there are a plurality of layers of enhancement layers, it is possible to decode only the necessary code data for the independent tile set and generate code data capable of reproducing the decoded image only by referring to the minimum image.

  When the MCTS SEI code exists in the bit stream, the tile_boundaries_aligned_flag code of vui_parameters, which is tile position matching information, is always set to 1. That is, in the vui_parameters, when the MCTS SEI code exists in the bit stream, the tile_boundaries_aligned_flag code as the code data can be omitted. If the MCTS SEI code does not exist in the bit stream, the value of the tile_boundaries_aligned_flag code is encoded, and the code data is included in the bit stream. If the MCTS SEI code is in the bit stream, the value of the tile_boundaries_aligned_flag code is not encoded, and a value of 1 is always set on the decoding side. By doing so, it becomes possible to reduce redundant tile_boundaries_aligned_flag codes.

  Also, in hierarchical coding, an important area is cut out, and an independent tile set is applied to the cut-out area and coding is performed, so that code data from which the important area can be read at high speed can be generated.

<Embodiment 2>
Hereinafter, an overview of each processing unit included in the image decoding device according to the present embodiment will be described with reference to FIG. FIG. 6 is a block diagram illustrating an image display device 600 including the image decoding unit 605 according to the present embodiment. In the present embodiment, a case where the bit stream generated in the first embodiment is decoded will be described as an example.

  An interface 601 inputs a bit stream through communication or the like. A storage unit 602 stores a bit stream input from the interface 601 or a previously recorded bit stream. A display control unit 603 specifies a display method for displaying the bit stream specified by the user. The display control unit 603 outputs the layer to be decoded (layer) and the area to be decoded (display area) to the image decoding unit 605 as a display control signal. In the present embodiment, the layer to be decoded is represented by the number of layers, and the display area is represented by the position of the tile to be displayed. However, the present invention is not limited to this. A selector 604 specifies an input destination of a bit stream to be input. Reference numeral 605 denotes an image decoding unit according to the present embodiment. A display unit 606 displays the decoded image generated by the image decoding unit 605.

  Next, an image display operation in the image display device 600 will be described below. The case where the display control unit 603 is instructed by the user to decode and display the base layer image of the bit stream will be described. This corresponds to a case where a bit stream obtained by encoding an image captured by a monitoring camera or the like is input and the entire captured image is monitored. The interface 601 receives a bit stream (input bit stream) input in units of frames from a monitoring camera or the like, and outputs the bit stream to the storage unit 602 and the selector 604. The storage unit 602 records the input bit stream, and the selector 604 outputs the input bit stream to the image decoding unit 605 according to an instruction from the display control unit 603. The image decoding unit 605 inputs information such as a layer to be displayed and a tile to be displayed from the display control unit 603 as a display control signal. That is, since the display control unit 603 is instructed by the user to decode and display the base layer of the bit stream, the image decoding unit 605 includes information indicating that the layer to be decoded is the base layer, Information indicating a tile is input.

  Hereinafter, an overview of each processing unit included in the image decoding unit 605 according to the present embodiment will be described with reference to FIG. FIG. 7 is a block diagram illustrating the image decoding unit 605 according to the present embodiment.

  Reference numeral 701 in FIG. 7 denotes a terminal for inputting the bit stream output from the selector 604. For ease of explanation, it is assumed that header data and code data for each frame are input to the bit stream. In the present embodiment, it is assumed that the code data in frame units includes all the hierarchical code data constituting one frame. However, the present invention is not limited to this, and input data is input in units such as slices. It does not matter. Further, the data configuration of the frame is not limited to this.

  Reference numeral 702 denotes a terminal for inputting a display control signal regarding decoding output from the display control unit 603 in FIG. As the display control signal, a layer to be decoded and position information of a tile to be decoded are input. Further, the display control signal input to terminal 702 is input to separation section 704, base layer decoding section 707, and enhancement layer decoding section 710. A buffer 703 stores one frame of hierarchical code data input from the terminal 701.

  Reference numeral 704 denotes a separation unit. The separation unit 704 separates header code data, base layer code data, and each enhancement layer code data from the hierarchical code data input from the buffer 703. Further, separation section 704 divides the hierarchical code data (base layer code data and each enhancement layer code data) separated for each layer into code data for each tile and outputs the code data. The separated code data is output to header decoding section 705, base layer decoding section 707, and enhancement layer decoding section 710. When outputting the code data separated for each tile to each processing unit, the separation unit 704 outputs the number of the output tile (the tile to be decoded) to the independent tile determination unit 706 as tile position information.

  705 is a header decoding unit. The header decoding unit 705 decodes header code data in sequence units and picture units, and reproduces parameters necessary for decoding. In particular, when the MCTS SEI code exists in the header code data, the header decoding unit 705 also decodes the MCTS SEI code. In particular, the header decoding unit 705 decodes and reproduces the independent tile flag and the independent tile position information. Reference numeral 706 denotes an independent tile determination unit that determines whether a tile to be decoded (a tile to be decoded) is an independent tile. The independent tile determination unit 706 determines whether the decoding target tile is an independent tile based on the independent tile flag and the independent tile position information input from the header decoding unit 705 and the position information of the decoding target tile input from the separation unit 704. Determine whether or not. Further, independent tile determination section 706 inputs the determination result to base layer decoding section 707 and enhancement layer decoding section 710.

  Reference numeral 707 denotes a base layer decoding unit. The base layer decoding unit 707 decodes the code data of the tile of the base layer separated by the separation unit 704, and generates a decoded image of the base layer. Reference numeral 708 denotes a frame memory which holds a decoded image of each tile of the base layer generated by the base layer decoding unit 707. An enlargement unit 709 enlarges the decoded image of the base layer to the resolution of the enhancement layer to generate an enlarged image. A selector 720 selects a desired decoded image from the decoded image of the base layer or the decoded image of the enhancement layer, and outputs the selected decoded image to the terminal 712. A terminal 712 outputs the decoded image input from the selector 720 to the outside of the image decoding unit 605.

  710 is an enhancement layer decoding unit. The enhancement layer decoding unit 710 decodes the code data of the tile of the enhancement layer separated by the separation unit 704, and generates a decoded image of the enhancement layer. A frame memory 711 holds a decoded image of each tile of the enhancement layer generated by the enhancement layer decoding unit 710.

  The overall control unit 714 controls each processing unit of the image decoding unit 605 and transmits parameters between the processing units. In FIG. 1, the connection between the overall control unit 714 and each processing unit in the image decoding unit 605 is omitted. Then, the overall control unit 714 can control each processing unit in the image decoding unit 605 and read and write parameters between the processing units through either the parameter signal line or the register bus. Further, in the present embodiment, the overall control unit 714 in FIG. 1 is installed in the image decoding unit 605, but the present invention is not limited to this. That is, the overall control unit 714 is provided outside the image decoding unit 605, and controls each processing unit in the image decoding unit 605, and reads and writes parameters between the processing units, using either a parameter signal line or a register bus. You may go through or.

  The image decoding operation of the image decoding unit 605 will be described below with reference to the flowchart shown in FIG.

  First, the case where the decoding target layer (decoding target layer) is only the base layer will be described. Here, it is assumed that the user instructs the display control unit 603 to decode and display the base layer in the bit stream input from the interface 601.

  In step S801, the header code data existing at the head of the bit stream input from the terminal 701 is input to the header decoding unit 705 through the processing by the buffer 703 and the separation unit 704. The header decoding unit 705 decodes video_parameter_set, which is one of the sequence headers. The video_parameter_set includes a vps_max_layers_minus1 code indicating the number of layers of the hierarchical coding. In the present embodiment, the code of vps_max_layers_minus1 is 1. Subsequently, the header decoding unit 705 decodes the Sequence parameter set code. The Sequence parameter set code also includes vui_parameters. vui_parameters includes a tile_boundaries_aligned_flag code that is tile position matching information. In the present embodiment, the tile_boundaries_aligned_flag code is 1.

  In step S802, the header decoding unit 705 decodes the Picture parameter set code. Since the decoding of the header code data is described in detail in Non-Patent Document 1, the description is omitted here.

  In step S803, the independent tile determination unit 706 determines whether there is an independent tile in the decoding target frame. Then, the determination result is set as an independent tile flag. In practice, the presence or absence of MCTS SEI is determined. If MCTS SEI exists in the header code data, the independent tile flag is set to 1 and the process proceeds to step S804. If the MCTS SEI does not exist in the header code data, the independent tile availability flag is set to 0, and the process proceeds to step S805. In the present embodiment, it is determined that MCTS SEI exists in the header code data, the independent tile flag is set to 1, and the process proceeds to step S804. When an independent tile exists in the frame to be decoded, the tile_boundaries_aligned_flag code of vui_parameters, which is tile position matching information, needs to be 1. If the tile_boundaries_aligned_flag code of vui_parameters is not 1, the header decoding unit 705 may return an error and stop decoding. Further, the header decoding unit 705 inputs the independent tile flag to the independent tile determination unit 706, the base layer decoding unit 707, and the enhancement layer decoding unit 710.

  In step S804, the header decoding unit 705 decodes the MCTS SEI code and acquires the independent tile flag and the independent tile position information.

  In step S805, the separation unit 704 inputs the position information of the tile related to the display portion input from the terminal 702. In the present embodiment, display of the entire base layer is instructed. Therefore, the tiles on the display portion are all the tiles of the base layer. That is, the separation unit 704 extracts the code data of the tile to be decoded of the base layer from the buffer 703 in tile order from tile 0 and outputs the code data to the base layer decoding unit 707.

  In step S806, the independent tile determination unit 706 inputs the number of the tile to be decoded from the separation unit 704. Further, the independent tile determination unit 706 inputs the independent tile position information from the header decoding unit 705. In the present embodiment, there is one independent tile set, and the independent tile position information is 5 and 6. The independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. If the tile number of the decoding target tile matches the tile number of the independent tile position information (YES in step S806), the independent tile determination unit 706 determines that the decoding target tile is an independent tile, and proceeds to step S807. If the tile number of the decoding target tile does not match the tile number of the independent tile position information (NO in step S806), it is determined that the decoding target tile is not a tile of the independent tile set, and the process proceeds to step S808.

  In step S807, the decoding target tile is an independent tile in the decoding target frame of the base layer. For this reason, the base layer decoding unit 707 outputs the independent tile in the independent tile set at a position relatively equal to the position of the decoding target tile in another frame of the decoded base layer and the decoded tile in the decoding target tile. Decoding is performed with reference to only pixels. That is, the base layer decoding unit 707 performs the inter-frame prediction with reference to the decoded image of the independent tile in the independent tile set at a position relatively equal to the position of the decoding target tile stored in the frame memory 708. Further, the base layer decoding unit 707 performs intra prediction with reference to the decoded image in the decoding target tile stored in the frame memory 708. Then, the base layer decoding unit 707 stores the decoded image of the decoding target tile of the decoded base layer in the frame memory 708. Note that the decoded image is referred to when decoding tiles thereafter. Further, the base layer decoding unit 707 outputs the decoded image of the tile of the base layer to the display unit 606 of FIG. 6 via the selector 720 and the terminal 712.

  In step S808, the decoding target tile is not an independent tile in the decoding target frame of the base layer. Therefore, the base layer decoding unit 707 performs decoding with reference to the decoded image of the base layer of the decoded frame and the decoded pixels of the base layer of the decoding target frame. That is, the base layer decoding unit 707 performs inter-frame prediction with reference to the decoded image stored in the frame memory 708. Furthermore, the base layer decoding unit 707 performs intra prediction with reference to the decoded image within the tile to be decoded. Then, the base layer decoding unit 707 stores the decoded image of the decoding target tile of the decoded base layer in the frame memory 708. Note that the decoded image is referred to when decoding the subsequent tiles. Further, the base layer decoding unit 707 outputs the decoded image of the tile of the base layer to the display unit 606 of FIG. 6 via the selector 720 and the terminal 712.

  In step S809, the overall control unit 714 determines whether or not code data of all tiles for one frame of the base layer has been decoded. If it is determined that the decoding process of the code data of all tiles for one frame of the base layer has not been completed (NO in step S809), the process returns to step S805, and the separating unit 704 extracts and outputs the next tile. ,continue processing. On the other hand, when it is determined that the decoding process of the code data of all tiles for one frame of the base layer has been completed (YES in step S809), the process proceeds to step S810.

  In step S810, separation section 704 determines whether or not a layer to be decoded and displayed includes an enhancement layer based on a display control signal input from display control section 603 in FIG. 6 via terminal 702. I do. If decoding and display of the enhancement layer has been instructed (YES in step S810), the process proceeds to step S811; otherwise (NO in step S810), the process proceeds to step S818. Here, since the decoding is performed only for the base layer, the process proceeds to step S818, and the enhancement layer decoding unit 710 does not perform the decoding process.

  In step S818, overall control unit 714 determines whether or not decoding processing of base layer code data or enhancement layer code data of all frames included in the sequence input from terminal 701 has been completed. Here, it is determined whether or not the overall control unit 714 has finished decoding the code data of the base layer of all frames. If there is code data of the base layer or the enhancement layer that has not been subjected to the decoding process (NO in step S818), the process proceeds to step S805, and the process of the next frame is performed. If there is no code data of the frame on which the decoding process has not been performed (YES in step S818), the decoding process ends.

  Note that the image decoded by the image decoding unit 605 is output to the display unit 606 in FIG. The display unit 606 displays the entire decoded image of the base layer output from the image decoding unit 605 in response to an instruction to display an image of the base layer from the display control unit 603.

  When the display control unit 603 instructs the display of the basic layer of the recorded video from the user, the input of the selector 604 is set to the storage unit 602. Then, the display control unit 603 selects a necessary bit stream from the storage unit 602 and controls to output the selected bit stream to the selector 604.

  Subsequently, a case where the decoding target layer is the enhancement layer will be described. The decoding process when the user instructs the display control unit 603 to decode and partially display the enhancement layer of the bit stream input from the interface 601 will be described. This corresponds to a case where a part of an image captured by a monitoring camera or the like is monitored in detail. The image decoding unit 605 is instructed by the display control unit 603 on the tile numbers included in the decoding and displaying areas of the base layer and the enhancement layer. In the present embodiment, for simplicity of description, the tiles included in the display area are the areas of the tiles 5 and 6 in FIG. Hereinafter, the decoding operation of the image of the enhancement layer in the image decoding unit 605 will be described with reference to the flowchart shown in FIG. Also, the description of the part that performs the same operation as the decoding of only the base layer will be simplified.

  In step S801, the header decoding unit 705 decodes the video_parameter_set and the Sequence parameter set as in the case where the display of only the base layer is instructed. Then, the header decoding unit 705 decodes the vps_max_layers_minus1 code and the tile_boundaries_aligned_flag code among them.

  In step S802, the header decoding unit 705 decodes the Picture parameter set code as in the case of displaying only the base layer.

  In step S803, the header decoding unit 705 determines whether or not there is an independent tile in the header code data, as in the case of displaying only the base layer.

  In step S804, the header decoding unit 705 decodes the MCTS SEI code and obtains the independent tile flag and the independent tile position information as in the case of displaying only the base layer.

  In step S805, the separation unit 704 inputs the position information of the tile related to the display portion input from the terminal 702. In this description, the positions of the tiles instructed to be displayed are the tiles 5 and 6. Here, first, the separation unit 704 sets the tile to be decoded to the tile 5 based on the position information of the tile instructed to be displayed, which is input from the terminal 702, and extracts the code data of the base layer of the tile 5; The extracted code data is output to base layer decoding section 707. Also, the tile position information instructed to be displayed is input to the independent tile determination unit 706.

  In step S806, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Here, since the tile 5 which is the tile to be decoded is an independent tile, the process proceeds to step 807.

  In step S807, the decoding target tile is an independent tile. As in the case of displaying only the base layer, the base layer decoding unit 707 decodes the code data of the tile 5 of the base layer to generate a decoded image, and stores the decoded image in the frame memory 708. Since the enhancement layer is displayed here, the base layer decoding unit 707 does not output the generated decoded image from the terminal 712. However, the present invention is not limited to this, and the base layer decoding unit 707 can output a decoded image. In that case, it is also possible to output both the decoded image generated by the base layer decoding unit 707 and the decoded image generated by the enhancement layer decoding unit 710, and select and display them on the display unit 606.

  In step S809, overall control unit 714 determines whether or not code data of all tiles of the base layer for the display portion input from separation unit 704 has been decoded. Here, since the decoding of the code data of the tile 6 has not been completed, the process returns to step S806 to decode the code data of the base layer of the tile 6.

  Hereinafter, decoding of the code data of the base layer of the tile 6 will be described.

  In step S805, separation section 704 extracts code data of the base layer of tile 6. In step S806, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Since the tile 6 being the decoding target tile is an independent tile, the process proceeds to step S807. In step S807, base layer decoding section 707 decodes the code data of the base layer of tile 6, and stores the decoded image in frame memory 708.

  In step S809, the overall control unit 714 determines that the code data of all tiles of the base layer for the display portion input from the separation unit 704 has been decoded, and proceeds to step S810.

  In step S810, separation section 704 determines whether or not a layer to be displayed includes an enhancement layer based on a display control signal input via terminal 702 from display control section 603 in FIG. Here, the process proceeds to step S811 because the display is performed up to the enhancement layer.

  In step S811, similarly to step S805, the separating unit 704 inputs the position information of the tile corresponding to the display portion input from the terminal 702. Here, the positions of the tiles instructed to be displayed are the tiles 5 and 6. The separation unit 704 extracts the code data of the enhancement layer of the tile 5 that is the decoding target tile based on the input position information of the tile instructed to be displayed, and extracts the extracted code data from the enhancement layer decoding unit 710. Output to Also, the tile position information instructed to be displayed is input to the independent tile determination unit 706.

  In step S812, as in step S806, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. If the tile numbers match, the process proceeds to step S813; otherwise, the process proceeds to step S815. Here, the independent tile position information is 5 and 6. Therefore, the independent tile determination unit 706 determines that the tile 5 that is the decoding target tile is a tile of the independent tile set, and proceeds to step S813.

  In step S813, the decoding target tile is an independent tile in the decoding target frame of the enhancement layer. The enlargement unit 709 inputs the decoded image included in the independent tile set at a position relatively equal to the position of the decoding target tile from the decoded base layer decoded image stored in the frame memory 708. The enlarging unit 709 generates an enlarged image by enlarging by filtering or the like using only the input decoded image of the independent tile, and outputs the enlarged image to the enhancement layer decoding unit 710.

  In step S814, enhancement layer decoding section 710 decodes the enhancement layer code data of the decoding target tile input from separation section 704. The enhancement layer decoding unit 710 refers to the enlarged image input from the enlargement unit 709, the decoded enhancement layer decoded image stored in the frame memory 711, and the decoded pixel of the decoding target tile. Generate That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the base layer generated in step S813. Also, the enhancement layer decoding unit 710 refers to the decoded image in the independent tile set at a position relatively equal to the position of the decoding target tile among the decoded images of the enhancement layer stored in the frame memory 711 to perform inter-frame prediction. I do. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. More specifically, with reference to FIG. 2, when decoding the tile 5 of the frame 206, the enlarged image of the frame 204, the decoded image of the tiles 5 and 6 of the decoded frame 205, and the tile 5 of the frame 206 are decoded. The decoding is performed with reference to the already decoded pixel. The decoded image of the tile of the enhancement layer generated by the enhancement layer decoding unit 710 is output to the frame memory 711 and held in the frame memory 711. The decoded image of the enhancement layer generated by enhancement layer decoding section 710 is output to display section 606 in FIG. 6 via selector 720 and terminal 712.

  In step S817, overall control unit 714 determines whether or not code data of all tiles of the enhancement layer for the display portion input from separation unit 704 has been decoded. Here, since decoding of the code data of the enhancement layer of the tile 6 has not been completed, the process returns to step S811 to decode the code data of the enhancement layer of the tile 6.

  Hereinafter, decoding of the code data of the enhancement layer of the tile 6 will be described.

  In step S811, the code data of the enhancement layer of tile 6 is extracted. In step S812, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Here, since the tile 6 as the decoding target tile is an independent tile, the process proceeds to step S813.

  In step S813, enlargement section 709 generates an enlarged image using only the decoded image of the input independent tile.

  In step S814, enhancement layer decoding section 710 decodes the code data of the enhancement layer of tile 6 to generate a decoded image, and stores the decoded image in frame memory 711. When decoding the code data of the enhancement layer of the tile 6, the enhancement layer decoding unit 710 includes the enlarged image input from the enlargement unit 709, the decoded image of the decoded enhancement layer stored in the frame memory 711, and the decoding target tile. And the decoded pixel of. That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the base layer generated in step S813. Also, the enhancement layer decoding unit 710 performs inter-frame prediction with reference to a decoded image in an independent tile set at a position relatively equal to the position of the tile to be decoded in the enhancement layer stored in the frame memory 711. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. More specifically, with reference to FIG. 2, when decoding the tile 6 of the frame 206, an enlarged image of the frame 204, a decoded image of the tiles 5 and 6 of the decoded frame 205, and a tile 6 of the frame 206 The decoding is performed with reference to the already decoded pixel. The decoded image of the tile of the enhancement layer generated by the enhancement layer decoding unit 710 is output to the frame memory 711 and held in the frame memory 711. The decoded image of the enhancement layer generated by enhancement layer decoding section 710 is output to display section 606 in FIG. 6 via selector 720 and terminal 712.

  In step S817, the overall control unit 714 determines that the code data of all tiles of the enhancement layer for the display portion has been decoded, and proceeds to step S818.

  In step S818, overall control unit 714 determines whether or not decoding processing of code data of tiles related to a display portion of all frames included in the sequence input from terminal 701 has been completed. If there is a frame that has not been decrypted (NO in step S818), the process advances to step S805 to process the next frame. If there is no frame for which the decoding process has not been performed (YES in step S818), the decoding process ends.

  As described above, the case where the display area (the tile to be decoded) is configured with the independent tile set has been described. The processing up to step S805 is as described above.

  In step S806, the independent tile determination unit 706 determines that the decoding target tile is not an independent tile, and proceeds to step S808. In step S808, similarly to the case where the decoding target layer is only the base layer, base layer decoding section 707 decodes the tile of the base layer to generate a decoded image, and stores the decoded image in frame memory 708. Since the enhancement layer is displayed here, the base layer decoding unit 707 does not output the generated decoded image from the terminal 712.

  In step S809, the overall control unit 714 determines whether or not code data of all tiles for one frame of the base layer has been decoded. Here, the overall control unit 714 determines that the encoded data of all tiles for one frame of the base layer has been decoded, and proceeds to step S810. In step S810, separation section 704 determines that display to the enhancement layer has been instructed based on the input display control signal, and proceeds to step S811. In step S811, the separation unit 704 inputs, from the terminal 702, the position information of the tile related to the display portion. The separation unit 704 extracts the code data of the enhancement layer of the tile to be decoded based on the input position information. In step S812, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Here, the independent tile determination unit 706 determines that the decoding target tile is not a tile of the independent tile set (the tile number of the decoding target tile does not match the tile number of the independent tile position information), and proceeds to step S815.

  In step S815, the decoding target tile is not an independent tile. The enlarging unit 709 extracts, from the decoded base layer decoded image stored in the frame memory 708, a base layer tile at a position relatively equal to the position of the decoding target tile and a decoded image around the tile. input. The enlargement unit 709 generates an enlarged image by performing an enlargement by filtering or the like using the input decoded image of the base layer, and outputs the enlarged image to the enhancement layer decoding unit 710.

  In step S816, enhancement layer decoding section 710 decodes the enhancement layer code data of the decoding target tile input from separation section 704. The enhancement layer decoding unit 710 refers to the enlarged image input from the enlargement unit 709, the decoded enhancement layer decoded image stored in the frame memory 711, and the decoded pixel of the decoding target tile. Generate That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the base layer generated in step S815. Further, the enhancement layer decoding unit 710 performs inter-frame prediction with reference to the decoded image of the enhancement layer stored in the frame memory 711. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. The decoded image of the tile of the enhancement layer generated by the enhancement layer decoding unit 710 is output to the frame memory 711 and held in the frame memory 711. The decoded image of the enhancement layer generated by enhancement layer decoding section 710 is output to display section 606 of FIG.

  In step S817, the overall control unit 714 determines whether all the related tiles have been decoded based on the tile position information on the display portion input to the separation unit 704 from the terminal 702. If decoding of all tiles has not been completed (NO in step S817), the process returns to step S811, and the separating unit 704 extracts and outputs the next tile, and continues the processing. If the decoding process of the code data of all the tiles related to the display portion has been completed (YES in step S817), the process proceeds to step S818.

  In step S818, overall control unit 714 determines whether or not decoding processing of code data for all frames has been completed. If there is code data for which decoding processing has not been performed (NO in step S818), the process advances to step S805 to process the next frame. If there is no code data for which the decoding process has not been performed (YES in step S818), the decoding process ends.

  Returning to FIG. 6, the display unit 606 is instructed by the display control unit 603 to display an image of the enhancement layer. For this reason, the display unit 606 displays the decoded image of the enhancement layer decoded by the image decoding unit 605. Since the enhancement layer has a higher resolution than the base layer, by displaying the decoded image of the enhancement layer, the display unit 606 can obtain the effect of enlarging and displaying a part of the image of the base layer.

  When the display control unit 603 instructs the display of the basic layer of the recorded video from the user, the input of the selector 604 is set to the storage unit 602. Then, the display control unit 603 selects a necessary bit stream from the storage unit 602 and controls to output the selected bit stream to the selector 604.

  With the above configuration and operation, when using the independent tile and the independent tile set, the relative positions of the tiles of the enhancement layer and the base layer can be matched. That is, if the tiles of the base layer are the tiles of the independent decoding tile set, the tiles at positions relatively equal to the positions of the tiles in all the enhancement layers can be the tiles of the independent decoding tile set. Thus, when decoding a layer-encoded bit stream, independent tiles can be decoded only by referring to the minimum image data in any layer. As described above, by reducing the image data to be referred to in the prediction, it is possible to suppress the data transfer amount, reduce the calculation amount, and realize low power consumption. In addition, in the independent tile decoding processing, high-speed processing can be performed by independently decoding tiles other than the independent tile in each layer from the base layer to the enhancement layer without referring to tiles other than the independent tile. In particular, when the bit stream is generated by performing encoding so as to adapt the independent tile set to the important area on the encoding side, the important area can be decoded at high speed when decoding the bit stream. .

  Note that, in the present embodiment, as shown in FIG. 2, an example is shown in which prediction and decoding are performed only on a frame temporally earlier than the decoding target frame as a reference frame, but the present invention is not limited to this. That is, it is obvious from the above description that the same reference is made in the case of prediction and decoding with reference to a plurality of frames.

  Further, in the present embodiment, the image decoding unit 605 using the enlargement unit 709 has been described, but the present invention is not limited to this. That is, the enlargement unit 709 may be omitted. Alternatively, the enlargement ratio may be set to 1, and the quantization parameter decoded by the enhancement layer decoding unit 710 may be smaller than the quantization parameter decoded by the base layer decoding unit 707. This makes it possible to decode the SNR hierarchical data.

  Further, in the present embodiment, an example has been described in which the code data of one frame includes the code data of all the hierarchies, but the present invention is not limited to this, and may be input for each layer. For example, the code data may be collectively stored for each layer in the storage unit 602, and the code data may be cut out and read from the enhancement layer as needed.

  Further, in the present embodiment, a case has been described in which there is a base layer and one enhancement layer (two layers in total). However, the present invention is not limited to this, and there may be three or more layers in total. In this case, by providing the enhancement layer decoding unit 710, the frame memory 711, and the enlargement unit 709 as one set and providing the set by the number of layers of the enhancement layer, it is possible to handle more layers. As shown in FIG. 9, an enhancement layer decoding unit 710, a frame memory 911, and an enlargement unit 909 may be provided one by one, and may be used for decoding of each layer. FIG. 9 is a block diagram of an image decoding device capable of decoding the enhancement layers of a plurality of layers, and having one enhancement layer decoding unit 710, one frame memory 911, and one enlargement unit 909. In FIG. 9, the components having the same functions as those of the respective processing units of the image decoding unit 605 in FIG. A frame memory 908 holds the decoded image generated by the base layer decoding unit 707. The frame memory 908 differs from the frame memory 708 of FIG. 7 in that a function of outputting to the selector 920 is added. An enlargement unit 909 is different from the enlargement unit 709 in FIG. 7 in that an input from the frame memory 911 and an input from the frame memory 908 can be selected and input. Reference numeral 911 denotes a frame memory, which is different from the frame memory 711 in FIG. 7 in that a function of outputting code data of a desired tile to the enlargement unit 909 and the selector 920 is provided. A selector 920 selects and inputs a desired decoded image from the frame memory 908 or the frame memory 911, and outputs the selected decoded image to the terminal 912. A terminal 912 outputs the decoded image input from the selector 920 to the outside of the image decoding unit 605.

  When the decoding process is performed using the image decoding unit 605 illustrated in FIG. 9, the operation of each processing unit will be described below with reference to the flowchart illustrated in FIG. FIG. 10 shows only the part obtained by changing step S818 from step S805 in FIG. In FIG. 10, steps that perform the same functions as the steps in FIG. 8 are given the same numbers as in FIG. 8, and descriptions thereof will be omitted. Further, in the present embodiment, an example in which the image coding apparatus 400 illustrated in FIG. 4 of the first embodiment decodes a bit stream generated by the coding method in FIG. Will be described. In steps S801 to S804 in FIG. 8, as described above, the header decoding unit 705 decodes the header code data. Here, the code of vps_max_layers_minus1 is 2.

  First, a case where the decoding target layer is only the base layer will be described. Here, it is assumed that the user instructs the display control unit 603 to start decoding and displaying the entire base layer in the bit stream input from the interface 601. Hereinafter, it is assumed that the decoding of one frame of the base layer has been completed by steps S805 to S809 in FIG. 8 as in the case of displaying only the base layer described above. However, all the decoded images generated by the base layer decoding unit 707 are stored in the frame memory 908.

  In step S1010, base layer decoding section 707 or enhancement layer decoding section 710 compares the number of layers of the decoded layers with the layer indicated by display control section 603, and determines whether the displayed layer has been decoded. Determine whether or not. If the number of decoded layers has reached the displayed layer (YES in step S1010), the process proceeds to step S1003, and if not (NO in step S1010), the process proceeds to step S1001. Here, it is assumed that separation section 704 has determined that the hierarchy to be displayed is only the base layer based on the display control signal input from terminal 702. Therefore, base layer decoding section 707 determines that the layer to be displayed has been reached in step S1010, and proceeds to step S1003.

  In step S1003, selector 920 selects the decoded image of the lowest layer among the decoded layers. In this case, since the lowest layer is the base layer, the selector 920 reads the decoded image of the base layer decoded from the frame memory 908 and sends the read decoded image to the display unit 606 of FIG. Output. Then, the display unit 606 displays the entire decoded image of the base layer output from the image decoding unit 605 in response to an instruction to display the image of the base layer from the display control unit 603.

  Subsequently, a case where the decoding target layer is the enhancement layer will be described. Here, decoding when the user instructs the display control unit 603 to decode the enhancement layer in the bit stream input from the interface 601 and to display a part of the decoded image of the enhancement layer will be described. Further, as an example, the description will be made assuming that the displayed hierarchy is the second enhancement layer (the number of layers is three). Further, in the present embodiment, for simplicity of description, the tiles included in the display area are the tiles 5 and 6 in FIG. The decoding operation will be described based on the flowchart shown in FIG. 10, as in the case where the decoding and display of only the base layer is instructed. Also, the description of the part that performs the same operation as the decoding of only the base layer will be simplified.

  In step S806, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Here, since the tile 5 which is the tile to be decoded is an independent tile, the process proceeds to step 807. In step S807, base layer decoding section 707 decodes the decoded data of tile 5 of the base layer to generate a decoded image, and stores the decoded image in frame memory 908. In step S809, overall control unit 714 determines whether or not code data of all tiles of the base layer for the display portion input from separation unit 704 has been decoded.

  In step S1010, base layer decoding section 707 or enhancement layer decoding section 710 compares the number of layers of the decoded layers with the layer indicated by display control section 603, and determines whether the displayed layer has been decoded. Determine whether or not. Here, according to the display control signal input from the terminal 702, the displayed hierarchy is the second enhancement layer (the number of hierarchy is three). Therefore, enhancement layer decoding section 710 determines that the layer to be displayed has not been decoded, and proceeds to step S1001.

  In step S1001, enhancement layer decoding section 710 sets the base layer decoded in steps S807 to S808 or the enhancement layer of the layer decoded in steps S1014 to S1016 described later as an upper layer. Further, the subsequent enhancement layer to be decoded is defined as a lower layer. First, the base layer encoded in steps S807 to S808 is set as an upper layer, and the first enhancement layer is set as a lower layer.

  In step S1011, the separation unit 704 inputs the position information of the tile corresponding to the display portion input from the terminal 702. In this description, the positions of the tiles related to the display portion are the tile 5 and the tile 6. Then, based on the position information input from the terminal 702, the separation unit 704 converts the code data of the lower layer (first enhancement layer) of the tile 5 which is the tile to be decoded among the hierarchical code data stored in the buffer 703. Extract. Further, separation section 704 outputs the extracted code data to enhancement layer decoding section 710. Further, the separation unit 704 inputs the tile position information to the independent tile determination unit 706.

  In step S812, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. If the tile numbers match, the process proceeds to step S1013; otherwise, the process proceeds to step S1015. Here, the independent tile position information is 5 and 6, and the tile 5 which is the tile to be decoded matches the tile number of the independent tile position information. Therefore, the independent tile determination unit 706 determines that the tile to be decoded is a tile of the independent tile set, and proceeds to step S1013.

  In step S1013, the decoding target tile is an independent tile. Since the upper layer is the base layer, the enlargement unit 909 determines whether or not the independent tile set included in the independent tile set at a position relatively equal to the position of the decoding target tile from the decoded image of the base layer stored in the frame memory 908. Input the decoded image. The enlarging unit 909 generates an enlarged image by enlarging by filtering or the like using only the input decoded image of the independent tile, and outputs the enlarged image to the enhancement layer decoding unit 710.

  In step S1014, similarly to step S814, enhancement layer decoding section 710 decodes the code data of the lower layer (first enhancement layer) of the decoding target tile input from separation section 704. The enhancement layer decoding unit 710 includes an enlarged image input from the enlargement unit 909, a decoded image of the decoded enhancement layer (first enhancement layer) stored in the frame memory 911, and a decoded pixel of the decoding target tile. To generate a decoded image. That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the upper layer (base layer) generated in step S1013. Also, the enhancement layer decoding unit 710 converts the decoded image in the independent tile set at a position relatively equal to the position of the decoding target tile among the decoded images of the lower layer (first enhancement layer) stored in the frame memory 911. Reference is made to inter-frame prediction. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. The decoded image of the tile of the lower layer (first enhancement layer) decoded by the enhancement layer decoding unit 710 is output to the frame memory 911 and held in the frame memory 911.

  In step S1017, overall control unit 714 determines whether or not code data of all tiles of the lower layer (first enhancement layer) relating to the display portion input from separation unit 704 has been decoded. Here, since the decoding of the code data of the enhancement layer of the tile 6 has not been completed, the process returns to step S1011 to decode the code data of the lower layer (first enhancement layer) of the tile 6.

  Hereinafter, decoding of the code data of the lower layer (first enhancement layer) of the tile 6 will be described.

  In step S1011, the separation unit 704 extracts code data of a lower layer (first enhancement layer) of the tile 6 that is a decoding target tile from among the hierarchical code data stored in the buffer 703. In step S812, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Here, the independent tile determination unit 706 determines that the tile 6 to be decoded is an independent tile, and proceeds to step S1013.

  In step S1013, the enlargement unit 909 generates an enlarged image using only the input decoded image of the independent tile of the upper layer (base layer). That is, the enlargement unit 909 receives the decoded image from the frame memory 908 and enlarges the image by filtering or the like to generate an enlarged image.

  In step S1014, enhancement layer decoding section 710 decodes the code data of the lower layer (first enhancement layer) of tile 6, generates a decoded image, and stores the decoded image in frame memory 911. When decoding the code data of the lower layer (first enhancement layer) of tile 6, enhancement layer decoding section 710 increases the decoded image input from expansion section 909 and the decoded image of the decoded enhancement layer stored in frame memory 911. And the decoded pixel of the tile to be decoded. That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the upper layer (base layer) generated in step S1013. Also, the enhancement layer decoding unit 710 converts the decoded image in the independent tile set at a position relatively equal to the position of the decoding target tile among the decoded images of the lower layer (first enhancement layer) stored in the frame memory 911. Reference is made to inter-frame prediction. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. Further, the decoded image of the tile of the lower layer (first enhancement layer) decoded by the enhancement layer decoding unit 710 is output to the frame memory 911 and held in the frame memory 911.

  In step S1017, the overall control unit 714 determines that the code data of all tiles in the lower layer (first enhancement layer) relating to the display portion has been decoded, and proceeds to step S1002.

  In step S1002, the overall control unit 714 determines whether or not encoding has been completed for all layers represented by the decoded vps_max_layers_minus1 code. If decoding of tiles of all layers has not been completed (NO in step S1002), the process returns to step S1010 to determine display. If decoding of tiles of all layers has been completed (YES in step S1002), the flow advances to step S1003. Here, since the decoding processing of the enhancement layer has not been completed, the enhancement layer decoding unit 710 determines that the decoding processing of the tiles of all the layers has not been completed, and returns to step S1010.

  Hereinafter, decoding of the second enhancement layer is performed. That is, in step S1010, enhancement layer decoding section 710 determines whether or not the layer to be displayed has been decoded. According to the display control signal input from the terminal 702, the layer to be displayed is the second enhancement layer. Here, enhancement layer decoding section 710 proceeds to step S1001 to determine that decoding has been performed only up to the first enhancement layer (the second enhancement layer has not been decoded). In step S1001, enhancement layer decoding section 710 sets the first enhancement layer decoded in steps S1014 to S1016 as an upper layer and sets the second enhancement layer as a lower layer.

  In step S1011, separation section 704 extracts the code data of the tile of the lower layer (second enhancement layer) from the hierarchical code data stored in buffer 703 and inputs the code data to enhancement layer decoding section 710. Here, first, separation section 704 extracts code data of a lower layer (second enhancement layer) of tile 5, and inputs the extracted code data to enhancement layer decoding section 710. In step S812, the independent tile determination unit 706 determines that the tile 5 to be decoded is an independent tile, and proceeds to step S1013. In step S1013, enlargement section 909 has an upper layer as an enhancement layer (first enhancement layer). For this reason, the enlargement unit 909 decodes the independent tile included in the independent tile set at a position relatively equal to the position of the decoding target tile from the decoded image of the upper layer (first enhancement layer) stored in the frame memory 908. Enter an image. That is, the enlargement unit 909 generates an enlarged image by enlarging by filtering or the like using only the input decoded image of the independent tile of the upper layer (first enhancement layer), and sends the enlarged image to the enhancement layer decoding unit 710. input.

  In step S1014, enhancement layer decoding section 710 decodes the code data of the lower layer (second enhancement layer) of the decoding target tile input from separation section 704. Enhancement layer decoding section 710 generates a decoded image with reference to the next image. That is, enhancement layer decoding section 710 decodes the enlarged image of the upper layer (first enhancement layer hierarchy) input from enlargement section 909 and the decoded enhancement layer (second enhancement layer) stored in frame memory 911. The image and the decoded pixel of the decoding target tile are referred to. That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the upper layer (first enhancement layer) generated in step S1013. Also, the enhancement layer decoding unit 710 converts the decoded image in the independent tile set at a position relatively equal to the position of the decoding target tile among the decoded images of the lower layer (first enhancement layer) stored in the frame memory 911. Reference is made to inter-frame prediction. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. Further, the decoded image of the tile of the lower layer (second enhancement layer) decoded by the enhancement layer decoding unit 710 is output to the frame memory 911 and held in the frame memory 911.

  In step S1017, overall control unit 714 determines whether or not code data of all tiles of the lower layer (second enhancement layer) relating to the display portion input from separation unit 704 has been decoded. Here, since the decoding of the code data of the enhancement layer of the tile 6 has not been completed, the process returns to step S1011 to decode the code data of the lower layer (second enhancement layer) of the tile 6. The decoding of the lower layer of the tile 6 is the same as the decoding process of the code data of the second enhancement layer of the tile 5 as described above, provided that the upper layer is the first enhancement layer and the lower layer is the second enhancement layer. Since there is, description is omitted.

  In step S1002, the overall control unit 714 determines that decoding of tiles of all layers has been completed, since decoding has been performed up to the second enhancement layer, and proceeds to step S1003. In step S1003, selector 920 selects the decoded image of the lowest layer among the decoded layers. In this case, since the lowest layer is the second enhancement layer, the selector 920 reads the decoded image of the second enhancement layer from the frame memory 911, and outputs the decoded image of the second enhancement layer via the terminal 912 in FIG. Output to the display unit 606. The display unit 606 displays the entire decoded image of the second enhancement layer output from the image decoding unit 605 when the display control unit 603 instructs the display of the image of the second enhancement layer. .

  In the above description, the layer to be displayed has been described as the second enhancement layer (the number of layers is three). However, when the number of layers of the coded data of the layer coding is three or more and the displayed layer is the first enhancement layer (the number of layers is two), decoding of the first enhancement layer is completed (NO in step S1002). Then, in step S1010, the process proceeds to step S1003. For this reason, decoding of code data of a layer higher than the second enhancement layer is not performed.

  As described above, the case where the display area (the tile to be decoded) is configured with the independent tile set has been described. The processing up to step S805 is as described above.

  In step S806, the independent tile determination unit 706 determines that the decoding target tile is not an independent tile, and proceeds to step S808. In step S808, similarly to the case where the decoding target layer is only the base layer, base layer decoding section 707 decodes the tile of the base layer to generate a decoded image, and stores the decoded image in frame memory 908.

  In step S809, the overall control unit 714 determines whether or not code data of all tiles for one frame of the base layer has been decoded. Here, the base layer decoding unit 707 determines that the encoded data of all the tiles for one frame of the base layer has been decoded, and proceeds to step S1010. In step S1010, since base layer decoding section 707 or enhancement layer decoding section 710 displays up to the second enhancement layer, it determines that the layer to be displayed has not been decoded, and proceeds to step S1001.

  In step S1001, enhancement layer decoding section 710 sets the base layer decoded in step S808 as the upper layer, and sets the subsequent enhancement layer (first enhancement layer) to be decoded as the lower layer. In step S1011, the separation unit 704 inputs the position information of the tile corresponding to the display portion input from the terminal 702. Then, the separating unit 704 extracts the code data of the lower layer (first enhancement layer) of the decoding target tile from the hierarchical code data stored in the buffer 703 based on the input position information. In step S812, the independent tile determination unit 706 compares the tile number of the decoding target tile with the tile number of the independent tile position information. Here, the tile 5 that is the decoding target tile does not match the tile number of the independent tile position information. Therefore, the independent tile determination unit 706 determines that the tile to be decoded is not a tile of the independent tile set, and proceeds to step S1015.

  In step S1015, the enlarging unit 909 calculates, from the decoded image of the upper layer (base layer) stored in the frame memory 908, the tile of the base layer at a position relatively equal to the position of the tile to be decoded and the periphery of the tile. Is input. The enlarging unit 909 generates an enlarged image by enlarging by filtering or the like using only the input decoded image of the tile of the base layer, and outputs the enlarged image to the enhancement layer decoding unit 710.

  In step S1016, enhancement layer decoding section 710 decodes the code data of the lower layer (first enhancement layer) of the decoding target tile input from separation section 704. The enhancement layer decoding unit 710 generates a predicted image by referring to the following. That is, the enlarged image of the upper layer (base layer) input from the enlargement unit 909, the decoded image of the decoded lower layer (first enhancement layer) stored in the frame memory 911, and the lower layer ( The decoded pixel of the first enhancement layer is referred to. Further, enhancement layer decoding section 710 generates a decoded image from the predicted image generated by reference and the decoded prediction error. That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the upper layer (base layer) generated in step S1015. Further, enhancement layer decoding section 710 performs inter-frame prediction with reference to a decoded image of a lower layer (first enhancement layer) stored in frame memory 711. Further, enhancement layer decoding section 710 performs intra prediction with reference to a decoded image in a decoding target tile of a lower layer (first enhancement layer). The decoded image of the tile of the lower layer (first enhancement layer) generated by the enhancement layer decoding unit 710 is output to the frame memory 911 and held in the frame memory 911.

  In step S1017, overall control unit 714 determines whether or not decoding processing of code data of all tiles of the lower layer (first enhancement layer) for the display portion has been completed. Here, enhancement layer decoding section 710 determines that decoding processing of code data of all tiles of the first enhancement layer has been completed, and proceeds to step S1002. In step S1002, overall control section 714 determines whether or not decoding processing has been completed for all layers. Here, enhancement layer decoding section 710 determines that the decoding processing of the second enhancement layer has not ended, and returns to step S1010.

  Hereinafter, decoding of the second enhancement layer is performed. In step S1010, enhancement layer decoding section 710 determines that decoding of the second enhancement layer, which is the layer to be displayed, has not been completed, and proceeds to step S1001. In step S1001, enhancement layer decoding section 710 sets the first enhancement layer decoded in step S1016 as an upper layer and sets the second enhancement layer as a lower layer. In step S1011, separation section 704 extracts code data of a decoding target tile of a lower layer (second enhancement layer) and outputs the code data to enhancement layer decoding section 710. In step S812, the independent tile determination unit 706 determines that the decoding target tile is not a tile of the independent tile set, and proceeds to step S1015. In step S1015, enlargement section 909 inputs the decoded image of the enhancement layer (first enhancement layer) stored in frame memory 908 because the upper layer is the enhancement layer (first enhancement layer). The enlargement unit 909 enlarges the input decoded image of the upper layer (first enhancement layer) by filtering or the like to generate an enlarged image. At this time, an enlarged image may be generated using a tile at a position relatively equal to the position of the decoding target tile and pixels around the tile. Further, the enlargement unit 909 inputs the generated enlarged image to the enhancement layer decoding unit 710.

  In step S1014, enhancement layer decoding section 710 decodes the code data of the lower layer (second enhancement layer) of the decoding target tile input from separation section 704. Enhancement layer decoding section 710 includes an enlarged image of an upper layer (first enhancement layer) input from enlargement section 909, a decoded image of a decoded enhancement layer (second enhancement layer) stored in frame memory 911, and decoding A decoded image is generated with reference to the decoded pixel of the target tile. That is, the enhancement layer decoding unit 710 performs inter-layer prediction with reference to the enlarged image of the upper layer (first enhancement layer) generated in step S1015. Further, enhancement layer decoding section 710 performs inter-frame prediction with reference to a decoded image of a lower layer (first enhancement layer) stored in frame memory 911. Further, enhancement layer decoding section 710 performs intra prediction with reference to the decoded image in the tile to be decoded. The decoded image of the tile of the lower layer (second enhancement layer) decoded by the enhancement layer decoding unit 710 is output to the frame memory 911 and held in the frame memory 911.

  In step S1017, overall control unit 714 determines whether or not code data of all tiles of the lower layer (second enhancement layer) relating to the display portion input from separation unit 704 has been decoded. Here, enhancement layer decoding section 710 determines that decoding of code data of all tiles of the second enhancement layer has been completed, and proceeds to step S1002. In step S1002, the overall control unit 714 determines that decoding of tiles of all layers has been completed, since decoding has been performed up to the second enhancement layer, and proceeds to step S1003. In step S1003, selector 920 selects the decoded image of the lowest layer among the decoded layers. In this case, since the lowest layer is the second enhancement layer, the selector 920 reads the decoded image from the frame memory 911 and outputs the read decoded image to the display unit 606 of FIG. Then, the display unit 606 displays the decoded image of the second enhancement layer output from the image decoding unit 605 when the display control unit 603 is instructed to display the image of the second enhancement layer.

  In the above description, the layer to be displayed has been described as the second enhancement layer (the number of layers is three). However, when the number of layers of the coded data of the layer coding is three or more and the displayed layer is the first enhancement layer (the number of layers is two), decoding of the first enhancement layer is completed (NO in step S1002). Then, in step S1010, the process proceeds to step S1003. For this reason, decoding of the code data of the hierarchy higher than the second enhancement layer is not performed.

  With the above configuration and operation, the relative positions of the independent decoding tiles can be matched in each of the enhancement layer and the base layer. That is, when a predetermined tile is set as an independent tile in the base layer, in each enhancement layer, a tile at a position relatively equal to the position of the independent tile in the base layer can be set as an independent tile. This makes it possible to limit the pixels to be referred to for predicting and decoding the code data of the independent tile in any layer of the hierarchical coding. In particular, in FIG. 6, if the tile instructed to be displayed by the display control unit 603 is an independent tile, necessary code data may be read from the storage unit 602, and the image decoding unit 605 may decode only the code data. For this reason, processing can be performed at a higher speed than before.

  Further, when the MCTS SEI code exists in the bit stream, the tile_boundaries_aligned_flag code of vui_parameters, which is tile position matching information, is always set to 1. That is, in the vui_parameters, when the MCTS SEI code exists in the bit stream, the tile_boundaries_aligned_flag code as the code data can be omitted. If the MCTS SEI code does not exist in the bit stream, the tile_boundaries_aligned_flag code is decoded, and is referred to in the subsequent decoding. If the MCTS SEI code is present in the bit stream, the tile_boundaries_aligned_flag code is not coded, so that a value of 1 is always set on the decoding side. By doing so, it becomes possible to perform decoding similarly without the tile_boundaries_aligned_flag code.

<Embodiment 3>
In the first and second embodiments, the respective processing units shown in FIGS. 1, 4, 6, 7, and 9 have been described as being configured by hardware. However, the processing performed by each processing unit shown in these figures may be executed by a computer program.

  FIG. 11 is a block diagram illustrating a configuration example of hardware of a computer that executes processing performed by each processing unit of the image encoding device and the image decoding device according to the first and second embodiments.

  The CPU 1101 controls the entire computer using the computer programs and data stored in the RAM 1102 and the ROM 1103, and performs the processing described above as being performed by the image encoding device and the image decoding device according to each of the above-described embodiments. Execute That is, the CPU 1101 functions as each processing unit shown in FIGS. 1, 4, 6, 7, and 9.

  The RAM 1102 has an area for temporarily storing computer programs and data loaded from the external storage device 1106, data acquired from the outside via an I / F (interface) 1107, and the like. Further, the RAM 1102 has a work area used when the CPU 1101 executes various processes. That is, the RAM 1102 can be allocated, for example, as a frame memory, or can appropriately provide various other areas.

  The ROM 1103 stores setting data of the computer, a boot program, and the like. The operation unit 1104 is configured by a keyboard, a mouse, and the like, and can input various instructions to the CPU 1101 when the user operates the computer. The output unit 1105 performs control for displaying a processing result by the CPU 1101. Further, the output unit 1105 performs control for displaying a processing result by the CPU 1601 on a display unit (not illustrated) including, for example, a liquid crystal display.

  The external storage device 1106 is a large-capacity information storage device represented by a hard disk drive. The external storage device 1106 stores an operating system (OS) and a computer program for causing the CPU 1101 to realize the functions of the respective units illustrated in FIGS. 1, 4, 6, 7, and 9. Further, each image data to be processed may be stored in the external storage device 1106.

  The computer programs and data stored in the external storage device 1106 are appropriately loaded into the RAM 1102 under the control of the CPU 1101, and are processed by the CPU 1101. A network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 1107, and the computer acquires or sends various information via the I / F 1107. Or you can. A bus 1108 connects the above-described units.

  In the operation in the above-described configuration, the operation described in the above-described flowchart is controlled mainly by the CPU 1101.

<Other embodiments>
In order to easily realize the present invention, it is useful to specify the presence or absence of an independent tile at a level near the head of the bit stream. For example, a method using vui_parameters will be described with reference to FIG. FIG. 12 is a diagram illustrating the syntax of vui_parameters. The vui_parameters includes a motion_constrained_tile_sets_flag code indicating that an independent tile always exists in the bit stream. If the value of this code is 1, it indicates that there is an independent tile in the bitstream including the MCTS SEI, and the relative positions of the tiles of the base layer and the enhancement layer match. That is, since the tile_boundaries_aligned_flag code is always fixed to 1, it is not necessary to encode the tile_boundaries_aligned_flag code. On the other hand, if the tile_boundaries_aligned_flag code is 0, no MCTS SEI is included and no independent tile exists in the bitstream. Therefore, the tile_boundaries_aligned_flag code needs to be encoded. An image decoding device that decodes such a bit stream can obtain information indicating that an independent tile is included in the bit stream before performing decoding processing on each tile. For this reason, when decoding a specific area, the image decoding apparatus can perform high-speed decoding processing using the independent tile. Further, as a result, it is possible to determine whether or not an application such as partial enlarged display is valid before performing decoding processing of each tile.

  Note that the size of the image, the number of divisions of the tile, and the position of the independent tile in one frame are not limited to the above embodiments.

  The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program and reads the program. This is the process to be performed.

Claims (14)

An image encoding apparatus that hierarchically encodes an image forming a moving image in a plurality of layers,
Generating means for generating a second image corresponding to the second layer, which is different in hierarchy from the first image corresponding to the first layer;
A code for encoding a first tile set composed of one or more tiles in the first image and a second tile set composed of one or more tiles in the second image Means,
Information encoding means ;
Flag setting means ,
The second tile set is at a position in the second image corresponding to the first tile set;
The encoding means,
In the first image, the first tile set is encoded without referring to the other than the first tile set, and in the second image, the first tile set is encoded without referring to the other than the second tile set. Encoding the second set of tiles;
The encoding unit refers to only the second tile set in the second image when encoding the first tile set with reference to at least a partial region of the second image. Encoding the first set of tiles with
The information encoding means encodes the SEI message indicating restrictions on decoding of said first set of tiles and the second tile set,
The flag setting means includes:
When the information encoding means encodes the SEI message indicating a restriction on decoding processing of the first tile set and the second tile set,
At least tile_boundaries_aligned_flag indicating that the position of the first tile set in the first image corresponds to the position of the second tile set in the second image is set to 1. Image coding device. The image encoding device according to claim 1, wherein the first image and the second image have different resolutions or image qualities. The first layer is an enhancement layer;
The image encoding device according to claim 1, wherein the second layer is a base layer. The said 1st tile set in the said 1st image, and the said 2nd tile set in the said 2nd image exist in the same position in each image. 2. The image encoding device according to claim 1. The image encoding device according to any one of claims 1 to 4, wherein the SEI message includes at least information indicating a position of the first tile set. An image decoding apparatus that decodes encoded data generated by hierarchically encoding images constituting a moving image in a plurality of layers,
Information decoding means for decoding an SEI message indicating a restriction on decoding processing of a tile set composed of one or a plurality of tiles;
The following SEI message, a first set of tiles in the first image that corresponds to the first layer, the first second tile in the second image that corresponds to the different second layer a layer Decoding means for decoding the set and
When the SEI message is decoded by the information decoding means, the second tile set is located at a position corresponding to the first tile set in the second image;
The decoding unit decodes the first tile set in the first image without referring to the other than the first tile set, and decodes other than the second tile set in the second image. Decoding the second set of tiles without reference;
The decoding means, when the SEI message is decoded by the information decoding means, and when decoding the first tile set with reference to at least a partial area of the second image, In the second image, the first tile set is decoded by restricting to refer to only the second tile set ,
When the information decoding means decodes the SEI message, at least the position of the first tile set in the first image and the position of the second tile set in the second image correspond to each other. An image decoding apparatus characterized in that tile_boundaries_aligned_flag indicating that the image decoding is to be performed is 1 . The image decoding device according to claim 6, wherein the first image and the second image have different resolutions or image qualities. The first layer is an enhancement layer;
The image decoding apparatus according to claim 6, wherein the second layer is a base layer. The said 1st tile set in the said 1st image, and the said 2nd tile set in the said 2nd image exist in the same position in each image. The method of Claim 6 characterized by the above-mentioned. 2. The image decoding device according to claim 1. The image decoding device according to any one of claims 6 to 9, wherein the SEI message includes at least information indicating a position of the first tile set. An image encoding method for hierarchically encoding images constituting a moving image in a plurality of layers,
A generation step of generating a second image corresponding to the second layer, which is different in hierarchy from the first image corresponding to the first layer;
A code for encoding a first tile set composed of one or more tiles in the first image and a second tile set composed of one or more tiles in the second image Process,
An information encoding step ;
A flag setting step ,
The second tile set is at a position in the second image corresponding to the first tile set;
In the encoding step, the first tile set is encoded without referring to anything other than the first tile set in the first image, and the second tile set is encoded in the second image. Encoding the second set of tiles without reference to
In the encoding step, when encoding the first tile set with reference to at least a part of the area of the second image, the second image refers only to the second tile set. Encoding the first set of tiles with
In the information encoding step, a SEI message indicating restrictions on decoding of said first set of tiles and the second tile set encoding,
In the information encoding step, when encoding the SEI message indicating a restriction on decoding processing of the first tile set and the second tile set,
In the flag setting step, at least tile_boundaries_aligned_flag indicating that the position of the first tile set in the first image corresponds to the position of the second tile set in the second image is set to 1. image coding method, characterized by. An image decoding method for decoding encoded data generated by hierarchically encoding images constituting a moving image in a plurality of layers,
An information decoding step of decoding an SEI message indicating a restriction on decoding processing of a tile set including one or a plurality of tiles;
The following SEI message, a first set of tiles in the first image that corresponds to the first layer, the first second tile in the second image that corresponds to the different second layer a layer And a decoding step of decoding the set and
When the SEI message is decoded by the information decoding step , the second tile set is located at a position corresponding to the first tile set in the second image;
In the decoding step, in the first image, the first tile set is decoded without referring to the other than the first tile set, and in the second image, the other than the second tile set is decoded. Decoding the second set of tiles without reference;
In the case where the SEI message has been decoded in the information decoding step , and the first tile set is decoded with reference to at least a partial area of the second image, in the decoding step, In the second image, the first tile set is decoded by restricting to refer to only the second tile set ,
In the case where the SEI message is decoded in the information decoding step, at least the position of the first tile set in the first image and the position of the second tile set in the second image correspond to each other. Image_decoding method, wherein tile_boundaries_aligned_flag indicating that the image decoding is to be performed is 1 .   A program for causing a computer to function as each unit of the image encoding device according to claim 1.   A program for causing a computer to function as each unit of the image decoding device according to claim 6.

Images