JP2014197795A - Dynamic image encoder, dynamic image decoder, method for encoding dynamic image, and method for decoding dynamic image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate encoded data for which a specific tile can be decoded independently on a dynamic image decoder side, irrespective of whether or not other tiles are normally decoded.SOLUTION: When a tile is given, among separated tiles, for which an independent tile flag indicates to the effect that an encoding process is performed without referring to information pertaining to other tiles, the encoding process for the tile is executed independently without referring to the information pertaining to the other tiles. When a tile is given for which the independent tile flag indicates to the effect that the encoding process is performed by referring to the information pertaining to the other tiles, the encode processing for the tile is executed by referring to the information pertaining to the other tiles.

Description

  The present invention relates to an image encoding device and an image encoding method for compressing and transmitting an image, and an image decoding device and an image decoding method for decoding an image from encoded data transmitted by the image encoding device. is there.

MPEG and ITU-TH In an international standard video coding scheme such as 26x, an input video frame is divided into square blocks called macroblocks (MB), coding tree units (CTU), and the like. Prediction processing, orthogonal transformation processing of prediction error signals, quantization processing, entropy coding processing, and the like are performed.
In addition, after processing for all macroblocks is completed and a local decoded image for one screen is created, parameter derivation / entropy encoding processing of the loop filter and filtering to the local decoded image based on the parameter are performed. Perform the process.
As the spatial resolution of the moving image data to be encoded increases, the number of CTUs that perform the above processing increases, and thus the amount of encoding processing and decoding processing increases.

Here, when the moving image content to be encoded has a very high spatial resolution, the user A who needs the entire screen of the moving image content and the moving image content, even if a large amount of processing is applied. Since only a partial area is necessary, consider a situation where there is a user B who does not want to apply a large amount of processing.
For example, when the moving image content to be encoded is a surveillance video, there are a user A who wants to browse the entire video and a user B who wants to browse only the important area of the video (for example, a person / entrance). Think about the situation.

Under these circumstances, most simply, the encoded data of the moving image content of the entire screen required by the user A and the moving image content extracted only from the partial area required by the user B are encoded. A method of creating two types of data and data may be employed.
However, in such a method, it is necessary to perform encoding separately, and it is necessary to transmit different encoded data to the user A and the user B.
In addition, when the area required by the user B cannot be specified at the time of encoding, such a method cannot be adopted. Therefore, a method of decoding only an arbitrary partial region from a single encoded data with a low processing amount is desirable.

However, in the video encoding process as described above, in order to perform context adaptive entropy encoding and intra-frame prediction / inter-frame prediction, in order to correctly decode a certain area of a certain frame, the area is predicted. For this reason, the area that is referred to in order to be decoded must also be correctly decoded.
Unless there is a restriction on the reference for prediction, IDR (Instantaneous Decoding Refresh: pictures subsequent to the IDR picture can be decoded in decoding order) and CRA (Clean Random Access: pictures subsequent to the CRA picture in display order) As long as all the frames after the random access point such as (decodable) are not normally decoded, normal decoding of an arbitrary partial area is not guaranteed. Therefore, even if the area to be decoded is a partial area, it is necessary to decode the entire screen.

In Non-Patent Document 1 below, among the above problems, a structure called a tile is used to solve a dependency relationship between image regions by context adaptive entropy coding or intra-frame prediction / inter-frame prediction. ing. As shown in FIG. 9, the tile is a rectangular area composed of a plurality of CTUs.
The tile has the following characteristics.

(1) The division of an image by tiles is performed in units of CTUs.
(2) The image is divided into tiles in which the width of each row / column is non-uniform. That is, the tiles that are adjacent in the vertical direction have the same coordinate at the left and right ends, and the tiles that are adjacent in the horizontal direction
The coordinates of the upper and lower ends are equal.
(3) Each CTU is processed in the raster scan order closed within the tile. It is also possible to perform slice division within the tile.
(4) In the CTU processed first in the tile, entropy coding can be started from the initial state.
(5) In intra-frame prediction processing, inter-frame prediction processing, and entropy encoding / decoding processing, when referring to the local decoded image and encoding mode of adjacent CTUs in the same frame, adjacent CTUs have different tiles If it belongs, the processing at the edge of the screen can be performed without referring to the adjacent CTU.
(6) In each tile, loop filter processing can be performed independently.
(7) In the inter-frame prediction process, when referring to the pixel of the encoded (decoded) frame,
Reference to a pixel at the CTU position outside the tile is not limited.

As described above, when performing encoding / decoding processing of CTUs belonging to a certain tile, only the results of CTUs belonging to the same tile in intra-frame prediction processing and orthogonal transformation / quantization / entropy encoding processing of prediction error signals. Can be configured as required. Therefore, when considering processing closed within a frame, intra-frame / inter-frame prediction processing, prediction error signal orthogonal transformation / quantization processing, entropy coding processing, loop filter processing can be performed on a single tile or on a specific tile. It becomes possible to process independently in groups.
However, since the pixel access to the position outside the tile is not restricted in the inter-frame prediction process, it is not possible to decode only the tile in the region at the specific position in all frames during the decoding process.

Benjamin Bross, Woo-Jin Han, Jens-Rainer Ohm, Gary J. Sullivan, Thomas Wiegand, “JCTVC-K1003: High Efficiency Video Coding (HEVC) text specification draft 9”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 11th Meeting: Shanghai, CN, 10-19 October 2012

Since the conventional moving image coding apparatus is configured as described above, there is a dependency relationship between image regions such as context adaptive entropy coding and intra-frame prediction / inter-frame prediction. There was a problem that could not be decrypted.
In addition, considering a process in which a moving image is divided into tiles and closed within a frame, intra-frame / inter-frame prediction, orthogonal transformation / quantization of prediction error signal, entropy coding, loop filter, etc. are performed independently. While it is possible to process independently on a tile or a specific group of tiles, inter-frame prediction does not restrict references to pixels present in other tiles. For this reason, the moving image decoding apparatus has a problem that a specific tile cannot be decoded unless other tiles are normally decoded.

The present invention has been made to solve the above-described problems, and a moving picture decoding apparatus independently decodes a specific tile regardless of whether other tiles are normally decoded or not. An object of the present invention is to obtain a moving picture coding apparatus and a moving picture coding method capable of generating encoded data that can be encoded.
It is another object of the present invention to obtain a moving picture decoding apparatus and a moving picture decoding method capable of independently decoding a specific tile regardless of whether or not other tiles are normally decoded. .

  The moving picture coding apparatus according to the present invention provides a flag indicating whether or not to perform coding processing with reference to information on other tiles for each tile which is a rectangular area into which a coding target image is divided. If the flag setting means to set and a tile indicating that the flag set by the flag setting means performs the encoding process without referring to information on other tiles among the separated tiles, If the tile encoding process is executed independently without referring to the information about other tiles, and the flag indicates that the encoding process is performed while referring to the information regarding the other tiles, Tile encoding means for executing the tile encoding processing with reference to information on other tiles, and the multiplexing means for each tile encoded by the tile encoding means. It is obtained so as to generate a bit stream by multiplexing flag for each tile set by Nos data and flag setting means.

  According to the present invention, when a tile indicating that the flag set by the flag setting unit performs the encoding process without referring to information on another tile among the separated tiles is given, the tile If the tile is given to indicate that the encoding process is performed while referring to the information related to the other tiles, the encoding process is executed independently without referring to the information related to the other tiles. Since the tile encoding process is configured to be executed with reference to information on other tiles, the moving picture decoding device side can execute a specific tile regardless of whether or not the other tiles are normally decoded. There is an effect that encoded data that can be decoded independently can be generated.

It is a block diagram which shows the moving image encoder by Embodiment 1 of this invention. It is a block diagram which shows the tile encoding part 5-n (n = 1, 2, ..., N) of the moving image encoder by Embodiment 1 of this invention. It is a block diagram which shows the moving image decoding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the tile decoding part 45-n (n = 1, 2, ..., N) of the moving image decoding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (moving image encoding method) of the moving image encoding device by Embodiment 1 of this invention. It is a flowchart which shows the processing content (moving image decoding method) of the moving image decoding apparatus by Embodiment 1 of this invention. It is a flowchart for the processing contents of the tile encoding unit 5-n (n = 1, 2,..., N). It is a flowchart for the processing contents of the tile decoding unit 45-n (n = 1, 2,..., N). It is explanatory drawing which shows an example of CTU division | segmentation and tile division | segmentation of an input image. It is explanatory drawing which shows the example by which the largest encoding block is divided | segmented hierarchically into several encoding object block. (A) shows the distribution of partitions after division, and (b) is an explanatory diagram showing a situation in which a coding mode m (B ⁿ ) is assigned by hierarchical division in a quadtree graph. It is explanatory drawing which shows the condition which prohibits the reference between frames across tile boundaries. (A) shows the status of inter-tile reference in the first embodiment, (b) shows the status of inter-tile reference in the second embodiment, and (c) shows the status of inter-tile reference in the third embodiment. It is explanatory drawing. FIG. 10 is an explanatory diagram illustrating a partial area decoding process on the video decoding device side in the third embodiment.

Embodiment 1 FIG.
In the first embodiment, each frame image of a video is input, motion compensation prediction or intraframe prediction is performed between adjacent frames, and an orthogonal transformation is performed on a prediction difference signal obtained by the prediction processing. Describes a moving picture coding apparatus that performs variable length coding and generates a bitstream after performing compression processing by quantization, and a moving picture decoding apparatus that decodes a bitstream output from the moving picture coding apparatus To do.
In addition, the bitstream generated by the moving picture encoding apparatus described in the first embodiment is configured to be able to independently decode a tile group at a specific spatial position in a specific continuous frame. In addition to decoding the entire image, the image decoding apparatus can independently decode a tile group at an arbitrary spatial position.

1 is a block diagram showing a moving picture coding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the encoding control unit 1 outputs tile division control information for controlling the tile division state of an input image (encoding target image), and information on other tiles for each tile divided from the input image. An independent tile flag indicating whether or not to perform the encoding process is set with reference, and a process of outputting the independent tile flag is performed. The encoding control unit 1 constitutes a flag setting unit.
Here, the other tiles are tiles other than the current tile in the same frame, and tiles at positions different from the current tiles of the other frames.
In the first embodiment, for convenience of explanation, when encoding processing is performed without referring to information regarding other tiles, independent tile flag = ON, and when encoding processing is performed while referring to information regarding other tiles, independent processing is performed. Assume that the tile flag is set to OFF.

The tile division unit 2 determines the tile division state of the input image in accordance with the tile division control information output from the encoding control unit 1, and performs a process of instructing the tile encoding start unit 3 to perform tile division of the input image.
The tile encoding start unit 3 divides the input image into a plurality of tiles and distributes the divided tiles (tile partial images) to the plurality of tile encoding units 5 under the instruction of the tile dividing unit 2. .
The tile dividing unit 2 and the tile encoding start unit 3 constitute image dividing means.

The tile encoding device 4 includes N tile encoding units 5-1 to 5-N and an intra-frame reference memory 6, and encodes a plurality of tiles distributed by the tile encoding start unit 3. It is.
The tile encoding units 5-1 to 5-N execute, in parallel, the encoding processing of tiles distributed by the tile encoding start unit 3 according to the independent tile flag output from the encoding control unit 1, A process of outputting the tile bit data and the tile local decoded image, which are the encoded data of, to the tile encoding end unit 7 is performed.
That is, among the tiles divided by the tile encoding start unit 3, the tile encoding units 5-1 to 5-N start tile encoding when the independent tile flag set by the encoding control unit 1 is ON. When distributed from the unit 3, the encoding process of the tile is executed independently without referring to information on other tiles, and a tile whose independent tile flag is OFF is distributed from the tile encoding start unit 3. In this case, the encoding process of the tile is executed with reference to information on other tiles. Note that the tile encoding units 5-1 to 5-N constitute tile encoding means.
The intra-frame reference memory 6 stores the local decoded image (local decoded image before the loop filter processing) of the current frame output from the tile encoding units 5-1 to 5-N, information on encoding parameters, motion vectors, and the like. . These pieces of information are appropriately referred to by the tile encoding unit 5 that performs tile encoding processing of the current frame.

In the first embodiment, when the tile encoding units 5-1 to 5-N generate a predicted image of a tile, intra prediction processing (intraframe prediction processing) or inter prediction processing (interframe prediction processing) is performed. If the independent tile flag is ON regardless of which prediction processing is executed, it is independent without referring to information on other tiles when generating a predicted image of the tile. To be executed.
However, the intra prediction process is generally performed without referring to information on other tiles, and it is rather special to perform it while referring to information on other tiles.
For this reason, when intra prediction processing is performed, the same prediction processing is performed regardless of whether the independent tile flag is ON / OFF, and only when the inter prediction processing is performed, ON / OFF of the independent tile flag is supported. You may make it implement the prediction process to perform.

The tile encoding end unit 7 waits until the tile bit data and tile local decoded images of all tiles are output from the tile encoding units 5-1 to 5-N, and the local decoded image for one frame is transmitted between frames. In addition to outputting to the reference memory 9, the tile bit data of each tile is multiplexed in a specific order, and the frame bit data that is the tile bit data after multiplexing is output to the variable-length code multiplexer 8. .
The variable length code multiplexing unit 8 multiplexes the frame bit data output from the tile encoding end unit 7 with the tile division control information output from the encoding control unit 1 and the independent tile flag of each tile to generate a bit stream. Is generated and the bitstream is output.
The tile encoding end unit 7 and the variable length code multiplexing unit 8 constitute multiplexing means.

  The inter-frame reference memory 9 stores a local decoded image for one frame output from the tile encoding end unit 7, and also encodes parameters and motion vectors output from the tile encoding units 5-1 to 5-N. Stores the information. Information on the locally decoded image, the encoding parameter, and the motion vector is held until it is not necessary for the tile encoding units 5-1 to 5-N to refer to the local decoded image.

In the example of FIG. 1, a coding control unit 1, a tile division unit 2, a tile coding start unit 3, a tile coding device 4, a tile coding end unit 7, and a variable length code that are components of a moving image coding device. Although each of the multiplexing units 8 is assumed to be configured by dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, or a one-chip microcomputer), a moving image encoding apparatus is You may be comprised with the computer.
When the moving image encoding device is configured by a computer, the interframe reference memory 9 is configured on the memory of the computer, and the encoding control unit 1, the tile dividing unit 2, the tile encoding start unit 3, and the tile encoding device. 4. A program describing the processing contents of the tile encoding end unit 7 and the variable length code multiplexing unit 8 is stored in the memory of the computer, and the CPU of the computer executes the program stored in the memory. You can do it.
FIG. 5 is a flowchart showing the processing contents (moving image coding method) of the moving image coding apparatus according to Embodiment 1 of the present invention.

FIG. 2 is a block diagram showing tile encoding units 5-n (n = 1, 2,..., N) of the moving picture encoding apparatus according to Embodiment 1 of the present invention.
In FIG. 2, the tile coding control unit 21 determines a coding block size that is the size of a block to be coded, and also selects a block dividing unit from one or more selectable intra coding modes and inter coding modes. The process which determines the encoding mode with the highest encoding efficiency with respect to the encoding object block output from 22 is implemented.
In addition, for example, when the coding mode having the highest coding efficiency is the intra coding mode, the tile coding control unit 21 uses the intra prediction process when performing the intra prediction process on the current block in the intra coding mode. A prediction parameter is determined, and when the coding mode having the highest coding efficiency is the inter coding mode, an inter prediction parameter to be used when performing the inter prediction process on the current block in the inter coding mode is determined. Perform the process.

Further, the tile encoding control unit 21 acquires an independent tile flag from the encoding control unit 1 in FIG. 1 and outputs the independent tile flag to the intra prediction unit 24, the motion compensation prediction unit 25, and the loop filter unit 31. Perform the process.
Further, the tile coding control unit 21 performs a process of determining a prediction differential coding parameter to be given to the transform / quantization unit 27, the inverse quantization / inverse transform unit 28, and the variable length coding unit 30.
Also, a process of outputting the determined coding mode, prediction difference signal parameter, intra prediction parameter, inter prediction parameter, and loop filter parameter (hereinafter collectively referred to as “coding parameter”) to the variable length coding unit 30. To implement.

When the block dividing unit 22 receives the tile (tile partial image data) distributed by the tile encoding start unit 3 in FIG. A process of dividing and outputting an encoding target block which is a block of a prediction processing unit is performed.
If the coding mode determined by the tile coding control unit 21 is the intra coding mode, the changeover switch 23 outputs the block to be coded output from the block dividing unit 22 to the intra prediction unit 24 and performs tile coding. If the coding mode determined by the control unit 21 is the inter coding mode, a process of outputting the coding target block output from the block dividing unit 22 to the motion compensation prediction unit 25 is performed.

The intra prediction unit 24 uses the intra prediction parameters determined by the tile encoding control unit 21 while referring to the locally decoded image stored in the intraframe reference memory 6, and encodes the encoding target output from the changeover switch 23. A process of generating an intra-predicted image by performing an intra-prediction process on the block is performed.
However, the intra prediction unit 24 switches the prediction process according to the independent tile flag output from the tile encoding control unit 21 when the reference destination pixel is a pixel belonging to another tile.
That is, when the independent tile flag is OFF, the intra prediction unit 24 waits until a local decoded image of another tile to be referred to is generated and stored in the intra-frame reference memory 6, When the local decoded image is stored in the intra-frame reference memory 6, intra prediction processing is performed using the reference destination pixel in the local decoded image. On the other hand, when the independent tile flag is ON, intra prediction processing that does not require pixel reference in the locally decoded image of another tile is performed.

The motion compensation prediction unit 25 searches for a motion vector by comparing the encoding target block output from the changeover switch 23 and the locally decoded image after filtering stored in the interframe reference memory 9, and the motion vector and tile Using the inter prediction parameters determined by the encoding control unit 21, an inter prediction process is performed on the encoding target block to generate an inter prediction image.
However, the motion compensation prediction unit 25 switches the prediction process according to the independent tile flag output from the tile coding control unit 21.
That is, when the independent tile flag is OFF, the motion compensation prediction unit 25 generates a prediction image using pixels outside the current tile (inter prediction processing using a motion vector pointing outside the current tile is performed inside the current tile. This is the same processing as the inter prediction processing using the motion vector to be pointed).
On the other hand, when the independent tile flag is ON, the generation of a predicted image using pixels outside the current tile is prohibited. When it is necessary to refer to a pixel outside the current tile in the generation of the predicted image, for example, the pixel at that position is generated by interpolating using the pixel in the current tile of the picture being referred to.

The subtraction unit 26 subtracts the intra prediction image generated by the intra prediction unit 24 or the inter prediction image generated by the motion compensation prediction unit 25 from the encoding target block output from the block division unit 22, A process of outputting a prediction difference signal (difference image) as a subtraction result to the transform / quantization unit 27 is performed.
The transform / quantization unit 27 refers to the prediction difference encoding parameter determined by the tile encoding control unit 21 and performs orthogonal transform processing (for example, DCT (discrete cosine transform)) on the prediction difference signal output from the subtraction unit 26. Or an orthogonal transform process such as a KL transform in which a base design is made in advance for a specific learning sequence) to calculate a transform coefficient and refer to the prediction difference encoding parameter to determine the transform coefficient A process of quantizing and outputting the quantized transform coefficient (hereinafter referred to as “quantized coefficient”) to the inverse quantization / inverse transform unit 28 and the variable length coding unit 30 is performed.

The inverse quantization / inverse transform unit 28 refers to the prediction difference encoding parameter determined by the tile encoding control unit 21 and inversely quantizes the quantized coefficient output from the transform / quantization unit 27, and Processing for calculating a local decoded prediction difference signal corresponding to the prediction difference signal output from the subtracting unit 26 by referring to the prediction difference encoding parameter and performing inverse orthogonal transformation processing on the transform coefficient after inverse quantization To implement.
The addition unit 29 is a difference image indicated by the local decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 28, an intra prediction image generated by the intra prediction unit 24, or a motion compensation prediction unit 25. A process of calculating a local decoded image corresponding to the encoding target block output from the block dividing unit 22 by adding the inter predicted image is performed.

  The variable length encoding unit 30 outputs the quantized coefficients output from the transform / quantization unit 27 and the encoding parameters (encoding mode, intra prediction parameter, inter prediction parameter, prediction) output from the tile encoding control unit 21. The differential encoding parameters and the loop filter parameters) and the motion vector output from the motion compensation prediction unit 25 (when the encoding mode is the inter encoding mode) are subjected to variable length encoding, and tile bit data that is a bit stream The process to generate is performed.

The loop filter unit 31 sequentially performs zero or more types of filtering processing based on the filter parameters output from the tile coding control unit 21.
However, the loop filter unit 31 switches the filtering process according to the independent tile flag output from the tile coding control unit 21.
That is, when the independent tile flag is OFF, the loop filter unit 31 performs a filtering process with reference to pixels outside the current tile. On the other hand, when the independent tile flag is ON, the reference of the pixels outside the current tile is prohibited, and the filtering process is performed by referring only to the pixels in the current tile.
Here, as a filtering process that refers only to the pixels in the current tile at the tile boundary, a method of padding the pixels at the boundary, a method of changing the filter shape, or the like can be considered. The special processing at the tile edge when the independent tile flag is ON needs to be matched between the moving image encoding device and the moving image decoding device.

FIG. 3 is a block diagram showing a moving picture decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 3, when the variable length code separating unit 41 receives the bit stream data output from the variable length code multiplexing unit 8 of the image coding apparatus in FIG. 1, the variable length code separating unit 41 separates the information multiplexed in the bit stream. Thus, the tile division control information multiplexed in the bitstream is output to the tile division unit 42, and the frame bit data multiplexed in the bitstream and the independent tile flag of each tile are output to the tile decoding start unit. The process which outputs to 43 is implemented. In addition, a process of outputting the independent tile flag of each tile to the tile decoding end unit 47 is performed. The variable length code separating unit 41 constitutes a separating unit.

The tile division unit 42 determines the tile division state of the input image (encoding target image) according to the tile division control information output from the variable length code separation unit 41, and combines the divided tiles with the tile decoding end unit 47. The process instructed to is performed.
The tile decoding start unit 43 separates the frame bit data output from the variable length code separation unit 41 into tile bit data of each tile, and the tile bit data of each tile and each tile output from the variable length code separation unit 41 The independent tile flag is distributed to the tile decoding units 45-1 to 45-N.
Further, when the tile decoding start unit 43 distributes the tile bit data and the independent tile flag of each tile to the tile decoding units 45-1 to 45-N, decoding tile designation information indicating the decoding target tile is input from the outside. If the tiles to be decoded indicated by the decoding tile designation information are all tiles, the tile decoding start units 45-1 to 45-45 receive the tile bit data of all the tiles regardless of the ON / OFF of the independent tile flag. Distribute to -N.
On the other hand, when some tiles are specified as the decoding target tiles indicated by the decoding tile specification information, the tile bits of the tiles whose independent tile flag is ON among the tiles specified by the decoding tile specification information Only the data is output to one of the tile decoding units 45 so that the tile bit data of the tile whose independent tile flag is OFF is not output to the tile decoding unit 45.
The tile decoding start unit 43 constitutes data distribution means.

The tile decoding device 44 includes N tile decoding units 45-1 to 45 -N and an intra-frame reference memory 46, and is a device that decodes a plurality of tiles distributed by the tile decoding start unit 43.
In accordance with the independent tile flag distributed by the tile decoding start unit 43, the tile decoding units 45-1 to 45-N execute processing for decoding tiles from tile bit data distributed by the tile decoding start unit 43 in parallel. A process of outputting the decoded image of each tile to the tile decoding end unit 47 is performed.
That is, the tile decoding units 45-1 to 45 -N are tile tile data of tiles separated by the variable length code separation unit 41, for tiles for which the independent tile flag distributed by the tile decoding start unit 43 is ON. When tile bit data is distributed from the tile decoding start unit 43, the process of decoding the tile from the tile bit data is executed independently without referring to information on other tiles, and the independent tile flag is OFF. When tile bit data of a tile is distributed from the tile decoding start unit 43, a process of decoding the tile from the tile bit data is executed with reference to information on other tiles. Note that the tile decoding units 45-1 to 45-N constitute tile decoding means.
The intra-frame reference memory 46 stores the decoded image of the current frame (decoded image before the loop filter process) output from the tile decoding units 45-1 to 45-N, information on the encoding parameter, the motion vector, and the like. These pieces of information are appropriately referred to by the tile decoding unit 45 that performs tile decoding processing of the current frame.

In the first embodiment, it is assumed that the tile decoding units 45-1 to 45-N perform intra prediction processing or inter prediction processing when generating a predicted image of a tile. Even when the independent tile flag is ON, when the predicted image of the tile is generated, it is executed independently without referring to information on other tiles.
However, the intra prediction process is generally performed without referring to information on other tiles, and it is rather special to perform it while referring to information on other tiles.
For this reason, when intra prediction processing is performed, the same prediction processing is performed regardless of whether the independent tile flag is ON / OFF, and only when the inter prediction processing is performed, ON / OFF of the independent tile flag is supported. You may make it implement the prediction process to perform.

The tile decoding end unit 47 waits until the decoded images of all the tiles are output from the tile decoding units 45-1 to 45-N. When the decoded images of all the tiles are output, the tile decoding unit 42 instructs Below, the decoded images of all the tiles are combined to generate a decoded image for one frame, the decoded image is output to the outside, and the decoded image is output to the inter-frame reference memory 48.
The inter-frame reference memory 48 stores the decoded image for one frame output from the tile decoding end unit 47. This decoded image is held until it is not necessary for the tile decoding units 45-1 to 45-N to refer to it.

In the example of FIG. 3, each of the variable-length code separation unit 41, the tile division unit 42, the tile decoding start unit 43, the tile decoding device 44, and the tile decoding end unit 47, which are components of the moving image decoding device, is dedicated hardware. (For example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer or the like) is assumed. However, the moving image decoding apparatus may be configured with a computer.
When the moving picture decoding apparatus is configured by a computer, the interframe reference memory 48 is configured on the memory of the computer, and a variable length code separation unit 41, a tile division unit 42, a tile decoding start unit 43, a tile decoding apparatus 44, and A program describing the processing contents of the tile decoding end unit 47 may be stored in a memory of a computer so that the CPU of the computer executes the program stored in the memory.
FIG. 6 is a flowchart showing the processing contents (moving image decoding method) of the moving image decoding apparatus according to Embodiment 1 of the present invention.

FIG. 4 is a block diagram showing tile decoding units 45-n (n = 1, 2,..., N) of the video decoding apparatus according to Embodiment 1 of the present invention.
4, the variable-length code decoding unit 61 uses the quantized coefficients related to the decoding target block (encoding target block), which is a block of the prediction processing unit, from the tile bit data distributed by the tile decoding start unit 43 in FIG. , Coding mode, intra prediction parameter (when the coding mode is the intra coding mode), inter prediction parameter (when the coding mode is the inter coding mode), prediction differential coding parameter, motion vector (coding) When the mode is the inter coding mode) and variable-length decoding of the loop filter parameter and outputting it.
In addition, the variable length code decoding unit 61 performs a process of outputting the independent tile flag distributed by the tile decoding start unit 43.

  If the coding mode variable-length decoded by the variable-length code decoding unit 61 is the intra-coding mode, the changeover switch 62 receives the intra prediction parameter and the independent tile flag output from the variable-length code decoding unit 61 as the intra prediction unit 63. If the encoding mode variable-length decoded by the variable-length code decoding unit 61 is the inter-coding mode, the inter prediction parameter, the motion vector, and the independent tile flag output from the variable-length code decoding unit 61 are moved. A process of outputting to the compensation unit 64 is performed.

The intra prediction unit 63 performs an intra prediction process on the decoding target block using the intra prediction parameter output from the changeover switch 62 while referring to the decoded image stored in the intra-frame reference memory 46, and performs the intra prediction image. The process to generate is performed.
However, the intra prediction unit 63 switches the prediction process according to the independent tile flag output from the changeover switch 62 when the reference destination pixel is a pixel belonging to another tile.
That is, when the independent tile flag is OFF, the intra prediction unit 63 waits until a decoded image of another tile to be referred to is generated, and the decoded image is stored in the intra-frame reference memory 46. When the image is stored in the intra-frame reference memory 46, intra prediction processing is performed using the reference pixel in the decoded image. On the other hand, when the independent tile flag is ON, intra prediction processing that does not require pixel reference in the decoded image of another tile is performed.

The motion compensation unit 64 uses the motion vector output from the changeover switch 62 and the inter prediction parameter while referring to the decoded image after filtering stored in the interframe reference memory 48, and performs inter prediction processing on the decoding target block. To generate an inter-predicted image.
However, the motion compensation unit 64 switches the prediction process according to the independent tile flag output from the changeover switch 62.
That is, when the independent tile flag is OFF, the motion compensation unit 64 generates a prediction image using pixels outside the current tile (inter prediction processing using a motion vector pointing outside the current tile points inside the current tile. This is the same processing as the inter prediction processing using motion vectors).
On the other hand, when the independent tile flag is ON, the generation of a predicted image using pixels outside the current tile is prohibited. When it is necessary to refer to a pixel outside the current tile in the generation of the predicted image, for example, the pixel at that position is generated by interpolating using the pixel in the current tile of the picture being referred to. The pixel interpolation method needs to use the same method as the method used by the motion compensation prediction unit 25 in FIG.

The inverse quantization / inverse transform unit 65 refers to the prediction difference encoding parameter output from the variable length code decoding unit 61 and inversely quantizes the quantized coefficient output from the variable length code decoding unit 61, With reference to the prediction difference encoding parameter, an inverse orthogonal transform process is performed on the orthogonal transform coefficient after inverse quantization, and a process of calculating a decoded prediction difference signal is performed.
The addition unit 66 includes a difference image indicated by the decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 65, an intra prediction image generated by the intra prediction unit 63, or an inter prediction generated by the motion compensation unit 64. A process of generating a decoded image by adding the image is performed.

The loop filter unit 67 sequentially performs zero or more types of filtering processing based on the filter parameters output from the variable length code decoding unit 61.
However, the loop filter unit 67 switches the filtering process according to the independent tile flag output from the variable length code decoding unit 61.
That is, when the independent tile flag is OFF, the loop filter unit 67 performs the filtering process with reference to pixels outside the current tile. On the other hand, when the independent tile flag is ON, the reference of the pixels outside the current tile is prohibited, and the filtering process is performed by referring only to the pixels in the current tile.
The filtering process that refers to only the pixels in the current tile at the tile boundary needs to use the same technique as that used by the loop filter unit 31 of FIG.

Next, the operation will be described.
1 divides an input image into rectangular areas called tiles, and completely cuts off the dependency between the specified tile and other tiles, so that the specified tile can be specified by the moving image decoding apparatus. It is characterized in that encoded data that can be decoded independently is created.
At the same time, in the decoding process, only the tile designated by the user can be decoded.

The decoding amount of the video signal generally increases the processing amount in proportion to the space / time / luminance level resolution of the input video.
For example, with the evolution of imaging devices, storage devices, and transmission devices, the time, space, and luminance level resolution of video signals tend to increase. For this reason, it is conceivable that the computation resources required for the video decoding device also increase proportionally.
On the other hand, apparatuses for decoding moving image content are also diversified. In addition to dedicated machines and high-end PCs with high calculation performance, there are portable terminals with low calculation performance. The increase in necessary computing resources accompanying the increase in video resolution is a problem particularly in decoding processing in a portable terminal or the like having low computing performance.

Here, of the content captured by the high-resolution imaging device, the information required by the user is limited to a part of the screen area (for example, the area A), and the entire screen is used by using a large amount of computing resources. There may be situations where it is not necessary to decrypt.
As one of the solutions corresponding to such a situation, when encoding a moving image, the region A of the input image is encoded so as not to depend on other regions (code without referring to other regions). Therefore, it is conceivable to construct encoded data that can independently decode only the region A when a moving image is decoded. If this method is adopted, when decoding a moving image, only the encoded data corresponding to the area A is processed, so that only the area A can be normally decoded and faster than decoding the entire screen. Can be decrypted.

Here, in the image encoding process, compression efficiency is increased by utilizing the fact that a general image has a high correlation between an adjacent region and an image feature. Therefore, if the region to be independently decoded is a specific limited region, it is desirable to encode the other regions with reference to the other regions as usual.
In the above example, the encoding of the area A does not depend on other areas, but it is desirable to configure so that the area A can be referred to in the encoding of other areas. By configuring in this way, it is possible to reduce the total amount of information compared to encoding the area A and other areas individually.
In the first embodiment, in order to realize a function required for decoding processing of only a partial region of such an image, the image is divided into rectangular regions called tiles, and each tile depends on other tiles. And a flag that indicates whether or not to refer to other tiles in the encoding / decoding process based on the flag.

The video signal format to be processed by the moving image encoding apparatus of FIG. 1 is a color video signal in an arbitrary color space such as a YUV signal composed of a luminance signal and two color difference signals, or an RGB signal output from a digital image sensor. In addition to the above, it is assumed that the video frame is an arbitrary video signal including a horizontal / vertical two-dimensional digital sample (pixel) sequence such as a monochrome image signal or an infrared image signal.
However, the gradation of each pixel may be 8 bits, or a gradation such as 10 bits or 12 bits.
In the following description, for convenience, unless otherwise specified, it is assumed that the video signal of the input image is a YUV signal, and the two color difference components U and V are subsampled with respect to the luminance component Y 4: 2: 0. The case of handling format signals will be described.
A processing data unit corresponding to each frame of the video signal is referred to as a “picture”.
In the first embodiment, “picture” is described as a video frame signal that is sequentially scanned (progressive scan). However, when the video signal is an interlaced signal, “picture” is a unit constituting a video frame. It may be a field image signal.

First, the processing content of the moving picture coding apparatus will be described.
The encoding control unit 1 outputs tile division control information for controlling the tile division state of the input image to the tile division unit 2 and the variable length code multiplexing unit 8. The tile split state must be the same split for all frames of the input image sequence or the same for a specific range of consecutive frames. In the first embodiment, at least a frame from one random access point to a frame immediately before the next random access point needs to be in the same tile division state.
In addition, the encoding control unit 1 sets an independent tile flag indicating whether or not to perform encoding processing with reference to information about other tiles for each tile divided from the input image, and the independent tile flag Are output to the tile encoding device 4 and the variable length code multiplexing unit 8 (step ST1 in FIG. 5). Note that the value of the independent tile flag of each tile must be the same value in consecutive frames at least from the frame of a certain random access point to immediately before the frame of the next random access point.
In the first embodiment, for convenience of explanation, when encoding processing is performed without referring to information regarding other tiles, independent tile flag = ON, and when encoding processing is performed while referring to information regarding other tiles, independent processing is performed. Assume that the tile flag is set to OFF.

Upon receiving tile division control information from the encoding control unit 1, the tile division unit 2 determines the tile division state of the input image according to the tile division control information, and instructs the tile encoding start unit 3 to perform tile division of the input image. To do.
The tile encoding start unit 3 divides the input image into a plurality of tiles under the instruction of the tile dividing unit 2 and distributes the divided tiles to the plurality of tile encoding units 5.
Here, various representations of the information for controlling the tile division (tile division control information) can be considered. Here, for example, the tile is a maximum unit of CU (Coding Tree Unit: CTU), and the CTU is defined as a minimum unit. Consider specifying tile division based on how many are included.

FIG. 9 is an explanatory diagram illustrating an example of CTU division and tile division of an input image.
As shown in FIG. 9, the input image is divided into CTUs arranged in a grid pattern starting from the upper left of the image. When the height and width of the input image and the length of one side of the CTU are designated, this division is uniquely determined.
Here, since tile division is performed using CTU as a minimum unit, the width and height of each tile can be represented by the number of CTUs included in the horizontal and vertical sides.
Therefore, ColumnIntegerArray [] and RowHeightArray [], which are two integer series, can be used as tile division control information.
ColumnWidthArray [] is an array of values representing the widths of tiles arranged in the horizontal direction in the number of CTUs in order from the right.
On the other hand, RowHeightArray [] is an array in which values representing the heights of tiles arranged in the vertical direction are represented by the number of CTUs in order from the top.
Thus, for example, when specified as ColumnWidthArray [] = {4, 3, 6, 4, 6}, RowHeightArray [] = {3, 3, 4, 5}, tile division as shown in FIG. Is done.

Further, since the CTU division of the input image is uniquely determined from the image size and the length of one side of the CTU, the width of the rightmost column and the height of the lowermost row are not specified, and other rows are not specified. It is possible to uniquely determine from the column information.
Therefore, the information indicating the width of the rightmost column and the height of the lowermost row can be omitted. In this case, ColumnWidthArray [] = {4, 3, 6, 4}, RowHeightArray [] = {3 , 3, 4}.

When the tile encoding units 5-1 to 5 -N receive the divided tile (tile partial image) from the tile encoding start unit 3, the tile encoding units 5-1 to 5 -N according to the independent tile flag of the tile output from the encoding control unit 1. The tile encoding process is executed in parallel, and the tile bit data and the tile local decoded image, which are the encoded data of each tile, are output to the tile encoding end unit 7 (step ST2).
Here, the tile encoding units 5-1 to 5-N execute the encoding process of each tile in parallel, but the encoding process of each tile may be executed serially.
The tile encoding units 5-1 to 5-N repeatedly perform the encoding process until encoding of all tiles is completed (steps ST3 and ST4).
Details of processing contents of the tile encoding units 5-1 to 5-N will be described later.

The tile encoding end unit 7 waits until tile bit data and tile local decoded images of all tiles are output from the tile encoding units 5-1 to 5-N, and the local decoded image for one frame is framed. To the inter-reference memory 9.
Further, the tile encoding end unit 7 multiplexes the tile bit data of each tile in a specific order, and stores the frame bit data that is the tile bit data after the multiplexing in the variable length code multiplexing unit 9.
When the variable length code multiplexing unit 8 receives the frame bit data from the tile encoding end unit 7, the frame bit data, the tile division control information output from the encoding control unit 1, and the independent tile flag of each tile, Are multiplexed to generate a bit stream, and the bit stream is output (step ST5). Note that the tile division state and the value of the independent tile flag of each tile do not change until the next random access point arrives as described above, and therefore it is only necessary to multiplex at each random access point. Of course, it may be multiplexed at a higher frequency, or may be multiplexed at a lower frequency as long as the value does not change.

Here, the processing content of the tile encoding unit 5-n (n = 1, 2,..., N) will be specifically described.
FIG. 7 is a flowchart showing the processing contents of the tile encoding unit 5-n (n = 1, 2,..., N).
The tile coding control unit 21 of the tile coding unit 5-n obtains an independent tile flag from the coding control unit 1 in FIG. Output to the filter unit 31.

The tile encoding control unit 21 divides the image area of the predetermined maximum encoding block (CTU) size into encoding target blocks having the encoding block size hierarchically until reaching the upper limit of the predetermined number of division layers. Then, the encoding mode for each encoding target block is determined (step ST31).
Here, FIG. 10 is an explanatory diagram illustrating an example in which the maximum encoding block is hierarchically divided into a plurality of encoding target blocks.
In FIG. 10, the maximum coding block is a coding target block whose luminance component indicated as “0th layer” has a size of (L ⁰ , M ⁰ ).
An encoding target block is obtained by hierarchically dividing a CTU size block to a predetermined depth separately defined by a quadtree structure.
At the depth n, the encoding target block is an image area of size (L ⁿ , M ⁿ ).
However, L ⁿ and M ⁿ may be the same or different, but FIG. 10 shows the case of L ⁿ = M ⁿ .

Hereinafter, the encoding block size determined by the tile encoding control unit 21 is defined as the size (L ⁿ , M ⁿ ) in the luminance component of the encoding target block.
Since quadtree partitioning is performed, (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2) always holds.
Note that in a color video signal (4: 4: 4 format) in which all color components have the same number of samples, such as RGB signals, the size of all color components is (L ⁿ , M ⁿ ), but 4: 2. : When the 0 format is handled, the encoding block size of the corresponding color difference component is (L ⁿ / 2, M ⁿ / 2).
Later, represents the encoding target block of the n hierarchy B ^n, the encoding modes selectable by the encoding target block B ⁿ as represented by m (B ^n).
In the case of a color video signal composed of a plurality of color components, the encoding mode m (B ⁿ ) may be configured to use an individual mode for each color component, or common to all color components. It may be configured to use a mode. Hereinafter, unless otherwise specified, description will be made assuming that it indicates a coding mode for a luminance component of a coding block of a YUV signal and 4: 2: 0 format.

The coding mode m (B ⁿ ) includes one or more intra coding modes (collectively referred to as “INTRA”), one or more inter coding modes (collectively referred to as “INTER”), The tile encoding control unit 21 has the highest encoding efficiency for the encoding target block ^Bn among all the encoding modes that can be used in the picture or a subset thereof. Select.
Furthermore, the encoding target block ^Bn is divided into one or a plurality of prediction processing units (partitions) by the block dividing unit 22 as shown in FIG.
Hereinafter, a partition belonging to the encoding target block B ⁿ is ^denoted as P _i ⁿ (i is a partition number in the nth layer).
How the partitioning of the encoding target block ^Bn is performed is included as information in the encoding mode m ( ^Bn ).

The partition P _i ⁿ is all subjected to prediction processing according to the encoding mode m (B ⁿ ), but a prediction parameter is selected for each encoding target block B ⁿ or partition P _i ⁿ .
For example, the tile encoding control unit 21 generates a block division state as illustrated in FIG. 11 for the maximum encoding block, and specifies an encoding target block.
The shaded portion in FIG. 11A shows the distribution of the partitions after the division, and FIG. 11B shows the situation where the encoding mode m (B ⁿ ) is assigned by the hierarchical division in a quadtree graph. Yes.
Nodes surrounded by □ in FIG. 11B are nodes (encoding target blocks) to which the encoding mode m (B ⁿ ) is assigned.

The changeover switch 23 is output from the block dividing unit 22 when the coding mode m (B ⁿ ) determined by the tile coding control unit 21 is the intra coding mode (when m (B ⁿ ) ∈INTRA). The encoding target block ^Bn is output to the intra prediction unit 24 (step ST32).
On the other hand, when the encoding mode m (B ⁿ ) determined by the tile encoding control unit 21 is the inter encoding mode (when m (B ⁿ ) ∈INTER), the encoding output from the block dividing unit 22 The target block ^Bn is output to the motion compensation prediction unit 25 (step ST32).

The intra prediction unit 24 is the encoding mode m (B ⁿ ) determined by the tile coding control unit 21 is the intra coding mode (when m (B ⁿ ) ∈INTRA), and the intra-prediction unit 24 performs encoding from the changeover switch 3. When the block B ⁿ is received, the tile encoding control unit 21 determines the intra reference image (local decoded image of the current frame before the loop filter process) stored in the intra-frame reference memory 6 is referred to. by using the intra prediction parameters, to implement the intra prediction process for each partition _P ^{i n} in the encoding target block ^{B n,} it generates an intra prediction image _{P INTRAi} ⁿ (step ST33).
However, the intra prediction unit 24 switches the prediction process according to the independent tile flag output from the tile encoding control unit 21 when the reference destination pixel is a pixel belonging to another tile.

That is, when the independent tile flag is OFF, the intra prediction unit 24 has no restriction on the reference image in generating the prediction image, so that a local decoded image of another tile to be referred to is generated, Wait until the local decoded image is stored in the intra-frame reference memory 6, and when the local decoded image is stored in the intra-frame reference memory 6, intra prediction processing is performed using the reference destination pixel in the local decoded image To do.
On the other hand, when the independent tile flag is ON, intra prediction processing that does not require pixel reference in the locally decoded image of another tile is performed.
For example, in the intra prediction process at the tile end, the adjacent pixel to be referred to may be a pixel in the locally decoded image of the adjacent tile (other tile). If the independent tile flag is ON, the pixel Therefore, the intra prediction process may be switched to another intra prediction process. As another intra prediction process, for example, an intra prediction process at a screen edge that does not need to refer to adjacent pixels, an intra prediction process that does not need to refer to other adjacent pixels, and the like can be considered.
Incidentally, the tile decoding unit 45-n in FIG. 4, it is necessary to generate exactly the same intra prediction image and the intra prediction image _{P INTRAi} ^n, intra prediction parameters used for generating the intra prediction image _{P INTRAi} ⁿ is The data is output from the tile encoding control unit 21 to the variable length encoding unit 30 and multiplexed into tile bit data.

The motion-compensated prediction unit 25 performs encoding from the changeover switch 23 when the encoding mode m (B ⁿ ) determined by the tile encoding control unit 21 is the inter encoding mode (when m (B ⁿ ) ∈INTER). When receiving the current block B ^n, the encoding and each partition P _i ⁿ in the target block B ^n, the motion compensation prediction reference image (other after performing the loop filtering process stored in the reference memory 9 between frames The motion vector and the inter prediction parameter determined by the tile encoding control unit 21, and using each of the partitions P in the encoding target block B ⁿ by carrying out inter-prediction processing for the _i ^n, it generates an inter prediction image _{P INTERi} ⁿ (step ST34).
However, the motion compensation prediction unit 25 switches the prediction process according to the independent tile flag output from the tile encoding control unit 21.

That is, when the independent tile flag is OFF, the motion compensated prediction unit 25 has no restriction on the reference image in generating the predicted image, so even if the pixel indicated by the motion vector is a pixel outside the current tile, A predicted image is generated using the pixel.
On the other hand, when the independent tile flag is ON, there is a restriction on the reference image in generating the predicted image (generation of the predicted image using pixels outside the current tile is prohibited), so as shown in FIG. Even if the motion vector points to a pixel outside the current tile, the pixel is not referred to.
Therefore, when the motion vector indicates a pixel outside the current tile, the motion compensation prediction unit 25 performs pixel interpolation processing using the pixel in the current tile, so that the pixel of the pixel indicated by the motion vector is indicated. Try to calculate the value.
As the pixel interpolation process, for example, a padding process for extending a pixel at the current tile boundary can be considered. The moving image encoding device and the moving image decoding device need to perform equivalent pixel interpolation processing.
Incidentally, the tile decoding unit 45-n in FIG. 4, it is necessary to generate exactly the same inter prediction image and the inter-predicted image _{P INTERi} ^n, the inter prediction parameters used for generating the inter prediction image _{P INTERi} ^n, The data is output from the tile encoding control unit 21 to the variable length encoding unit 30 and multiplexed into tile bit data.

Subtraction portion 26 receives the encoding target block ^{B n} from the block dividing unit 22 from its partition _P ^{i n} in the encoding target block ^{B n,} the intra prediction image _{P INTRAi} ⁿ generated by the intra prediction unit 24, _{Alternatively,} the inter prediction image P _INTERIn ⁿ generated by the motion compensation prediction unit 25 is subtracted, and the prediction difference signal as the subtraction result is output to the transform / quantization unit 27 (step ST35).

When the transform / quantization unit 27 receives the prediction difference signal from the subtraction unit 26, the transform / quantization unit 27 refers to the prediction difference coding parameter determined by the tile coding control unit 21 and performs orthogonal transform processing on the prediction difference signal (for example, DCT (discrete cosine transformation) or orthogonal transformation processing such as KL transformation in which a base design is made in advance for a specific learning sequence is performed, and the transformation coefficient is calculated.
In addition, the transform / quantization unit 27 refers to the prediction difference encoding parameter, quantizes the transform coefficient, and performs inverse quantization / inverse transform unit 28 on the quantized coefficient that is the transform coefficient after quantization. It outputs to the variable length encoding part 30 (step ST36).

When the inverse quantization / inverse transform unit 28 receives the quantized coefficient from the transform / quantization unit 27, the inverse quantization / inverse transform unit 28 refers to the prediction difference encoding parameter determined by the tile encoding control unit 21, and the quantized coefficient Is dequantized.
In addition, the inverse quantization / inverse transform unit 28 refers to the prediction differential encoding parameter and performs inverse orthogonal transform processing (for example, inverse DCT, inverse KL transform) on the transform coefficient after inverse quantization. Then, a local decoded prediction difference signal corresponding to the prediction difference signal output from the subtracting unit 26 is calculated (step ST37).

When the addition unit 29 receives the local decoded prediction difference signal from the inverse quantization / inverse conversion unit 28, the addition unit 29 indicates the difference image indicated by the local decoded prediction difference signal, the intra prediction image P _INTRAi ⁿ generated by the intra prediction unit 24, or, by adding the inter prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 25, a local decoded partition image, or as a collection of the local decoded partition image coded output from the block dividing unit 22 A locally decoded image corresponding to block ^Bn is calculated (step ST38).
The adder 29 stores the locally decoded image in the intra-frame reference memory 6. This local decoded image becomes a reference image signal for intra prediction processing and loop filter processing in the current tile and tiles adjacent to the current tile.

Loop filter unit 31, the process of step ST32~ST38 of all the coding target block ^{B n} is completed (step ST39, ST40), the local decoded image (loop filter of the current tile stored in the reference memory 6 frame Based on the loop filter parameters output from the tile encoding control unit 21, using the local decoded image before processing) and the local decoded image of other tiles (local decoded image before loop filter processing), for the current tile, Zero or more types of loop filter processing are performed, and the local decoded image after the loop filter processing is output to the tile encoding end unit 7 (step ST41).
However, the loop filter unit 31 switches the filtering process in accordance with the independent tile flag output from the tile encoding control unit 21.

That is, when the independent tile flag is OFF, the loop filter unit 31 performs the filtering process with reference to the pixels outside the current tile because the reference image is not limited in performing the loop filter process.
On the other hand, when the independent tile flag is ON, there is a restriction on the reference image in performing the loop filter process (referencing pixels outside the current tile is prohibited), so only the pixels within the current tile are referenced. To implement the filtering process.
For example, when it is necessary to refer to pixels outside the current tile in filtering processing at the edge of the tile, etc., pixel interpolation processing such as padding processing that stretches pixels at the current tile boundary (pixels in the current tile) is performed, If the value of the pixel outside the current tile to be referred to is calculated, the filtering process can be performed with reference to only the pixel within the current tile.
Alternatively, if the reference to the pixels outside the current tile becomes unnecessary by changing the filter shape of the loop filter unit 31, the filtering process can be performed with reference to only the pixels in the current tile.
The special processing of the tile edge when the independent tile flag is ON needs to be matched between the moving image encoding device and the moving image decoding device.

When the loop filter process is completed, the variable length encoding unit 30 outputs the quantized coefficient output from the transform / quantization unit 27, the encoding mode m (B ⁿ ) output from the tile encoding control unit 21, Prediction differential encoding parameter and loop filter parameter, intra prediction parameter (when encoding mode is intra encoding mode) output from tile encoding control unit 21 or inter prediction parameter (encoding mode is inter encoding mode) And the motion vector output from the motion compensation prediction unit 25 (when the encoding mode is the inter encoding mode) are variable-length encoded, and tile bit data indicating the encoding result is generated. (Step ST42).

As a variable length coding method, for example, when context adaptive arithmetic coding is used, the internal state of the variable length coding unit 30 is updated as the coding progresses, but the independent tile flag is ON. In this case, the initial value of the internal state is a predetermined fixed value or a value output from the tile encoding start unit 3 in FIG. 1 so as not to take over the state between tiles. .
Further, when encoding encoding parameters (encoding mode m (B ⁿ ), prediction differential encoding parameter, intra prediction parameter / inter prediction parameter, loop filter parameter), they are stored in the intraframe reference memory 6. Refer to the encoding parameters and motion vector information of adjacent blocks that are present, and the encoding parameters and motion vector information of the same position blocks of temporally adjacent frames stored in the interframe reference memory 9 A process for predicting the value is performed.
In this prediction process, when the independent tile flag is ON, the encoding parameter / motion vector of a block belonging to a tile at a position different from the current tile is not referred to, and a specific prediction process that does not require reference is performed.

In the encoding process of the tile encoding unit 5-n, when the independent tile flag is ON, in any process including the intra prediction process, the motion compensation prediction process, and the variable length code process, a tile different from the current tile is used. The encoding parameter, the locally decoded image, and the motion compensated prediction reference image of the block to which the block belongs are not referred to.
The reference process at the tile end is replaced with a process that does not require any reference. The type of processing that needs to be replaced needs to be matched between the video encoding device and the video decoding device, but the same processing may be fixedly performed by the video encoding device and the video decoding device. A signal indicating the processing content may be signaled.
As described above, the processing of the tile encoding unit 5-n in which the independent tile flag is ON is independent for each tile, and can be encoded independently.

Next, processing contents of the video decoding device will be described.
When the variable length code separating unit 41 receives the bit stream data output from the variable length code multiplexing unit 8 of the image encoding device of FIG. The tile division control information multiplexed in the bit stream is output to the tile division unit 42, and the frame bit data multiplexed in the bit stream and the independent tile flag of each tile are output to the tile decoding start unit 43. To do. Further, the independent tile flag of each tile is output to the tile decoding end unit 47 (step ST21 in FIG. 6).
Upon receiving the tile division control information from the variable length code separation unit 41, the tile division unit 42 determines the tile division state of the input image according to the tile division control information in the same manner as the tile division unit 2 in FIG. The tile decoding end unit 47 is instructed to combine the divided tiles (step ST22).

Upon receiving the frame bit data and the independent tile flag of each tile from the variable length code separation unit 41, the tile decoding start unit 43 separates the frame bit data into tile bit data of each tile, and tile bit data of each tile. And the independent tile flag are distributed to the tile decoding units 45-1 to 45-N.
When the tile decoding start unit 43 distributes the tile bit data of each tile and the independent tile flag to the tile decoding units 45-1 to 45-N, when decoding tile designation information indicating a decoding target tile is input from the outside If the decoding target tiles indicated by the decoding tile designation information are all tiles, the tile bit data of all tiles are distributed to 45-1 to 45-N regardless of ON / OFF of the independent tile flag ( Step ST23).
On the other hand, when some tiles are specified as the decoding target tiles indicated by the decoding tile specification information, the tile bits of the tiles whose independent tile flag is ON among the tiles specified by the decoding tile specification information Only the data is output to any tile decoding unit 45 (step ST23).
If a tile whose independent tile flag is OFF is included in the tiles specified by the decoding tile specification information, the tile bit data of the tile is not output to the tile decoding unit 45 and the tile is decoded. Skip processing. Alternatively, predetermined error processing is performed (step ST23).

When receiving the tile bit data and the independent tile flag from the tile decoding start unit 43, the tile decoding units 45-1 to 45-N execute in parallel the process of decoding the tile from the tile bit data according to the independent tile flag. Then, the decoded image of each tile is output to the tile decoding end unit 47 (step ST24).
The details of the processing contents of the tile decoding units 45-1 to 45 -N will be described later. Among the tile bit data of each tile separated by the variable length code separation unit 41, the tile bit of the tile whose independent tile flag is ON. When the data is distributed from the tile decoding start unit 43, the process of decoding the tile from the tile bit data is executed independently without referring to information on other tiles.
On the other hand, when tile bit data of a tile whose independent tile flag is OFF is distributed from the tile decoding start unit 43, a process of decoding the tile from the tile bit data is executed with reference to information on other tiles. .

In the moving picture decoding apparatus of FIG. 3, the size of the inter-frame reference memory 48 and the intra-frame reference memory 46 only needs to be prepared for the image area specified in the decoding tile, and the area of the decoding specified tile is the entire image. When the area is small, memory resources required for decoding can be reduced.
Here, the tile decoding units 45-1 to 45-N execute the decoding process of each tile in parallel, but the decoding process of each tile may be executed serially.
The tile decoding units 45-1 to 45-N repeatedly perform the decoding process until decoding of all the tiles to be decoded is completed (steps ST25 and ST26).

If the decoding target tiles indicated by the decoding tile designation information are all tiles, the tile decoding end unit 47 waits until the decoded images of all tiles are output from the tile decoding units 45-1 to 45-N. When the decoded images of all tiles are output, under the instruction of the tile dividing unit 42, the decoded images of all the tiles are combined to generate a decoded image for one frame, and the decoded image is output to the outside. The decoded image is stored in the inter-frame reference memory 48.
On the other hand, when the decoding target tiles indicated by the decoding tile designation information are some tiles, all of the decoded images of the tiles designated by the decoding tile designation information and the independent tile flag is ON When the decoded image of the tile is output, the designated area decoded image is generated by combining the decoded images of the tiles under the instruction of the tile dividing unit 42. The decoded image is output to the outside, and the designated region decoded image is stored in the interframe reference memory 48.

Here, the processing content of the tile decoding unit 45-n (n = 1, 2,..., N) will be specifically described.
FIG. 8 is a flowchart showing the processing contents of the tile decoding unit 45-n (n = 1, 2,..., N).
When the variable-length code decoding unit 61 receives tile bit data from the tile decoding start unit 43 of FIG. 3, the variable length code decoding unit 61 uses the same method as the tile coding control unit 21 of FIG. To decide. When information indicating the maximum encoding block size and the upper limit of the number of division layers is multiplexed in the tile bit data, the upper limit of the maximum encoding block size and the number of division hierarchies is decoded by decoding the information. May be determined.
Next, the variable length code decoding unit 61 decodes the encoding mode assigned to the maximum encoding block multiplexed in the tile bit data, and determines the maximum encoding block included in the encoding mode. Information indicating the division state (tile division control information) is decoded.

When the variable length code decoding unit 61 decodes the information indicating the division state of the maximum coding block, the variable length code decoding unit 61 hierarchically divides the decoding target block (the tile coding of FIG. 2) based on the division state of the maximum coding block. The block corresponding to the “encoding target block” of the unit 5-n is specified (step ST51).
The variable-length code decoding unit 61 further divides the decoding target block into one or more prediction processing units based on the division state of the decoding target block, and assigns the decoding target block to the encoding target block unit or the prediction processing unit. The encoded parameters are decoded (step ST51).
When the encoding mode (the encoding mode is included in the encoding parameter) assigned to the decoding target block (the encoding target block) is the intra encoding mode, the variable length code decoding unit 61 performs the decoding thereof. The intra prediction parameter is decoded for each of one or more partitions included in the target block.
On the other hand, when the encoding mode assigned to the decoding target block is the inter encoding mode, the inter prediction parameter is decoded for each decoding target block or for each of one or more partitions included in the decoding target block. (Step ST51).

Furthermore, the variable-length code decoding unit 61 uses one or more partitions that are prediction processing units as transform processing units based on information on transform block sizes in the prediction difference coding parameters included in the coding parameters. Then, the quantized coefficients are decoded for each partition as a conversion processing unit (step ST51).
The variable length code decoding unit 61 decodes the filter parameter multiplexed on the tile bit data, and outputs the filter parameter to the loop filter unit 67 (step ST51).
The variable-length code decoding unit 61 outputs the independent tile flag distributed by the tile decoding start unit 43 to the changeover switch 62 and the loop filter unit 67.

In addition, when the tile bit data output from the tile decoding start unit 43 is encoded using an encoding method in which the internal state changes depending on the past block encoding results, such as a context adaptive arithmetic code, the variable length The initial value of the internal state of the code decoding unit 61 uses the same value as that of the variable length coding unit 30 in FIG. Therefore, when the independent tile flag of the current tile is ON, the state is not inherited from other tiles.
In addition, when decoding the encoding parameter, a process for predicting the parameter value is performed by referring to the encoding parameter of the adjacent block or the same position block of the temporally adjacent frame. 12 is ON, as shown in FIG. 12, a specific prediction process that does not require parameter reference is performed without referring to the parameter of a block belonging to a tile at a position different from the current tile (the variable-length encoding in FIG. 2). The same prediction process as that of the unit 30 is performed).

If the encoding mode m (B ⁿ ) variable-length decoded by the variable-length code decoding unit 61 is an intra encoding mode (when m (B ⁿ ) ∈INTRA), the changeover switch 62 is a variable-length code decoding unit. The intra prediction parameter and the independent tile flag variable-length decoded by 61 are output to the intra prediction unit 63 (step ST52).
On the other hand, if the encoding mode m (B ⁿ ) subjected to variable length decoding by the variable length code decoding unit 61 is an inter encoding mode (in the case of m (B ⁿ ) ∈INTER), the variable length code decoding unit 61 can change the encoding mode m (B ⁿ ). The long-decoded inter prediction parameter, motion vector, and independent tile flag are output to the motion compensation unit 64 (step ST52).

In the intra prediction unit 63, the encoding mode m (B ⁿ ) that has been subjected to variable length decoding by the variable length code decoding unit 64 is an intra encoding mode (when m (B ⁿ ) ∈INTRA), When receiving the prediction parameter and the independent tile flag, the intra reference image stored in the intraframe reference memory 46 (the decoded image of the current frame before the loop filter process is performed) is performed in the same procedure as the intra prediction unit 24 in FIG. with reference to) the intra using the prediction parameters, to implement the intra prediction process for each partition _P ^{i n} the decoding target block ^{B n,} it generates an intra prediction image _{P INTRAi} ⁿ (step ST53).
However, the intra prediction unit 63 switches the prediction process according to the independent tile flag output from the changeover switch 62 when the reference destination pixel is a pixel belonging to another tile.

That is, when the independent tile flag is OFF, the intra prediction unit 63 generates a decoded image of another tile to be referred to as in the case of the intra prediction unit 24 in FIG. The process waits until the decoded image is stored in the memory 46. When the decoded image is stored in the intra-frame reference memory 46, intra prediction processing is performed using the reference pixel in the decoded image.
On the other hand, when the independent tile flag is ON, an intra prediction process that does not need to refer to pixels in the decoded image of another tile is performed in the same manner as the intra prediction unit 24 in FIG.
For example, in the intra prediction process at the tile end, the adjacent pixel to be referred to may be a pixel in the decoded image of the adjacent tile (other tile), but if the independent tile flag is ON, the pixel is Since it cannot be referred to, the intra prediction process may be switched to another intra prediction process. As another intra prediction process, for example, an intra prediction process at a screen edge that does not need to refer to adjacent pixels, an intra prediction process that does not need to refer to other adjacent pixels, and the like can be considered.

In the motion compensation unit 64, the encoding mode m (B ⁿ ) variable-length decoded by the variable-length code decoding unit 31 is the inter encoding mode (when m (B ⁿ ) ∈INTER), and the changeover switch 62 receives the inter When the prediction parameter, the motion vector, and the independent tile flag are received, the decoding target block B ^{n is referred} to using the motion vector and the inter prediction parameter while referring to the decoded image after the filtering process stored in the inter-frame reference memory 48. or partition _P ⁱ to implement the inter-prediction process for ⁿ generates an inter prediction image _{P INTERi} ⁿ (step ST54).
However, the motion compensation unit 64, for all the partitions P _i ⁿ the decoding target block B ^n, if the common reference picture indication index and the common inter prediction mode is determined, its reference picture indication index a common reference picture in all the partitions _P ^{i n} that indicates, by using the motion vector for each partition _P ^{i n,} to implement intra prediction processing for the partitions _P ^{i n} to generate an intra prediction image _{P INTERi} ^n.

Note that the motion compensation unit 64 switches the prediction process according to the independent tile flag output from the changeover switch 62 in the same manner as the motion compensation prediction unit 25 in FIG. .
That is, when the independent tile flag is OFF, the motion compensator 64 has no restriction on the reference image in generating the predicted image. Therefore, even if the pixel indicated by the motion vector is a pixel outside the current tile, A predicted image is generated using pixels.
On the other hand, when the independent tile flag is ON, there is a restriction on the reference image in generating the predicted image (generation of the predicted image using pixels outside the current tile is prohibited), so as shown in FIG. Even if the motion vector points to a pixel outside the current tile, the pixel is not referred to.
Therefore, when the motion vector indicates a pixel outside the current tile, the motion compensation unit 64 performs pixel interpolation processing using the pixel in the current tile, so that the value of the pixel indicated by the motion vector is indicated. Is calculated.
As the pixel interpolation process, for example, a padding process for extending a pixel at the current tile boundary can be considered.

  When the inverse quantization / inverse transform unit 65 receives the quantized coefficient and the prediction differential encoding parameter from the variable length code decoding unit 61, the inverse quantization / inverse transform unit 65 performs the prediction in the same procedure as the inverse quantization / inverse transform unit 28 in FIG. With reference to the differential encoding parameter, the quantized coefficient is dequantized, and with reference to the prediction differential encoding parameter, an inverse orthogonal transform process is performed on the dequantized transform coefficient. 2 calculates a decoded prediction difference signal corresponding to the prediction difference signal output from the subtracting unit 26 (step ST55).

Addition unit 66, a differential image indicated by the decoded prediction difference signal calculated by the inverse quantization and inverse transform unit 65, or the intra prediction image P _{INTRAi ^n,} generated by the intra prediction unit 63, generated by the motion compensation section 64 It has been by adding the inter prediction image P _INTERi ^n, as a collection of one or more of the decoded partition image included in the decoding target block, and stores the decoded image in the frame in the reference memory 46 (step ST56).
This decoded image becomes a reference source image signal for intra prediction processing and loop filter processing.

Loop filter unit 67, the process of step ST51~ST56 of all the coding target block ^{B n} is completed (step ST57, ST58), the decoded image of the current tile and the other tiles stored in the reference memory 46 frame (Decoded image before loop filter processing) is used to perform zero or more types of loop filter processing on the current tile based on the filter parameters output from the variable-length code decoding unit 61, and after the loop filter processing The decoded image is output to the inter-frame reference memory 48 and the tile decoding end unit 47 (step ST59).
However, similarly to the loop filter unit 31 of FIG. 2, the loop filter unit 31 switches the filtering process according to the independent tile flag output from the variable length code decoding unit 61.

That is, when the independent tile flag is OFF, the loop filter unit 67 performs the filtering process with reference to the pixels outside the current tile because the reference image is not limited in performing the loop filter process.
On the other hand, when the independent tile flag is ON, there is a restriction on the reference image in performing the loop filter process (referencing pixels outside the current tile is prohibited), so only the pixels within the current tile are referenced. To implement the filtering process.
For example, when it is necessary to refer to pixels outside the current tile in filtering processing at the edge of the tile, etc., pixel interpolation processing such as padding processing that stretches pixels at the current tile boundary (pixels in the current tile) is performed, If the value of the pixel outside the current tile to be referred to is calculated, the filtering process can be performed with reference to only the pixel within the current tile.
Alternatively, if the filter shape of the loop filter unit 67 is changed to make it unnecessary to refer to pixels outside the current tile, the filtering process can be performed with reference to only the pixels within the current tile.
The special processing of the tile edge when the independent tile flag is ON needs to be matched between the moving image encoding device and the moving image decoding device.

In the decoding process of the tile decoding unit 45-n, when the independent tile flag is ON, in any process including the intra prediction process, the inter prediction process, and the variable length decoding process, a block belonging to a tile different from the current tile is used. The encoding parameter, the decoded image, and the motion compensated prediction reference image are not referred to.
The reference process at the tile end is replaced with a process that does not need to be referenced as in the tile encoding unit 5-n in FIG. The type of processing that needs to be replaced needs to be matched between the video encoding device and the video decoding device, but the same processing may be fixedly performed by the video encoding device and the video decoding device. A signal indicating the processing content may be signaled.
As described above, the processing of the tile decoding unit 45-n in which the independent tile flag is ON is independent for the tile and can be decoded independently.

  As is apparent from the above, according to the first embodiment, in the moving picture coding apparatus, the independent tile flag set by the coding control unit 1 among the separated tiles is framed with information on other tiles. When a tile indicating that encoding processing is performed without reference within or between frames is given, the encoding processing of the tile is performed independently without referring to information about other tiles within or between frames. Executed, if a tile indicating that the independent tile flag is to be encoded while referring to information on other tiles within / between frames is given to the tile Since the video decoding device side is configured to execute information while referring to the information on the intraframe / between frames, the continuous frame after the random access point is In an effect that can generate encoded data capable of independently decoding regardless tiles at a specific position, of whether the other tiles are decoded successfully.

  On the other hand, in the video decoding device, the tile processing of the tiles separated by the variable-length code separation unit 41 is performed by the independent tile flag without referring to information on other tiles within / between frames. When tile bit data of a tile indicating to be performed is given, a process of decoding the tile from the tile bit data is independently executed without referring to information on other tiles within / between frames, When tile bit data of a tile indicating that encoding processing is performed while an independent tile flag refers to information on another tile within / between frames, the tile is decoded from the tile bit data. Since the process is configured to be executed while referring to information on other tiles within / between frames, In successive frames of random access point later, an effect that can tile at a particular position, independently decoding regardless of whether the other tiles are decoded successfully.

That is, according to the first embodiment, when the bit stream generated by the video encoding device is given to the video decoding device, the video decoding device can decode the entire screen, It is also possible to independently decode only tiles for which the independent tile flag is ON.
If only a part of tiles is always independently decoded and the entire screen is not decoded, it is possible to reduce resources such as memory and CPU required for decoding the entire screen in the video decoding device. It is.

Embodiment 2. FIG.
In the moving picture coding apparatus of the first embodiment, there are two types of tiles, that is, an independent tile flag = OFF tile and an independent tile flag = ON tile, and the independent tile flag = OFF tile is restricted in interframe reference. There is no tile, and the independent tile flag = ON tile is configured so that reference to pixels outside the current tile and encoding parameters is prohibited in inter-frame reference. By configuring the encoding device in this way, the moving image decoding device can decode only tiles with independent tile flag = ON without depending on other tiles in consecutive frames between random access points. Streams that can be created can be created.
However, a tile with the independent tile flag = ON has a problem in that it has a great influence on the coding efficiency because the restriction on inter-frame reference is large.
The second embodiment is similar to the first embodiment in that the tiles are divided and the reference positions in the inter-frame reference are constrained by the tiles in the same way as in the first embodiment. A decoding apparatus will be described.

FIG. 13B shows an outline of reference destination restrictions in interframe reference in the second embodiment.
First, in the second embodiment, a “tile hierarchy index” is set for each tile instead of the independent tile flag. The “tile level index” is information indicating the decoding priority of tiles. FIG. 13B shows an example in which the priority is higher as the index value is smaller.
Here, as shown in the lower part of FIG. 13B, in each tile, when referring between frames, it is prohibited to refer to a pixel at a tile position having an index larger than the tile hierarchy index of the current tile and an encoding parameter. However, the pixel and the encoding parameter of the tile position having the same index as that of itself or a small index and the encoding parameter can be freely referred to. In addition, the tile hierarchy index of each tile is set to the same value in consecutive frames between random access points, and is configured to be multiplexed in the bitstream instead of the independent tile flag.

When decoding the bitstream created by the moving image coding apparatus configured as described above, a specific image position is determined in successive frames after the random access point based on the tile hierarchy index information of each tile. Only tiles can be normally decoded without decoding other tiles.
Other configurations are in accordance with the moving picture coding apparatus shown in FIGS. 1 and 2 and the moving picture decoding apparatus shown in FIGS.

The effect of the invention in the second embodiment will be described.
In the example of FIG. 13B, the tile with the highest decoding priority, the tile hierarchy index = 0, can be correctly decoded by decoding only the tile with the tile hierarchy index = 0.
The tile with the tile hierarchy index = 3 can be correctly decoded by decoding only the tiles with the tile hierarchy index = 0, 1, 2, 3.
Therefore, as shown in the example of FIG. 13B, for example, there is important information in the center of the screen, and information on the periphery of the screen is to be displayed / hidden according to the platform. By configuring the bitstream so that the tile hierarchy index is gradually reduced in the region of higher importance, it is possible to perform decoding of only a specific tile in several variations.
In addition, since the restriction of inter-tile reference is less than that of the moving picture encoding apparatus shown in the first embodiment, in a case where the tile position to be decoded independently is determined to some extent, etc., the deterioration of the encoding efficiency is suppressed. There is an effect that a bit stream capable of decoding only a specific tile can be generated.

Embodiment 3 FIG.
As described above, the moving picture coding apparatus according to the first embodiment has a problem that the coding efficiency is reduced. In the moving picture coding apparatus according to the second embodiment, the reduction in coding efficiency can be suppressed to some extent. However, there is a problem that the degree of freedom of tile positions that can be decoded independently is low.
In the third embodiment, a video encoding apparatus that can suppress a decrease in encoding efficiency to some extent and can select a tile that can be independently decoded with a high degree of freedom will be described.
FIG. 13C shows an outline of reference destination restrictions in interframe reference in the third embodiment.

First, in the third embodiment, “inter-tile reference type information” is set for each tile instead of the independent tile flag of the first embodiment. The “inter-tile reference type information” is information that defines reference restrictions across tile boundaries in the encoding process and the decoding process. In the third embodiment, there are the following three inter-tile reference types.
(1) Non-independent tile (D in FIG. 13C): Tile with no restriction on reference between tiles Information on any tile position can be referred to in intra-frame / inter-frame reference.
However, this does not apply if intra-frame / inter-frame references are prohibited in other flags. Equivalent to independent tile flag = OFF tile in the first embodiment.
(2) Independent tile (I in FIG. 13C): a tile in which all references across the tile boundary are prohibited. This is equivalent to the independent tile flag = ON tile in the first embodiment.
(3) Semi-independent tile (S in FIG. 13C): a tile in which only an adjacent independent tile can be referred to. Even if an independent tile is not adjacent to the current tile, reference is prohibited.

The inter-tile reference type information of each tile is configured to be set to the same value in consecutive frames between random access points and multiplexed into a bitstream instead of the independent tile flag of the first embodiment.
The non-independent tile is an inter-tile reference type prepared for the case where the function of independently decoding a specific tile is not used. Hereinafter, effects of the independent tile and the semi-independent tile will be described.

FIG. 14A shows an arrangement example of independent tiles and semi-independent tiles.
The checkered pattern is used, or the arrangement is such that the independent tile and the quasi-independent tile are switched for each tile row. In this way, it is desirable that the quasi-independent tile is adjacent to the independent tile.
When arranged in this way, since all quasi-independent tiles can refer to adjacent independent tiles within a frame / between frames, a reduction in encoding efficiency can be suppressed.

  When a moving picture encoding apparatus configured as described above decodes a bitstream encoded by specifying only independent tiles and semi-independent tiles as inter-tile reference type information as shown in FIG. 14A. As shown in FIG. 14B, when a display target area is designated, if a region including up to a tile adjacent to the display target region is decoded as a decoding target region, tiles outside the decoding target region can be decoded. At least, the display target area is correctly decoded independently. This is because independent tiles can be normally decoded without decoding other tiles, and quasi-independent tiles refer only to adjacent independent tiles (in which normal decoding is guaranteed). Note that quasi-independent tiles that are included in the decoding target area but not included in the display target area are not guaranteed to be correctly decoded.

  As described above, the moving picture coding apparatus according to the third embodiment performs coding with higher coding efficiency than when coding using only independent tiles by encoding using independent tiles and semi-independent tiles. The effect which enables it to do is produced. When decoding the bitstream created by the moving image encoding apparatus, the moving image decoding apparatus according to the third embodiment assigns arbitrary tiles, tiles adjacent to them, in consecutive frames after the random access point. Decoding only this produces the effect of enabling normal decoding.

In addition to the configurations described in the first, second, and third embodiments, the moving image encoding apparatus according to the present invention is not limited to the “tile” in which the encoding control means indicates the inter-tile reference constraint condition for each tile. "Inter-reference constraint information" is determined independently or in common, and the tile encoding means performs encoding while prohibiting intra-frame / inter-frame reference to the tile specified by the inter-tile reference constraint information of the current tile, and is variable length The code multiplexing means can be configured to have a function of multiplexing the inter-tile reference constraint information determined for each tile into a bit stream and transmitting the multiplexed information to the video decoding device. In addition, hint information effective for the process of decoding a specific tile in the moving image decoding apparatus can be configured to be multiplexed into a bitstream as necessary.
In addition to the configurations described in the first, second, and third embodiments, the moving picture decoding apparatus according to the present invention has “inter-tile reference constraint information” multiplexed in a bitstream, and a user can Based on the designated decoding tile designation information, only a specific tile in a continuous frame can be configured to be decoded independently and normally even if other tiles are not decoded. At that time, the hint information multiplexed in the stream can be used as necessary.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Coding control part (flag setting means), 2 Tile division part (Image division means), 3 Tile encoding start part (Image division means), 4 Tile coding apparatus, 5-1 to 5-N Tile coding part (Tile coding means), 6 frame reference memory, 7 tile coding end section (multiplexing means), 8 variable length code multiplexing section (multiplexing means), 9 interframe reference memory, 21 tile coding control section , 22 block division unit, 23 changeover switch, 24 intra prediction unit, 25 motion compensation prediction unit, 26 subtraction unit, 27 transform / quantization unit, 28 inverse quantization / inverse transform unit, 29 addition unit, 30 variable length coding Unit, 31 loop filter unit, 41 variable length code separation unit (separation unit), 42 tile division unit, 43 tile decoding start unit (data distribution unit), 44 tile decoding device, 45-1 to 45- Tile decoding unit (tile decoding means), 46 intra-frame reference memory, 47 tile decoding end unit, 48 inter-frame reference memory, 61 variable length code decoding unit, 62 changeover switch, 63 intra prediction unit, 64 motion compensation unit, 65 inverse Quantization / inverse transform unit, 66 adder unit, 67 loop filter unit.

Claims (11)

Flag setting means for setting a flag indicating whether or not to perform encoding processing with reference to information on other tiles for each tile that is a rectangular area into which an image to be encoded is divided;
Among the separated tiles, when a tile that indicates that the flag set by the flag setting unit indicates that the encoding process is performed without referring to information on other tiles is performed, the tile encoding process is performed. If the tile is executed independently without referring to the information regarding other tiles and the flag indicates that the encoding processing is performed while referring to the information regarding the other tiles, the encoding processing of the tile is performed. Tile encoding means for performing the steps with reference to information on other tiles;
A video encoding device comprising: multiplexing means for multiplexing the encoded data of each tile encoded by the tile encoding means and the flag of each tile set by the flag setting means to generate a bitstream . While implementing multiple tile encoding means,
An image dividing unit that divides an image to be encoded into a plurality of tiles and distributes the divided tiles to the plurality of tile encoding units;
2. The moving image according to claim 1, wherein the plurality of tile encoding units execute, in parallel, encoding processing of tiles distributed by the image dividing unit in accordance with the flag set by the flag setting unit. Encoding device.   The tile encoding unit is present in another tile when the flag set by the flag setting unit is given a tile indicating that the encoding process is performed without referring to information on the other tile. 3. The moving picture coding apparatus according to claim 1, wherein a predicted image of the tile is generated without referring to a pixel, and a difference image between the tile and the predicted image is quantized and coded.   The tile encoding means inversely quantizes the difference image after quantization, generates a local decoded image of the tile from the difference image after dequantization and the prediction image, and loop filter processing that reduces distortion of the local decoded image 4. The moving picture coding apparatus according to claim 3, wherein the loop filter process is performed without referring to a locally decoded image of another tile. Separating means for separating encoded data and flags of each tile multiplexed in the bitstream;
Of the encoded data of each tile separated by the separating means, when the encoded data of a tile indicating that the flag is to be encoded without referring to information on other tiles is given, The process of decoding the tile from the encoded data is independently executed without referring to the information regarding the other tile, and the flag indicates that the encoding process is performed while referring to the information regarding the other tile. And a tile decoding unit that executes a process of decoding the tile from the encoded data with reference to information on other tiles when the encoded data is given.   When the tile decoding means receives the decoding tile designation information designating the decoding target tile, the tile decoding means designates the tile designated by the decoding tile designation information, and the tile flag does not refer to information on other tiles. 6. The moving picture decoding apparatus according to claim 5, wherein the tile decoding process is performed only when it is indicated that the process is performed. While implementing multiple tile decoding means,
Data distribution means for distributing the encoded data and flag of each tile separated by the separation means to the plurality of tile decoding means;
6. The plurality of tile decoding units, in parallel, execute a process of decoding tiles from encoded data distributed by the data distribution unit according to a flag distributed by the data distribution unit. Or the moving image decoding apparatus of Claim 6. The tile decoding means is present in the other tile when the encoded data of the tile indicating that the flag separated by the separating means performs the coding process without referring to the information on the other tile is given. The predictive image of the tile is generated without referring to a pixel that is present, and the tile is decoded from the encoded data and the predictive image. 8. Video decoding device.   The tile decoding unit generates a differential image by decoding and inverse-quantizing the encoded data separated by the separating unit, and also generates a decoded image of the tile from the differential image and the predicted image. 9. The moving picture decoding apparatus according to claim 8, wherein when performing the loop filter process for reducing distortion, the loop filter process is performed without referring to a decoded image of another tile. Flag setting processing step in which the flag setting means sets a flag indicating whether or not to perform encoding processing with reference to information on other tiles for each tile that is a rectangular area into which the image to be encoded is divided. When,
When the tile encoding means is given a tile indicating that the flag set in the flag setting process step performs the encoding process without referring to the information on the other tile among the separated tiles, When the tile encoding process is executed independently without referring to information on other tiles and the flag indicates that the encoding process is performed while referring to information on other tiles. Tile encoding processing step for executing the tile encoding processing with reference to information on other tiles;
A multiplexing processing step in which a multiplexing means multiplexes the encoded data of each tile encoded in the tile encoding processing step and the flag of each tile set in the flag setting processing step to generate a bitstream; A video encoding method comprising: A separation processing step for separating the encoded data and the flag of each tile multiplexed in the bitstream;
Of the encoded data of each tile separated in the separation processing step, the tile decoding means has the encoded data of the tile indicating that the flag performs the encoding process without referring to the information on other tiles. If given, the process of decoding the tile from the encoded data is executed independently without referring to the information about other tiles, and the flag performs the encoding process while referring to the information about other tiles. And a tile decoding process step of executing a process of decoding the tile from the encoded data with reference to information on other tiles when the encoded data of the tile indicating .

Images