JP7523856B2 - Video encoding device and video decoding device - Google Patents

Description

Embodiments of the present invention relate to a video decoding device and a video encoding device.

To efficiently transmit or record video, video encoding devices are used that generate encoded data by encoding video, and video decoding devices are used that generate decoded images by decoding the encoded data.

Specific video coding methods include, for example, methods proposed in H.264/AVC and HEVC (High-Efficiency Video Coding).

In such a video coding method, images (pictures) constituting a video are managed in a hierarchical structure consisting of slices obtained by dividing images, coding tree units (CTUs) obtained by dividing slices, coding units (sometimes called coding units: CUs) obtained by dividing coding tree units, and transform units (TUs) obtained by dividing coding units, and are coded/decoded for each CU.

In such video coding methods, a predicted image is usually generated based on a locally decoded image obtained by encoding/decoding an input image, and the prediction error (sometimes called a "difference image" or "residual image") obtained by subtracting the predicted image from the input image (original image) is coded. Methods for generating predicted images include inter-frame prediction (inter prediction) and intra-frame prediction (intra prediction).

Also, methods of dividing a screen into multiple units for parallel processing include dividing into slices, CTU lines (wavefront segments), and tiles. Conventionally, dividing into tiles was limited to division in CTU units.

Another recent example of video encoding and decoding technology is Non-Patent Document 1.

"Algorithm Description of Joint Exploration Test Model 7", JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 2017-08-19

As explained above, there is a restriction that the tile size must be an integer multiple of the CTU, which poses the challenge of making it difficult to divide tiles into equal sizes for load balancing or to configure tiles that match the face size of a 360-degree video.

In addition, if the tile size is not limited to an integer multiple of the CTU size, there is a problem in that when referring to a motion vector of a reference picture to perform temporal motion vector reference, there is a possibility that motion vectors will be referenced from many positions across CTU boundaries.

Furthermore, if the tile size is not limited to an integer multiple of the CTU size, there is a problem in that multiple lines may be referenced across CTU boundaries in multi-line intra prediction.

Therefore, the present invention has been made in consideration of the above problems, and its purpose is to provide tile division without restrictions on integer multiples of CTU, efficient temporal motion vector reference, and multi-line intra prediction.

The video decoding device according to one aspect of the present invention is characterized by comprising a header information decoding unit that decodes a tile size that is an integer multiple of the tile unit size, a CT partitioning unit that performs multi-tree partitioning into quadtrees, binary trees, and ternary trees using intra-picture coordinates and tile size, and a temporal motion prediction unit that compares the intra-picture coordinates yColBrInPic of a reference block with the virtual CTU coordinates yVirCtbB derived from the intra-picture coordinates of a target picture to determine the referenceability of the reference block and derive a temporal motion vector. The video decoding device also comprises a header information decoding unit that decodes a tile size that is an integer multiple of the tile unit size, a CT partitioning unit that performs multi-tree partitioning into quadtrees, binary trees, and ternary trees using the intra-picture coordinates and tile size, and an intra prediction unit that converts from intra-picture coordinates to intra-tile coordinates to determine CTU boundaries.

FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. FIG. 13 is a diagram illustrating an example of division of a CTU. FIG. 1 is a diagram illustrating tiles. 13 is a syntax table relating to tile information, etc. FIG. 1 is a schematic diagram showing a configuration of a video decoding device. 11 is a flowchart illustrating a schematic operation of the video decoding device. 13 is a flowchart illustrating an operation of a CT information decoding unit. FIG. 13 is a diagram showing an example of the structure of a syntax table of CTU and QT information. FIG. 11 is a diagram showing an example of the structure of a syntax table of MT (Multi Tree) information. FIG. 11 is a diagram showing an example of the structure of a syntax table of MT (Multi Tree) information. FIG. 13 is a diagram illustrating tile division at a non-CTU size. FIG. 13 is a diagram illustrating tile division at a non-CTU size. 13 is a syntax structure of encoded slice data. 11 is a diagram showing a reference range when referring to a motion vector of a reference picture while performing virtual CTU line restriction. FIG. 13 is a flowchart for referring to a motion vector of a reference picture while performing virtual CTU line restriction. 11 is a diagram showing a reference range when referring to a motion vector of a reference picture while performing virtual CTU line restriction. FIG. 13 is a flowchart for referring to a motion vector of a reference picture while performing virtual CTU line restriction. FIG. 13 is a diagram showing the reference position of a reference picture in temporal motion derivation (temporal motion vector derivation). FIG. 1 is a block diagram showing a configuration of a video encoding device. 1 is a diagram showing the configuration of a transmitting device equipped with a video encoding device according to the present embodiment, and a receiving device equipped with a video decoding device, where (a) shows the transmitting device equipped with the video encoding device, and (b) shows the receiving device equipped with the video decoding device. 1 is a diagram showing the configuration of a recording device equipped with a video encoding device according to the present embodiment, and a playback device equipped with a video decoding device, where (a) shows the recording device equipped with the video encoding device, and (b) shows the playback device equipped with the video decoding device. 1 is a schematic diagram showing a configuration of an image transmission system according to an embodiment of the present invention. 13 is another example of a syntax table relating to tile information, etc. 13 is another example of a syntax table relating to tile information, etc. 13 is another example of a syntax table relating to tile information, etc. 13 is another example of a syntax table relating to tile information, etc. A diagram showing an example of the syntax configuration of encoded slice data and CTU information. FIG. 13 is a diagram showing an example of the structure of a syntax table of QT information. FIG. 2 is a diagram showing an example of the structure of a syntax table of MT information. A diagram showing an example of the syntax configuration of CTU information. 13 shows a syntax structure of encoded tile data. FIG. 13 is a diagram showing an example of the structure of a syntax table of CTU and QT information. FIG. 11 is a diagram showing an example of the structure of a syntax table of MT (Multi Tree) information. FIG. 13 is a diagram showing an example of the structure of a syntax table of CTU and QT information. 13 is a diagram illustrating an operation of determining an intra reference line according to a CTU boundary within a tile in multi-line intra prediction. 13 is a flowchart illustrating an operation of determining an intra reference line according to a CTU boundary within a tile in multi-line intra prediction. 13 is a flowchart illustrating an operation of determining an intra reference line according to a CTU boundary within a tile in multi-line intra prediction. FIG. 13 is a schematic diagram showing a configuration of a merge prediction parameter derivation unit. FIG. 11 is a diagram illustrating an example of a process of referring to a motion vector in a reference picture as a temporal motion vector. A diagram showing the relationship between a target CTU, a motion vector, and a reference range when the reference picture and the target picture are divided into the same tiles. FIG. 13 is a diagram showing a syntax configuration in multi-line intra prediction. 13 is a flowchart showing an MPM derivation process when a picture is divided into tiles of a size not limited to an integer multiple of a CTU. 13 is a flowchart showing an MPM derivation process when a picture is divided into tiles of a size not limited to an integer multiple of a CTU.

(First embodiment)
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

Figure 22 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an image to be encoded, and decodes the transmitted encoded stream to display an image. The image transmission system 1 includes a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41.

Image T is input to video encoding device 11.

The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be a network that transmits digital media such as DVDs (Digital Versatile Discs: registered trademark), BDs (Blue-ray Discs: registered trademark), etc.
The encoded stream Te may be replaced by a storage medium having recorded thereon the encoded stream Te.

The video decoding device 31 decodes each of the encoded streams Te transmitted by the network 21 and generates one or more decoded images Td.

The video display device 41 displays all or part of one or more decoded images Td generated by the video decoding device 31. The video display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the display form include a stationary display, a mobile display, and an HMD. When the video decoding device 31 has high processing power, it displays high quality images, and when it has only low processing power, it displays images that do not require high processing power or display power.

<Operator>
The operators used in this specification are listed below.

>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR
, |= is the OR assignment operator, and || indicates logical OR.

x?y:z is a ternary operator that takes y if x is true (non-zero) and z if x is false (0).

! indicates an operation that converts 0 to 1 and non-zero to 0.

Clip3(a,b,c) is a function that clips c to a value greater than or equal to a and less than or equal to b. If c<a, it returns a, if c>b, it returns b, and otherwise it returns c (where a<=b).

abs(a) is a function that returns the absolute value of a.

Int(a) is a function that returns the integer value of a.

floor(a) is a function that returns the largest integer less than or equal to a.

ceil(a) is a function that returns the largest integer greater than or equal to a.

a/d represents the division of a by d (truncated to an integer).

<Structure of the coded stream Te>
Before describing in detail the video encoding device 11 and the video decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 will be described.

Fig. 1 is a diagram showing a hierarchical structure of data in an encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. (a) to (f) of Fig. 1 are diagrams showing an encoded video sequence that defines a sequence SEQ, an encoded picture that defines a picture PICT, an encoded slice that defines a slice S, encoded slice data that defines slice data, an encoding tree unit included in the encoded slice data, and an encoding unit included in the encoding tree unit, respectively.

(Coded Video Sequence)
In the coded video sequence, a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed is specified. As shown in Fig. 1(a), the sequence SEQ includes a video parameter set (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and supplemental enhancement information SEI (Supplemental Enhancement Information).

The video parameter set VPS specifies a set of coding parameters common to multiple videos composed of multiple layers, as well as a set of coding parameters related to multiple layers and each individual layer included in the video.

The sequence parameter set SPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode the target sequence. For example, the width and height of a picture are specified. Note that there may be multiple SPSs. In that case, one of the multiple SPSs is selected from the PPS.

The picture parameter set PPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode each picture in the target sequence. For example, the picture parameter set PPS includes a reference value of the quantization width (pic_init_qp_minus26) used in decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. Note that there may be multiple PPSs. In that case, one of the multiple PPSs is selected for each picture in the target sequence.

(Encoded Picture)
A coded picture defines a set of data that the video decoding device 31 refers to in order to decode a picture PICT to be processed. As shown in FIG. 1B, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

In the following description, when there is no need to distinguish between slices 0 to NS-1, the subscripts of the symbols may be omitted. The same applies to other data that are included in the coded stream Te and that are to be described below and that have subscripts.

(Coding Slice)
A coded slice defines a set of data to be referenced by the video decoding device 31 in order to decode a target slice S. As shown in Fig. 1(c), a slice includes a slice header and slice data.

The slice header includes a set of coding parameters that the video decoding device 31 refers to in order to determine the decoding method for the target slice. Slice type specification information (slice_type) that specifies the slice type is an example of a coding parameter included in the slice header.

Slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction when encoding, (2) a P slice that uses unidirectional prediction or intra prediction when encoding, and (3) a B slice that uses unidirectional prediction, bidirectional prediction, or intra prediction when encoding. Note that inter prediction is not limited to uni-prediction or bi-prediction, and a predicted image may be generated using more reference pictures. Hereinafter, P
, B slice refers to a slice that includes blocks for which inter prediction can be used.

Note that the slice header may also include a reference to the picture parameter set PPS (pic_parameter_set_id).

(Encoded slice data)
The coded slice data defines a set of data to be referenced by the video decoding device 31 in order to decode the slice data to be processed. As shown in Fig. 1(d), the slice data includes a CTU. A CTU is a block of a fixed size (e.g., 128x128) that constitutes a slice, and is also called a Largest Coding Unit (LCU).

(tile)
FIG. 3(a) is a diagram showing an example of dividing a picture into N tiles (solid rectangles, the diagram shows an example where N=9). The tiles are further divided into multiple CTUs (dashed rectangles). As shown in the center of FIG. 3(a), the upper left coordinates of the tile are denoted as (xTile, yTile), its width as wTile, and its height as hTile. The width of the picture is denoted as wPict, and its height as hPict. Note that information about the number of tiles divided and their sizes is called tile information, and will be described in detail later. The units of xTile, yTile, wTile, hTile, wPict, and hPict are pixels. The width and height of the picture are set in pic_width_in_luma_samples and pic_height_in_luma_samples, which are notified by sequence_parameter_set_rbsp() (referred to as SPS) shown in FIG. 4(a).

wPict = pic_width_in_luma_samples
hPict = pic_height_in_luma_samples
Fig. 3(b) shows the coding and decoding order of CTUs when a picture is divided into tiles. The number written on each tile is a TileId (a tile identifier within a picture), and The tiles in the image may be assigned TileId numbers in raster scan order from top left to bottom right. Also, the CTU is processed in raster scan order from top left to bottom right within each tile, and processing within one tile is completed. Then, the CTU in the next tile is processed.

Figure 3(c) is a diagram showing tiles that are consecutive in the time direction. As shown in Figure 3(c), a video sequence is composed of multiple pictures that are consecutive in the time direction. A tile sequence is composed of tiles at one or more times that are consecutive in the time direction. Tile(n,tk) in the figure w represents the tile with TileId=n at time tk. Note that the CVS (Coded Video Sequence) in the figure is a group of pictures from one intra picture to the picture immediately preceding another intra picture in decoding order.

Figure 4 is an example of syntax for tile information, etc.

In the PPS (pic_parameter_set_rbsp()) shown in FIG. 4B, a tile-related parameter tile_parameters() is notified. Hereinafter, notifying a parameter means including the parameter in the encoded data (bit stream), and the video encoding device encodes the parameter, and the video decoding device decodes the parameter. As shown in FIG. 4C, when tile_enabled_flag indicating whether a tile exists is 1, tile_parameters() notifies tile information tile_info(). When tile_enabled_flag is 1, independent_tiles_flag (independent_decoding_tile_flag) indicating whether a tile can be decoded independently across multiple pictures that are consecutive in time is notified. When independent_tiles_flag is 0, the tile is decoded by referring to adjacent tiles in the reference picture (it cannot be decoded independently). When independent_tiles_flag is 1, the tile is decoded without referring to adjacent tiles in the reference picture. When tiles are used, decoding is performed without referring to adjacent tiles in the target picture regardless of the value of independent_tiles_flag, so that multiple tiles can be decoded in parallel. This independent_tiles_flag flag may be a flag independent_decoding_picture_region_flag for each region, such as a tile group unit or a tile unit. Note that independent_tiles_flag is independent (no inter-picture reference) when it is 1 and dependent (with inter-picture reference) when it is 0, but a flag with 0 and 1 reversed may also be used. For example, a flag named tile_inpicture_ref_enabled_flag, which is independent when it is 0 and dependent when it is 1, may be used.
As shown in Fig. 4(c), loop_filter_across_tiles_enable_flag is transmitted (presented), which indicates whether the loop filter at the tile boundary to be applied to the reference picture is on or off when independent_tiles_flag is 0. When independent_tiles_flag is 1, loop_filter_across_tiles_enable_flag may be always set to 0 without being transmitted (presented).

When processing tiles independently throughout a sequence, the independent tile flag independent_tiles_flag may be signaled in the SPS as shown in Figure 4(a).

The tile information tile_info() is, for example, num_tile_columns_minus1, num_tile_rows_minus1, tile_unit_width_idc, tile_unit_height_idc, uniform_spacing_flag, column_width_in_unit_minus1[i], and row_height_in_unit_minus1[i], as shown in FIG. 4(d). Here, num_tile_columns_minus1 and num_tile_rows_minus1 are the values obtained by subtracting 1 from the number of tiles M and N in the horizontal and vertical directions in the picture, respectively. uniform_spacing_flag is a flag indicating whether the picture is divided into tiles as evenly as possible. When the value of uniform_spacing_flag is 1,
In this case, the width and height of each tile in the picture are derived in the video encoding device and video decoding device (header decoding unit 3020) from the number of tiles in the horizontal and vertical directions in the picture and an identifier indicating the smallest unit of a tile (tile unit identifier) (tile_unit_width_idc, tile_unit_height_idc) so that the width and height of each tile in the picture are as equal as possible, using a specified size as a unit.

The header decoding unit 3020 derives the minimum unit of a tile (unit size of a tile) wUnitTile and hUnitTile from the tile unit identifiers tile_unit_width_idc and tile_unit_height_idc (details will be described later). A tile whose unit size is not limited to an integer multiple of the CTU size will be referred to as a flexible tile hereinafter.

wUnitTile = 1<<(log2CbSize+tile_unit_width_idc)
hUnitTile = 1<<(log2CbSize+tile_unit_height_idc)
The header decoding unit 3020 derives the number of tiles M and N in the horizontal and vertical directions in a picture as follows, where M is numTileColumns and N is numTileRows.

M = num_tile_columns_minus1+1
N = num_tile_rows_minus1+1
numTileColumns = M = num_tile_columns_minus1+1
numTileRows = N = num_tile_columns_minus1+1
The header decoder 3020 derives the tile size as follows so that the tile size is a multiple of the minimum tile size (minimum unit) wUnitTile and hUnitTile: Here, wPictInUnitTile and hPictInUnitTile are values that express wPict and hPict in units of wUnitTile and hUnitTile, respectively.

wPictInUnitTile = ceil(wPict/wUnitTile)
hPictInUnitTile = ceil(hPict/hUnitTile)
wTile[m] = ceil(wPictInUnitTile/M)*wUnitTile (0<=m<M)
hTile[n] = ceil(hPictInUnitTile/N)*hUnitTile (0<=n<N)
Alternatively, the header decoder 3020 may derive it using the following formula.

wTile[m] = floor(wPict/M/wUnitTile)*wUnitTile (0<=m<M)
hTile[n] = floor(hPict/N/hUnitTile)*hUnitTile (0<=n<N)
Alternatively, the header decoder 3020 may derive it using the following formula.

for(m=0;m<M;m++)
wTile[m] = ((m+1)*wPictInUnitTile/Mm*wPictInUnitTile/M)*wUnitTile
for(n=0;n<N;n++)
hTile[n] = ((n+1)*hPictInUnitTile/Nn*hPictInUnitTile/N)*hUnitTile
When the value of uniform_spacing_flag is 0, the width and height of each tile of the picture are set individually. The video encoding device encodes the width column_width_in_unit_minus1[i] (a value obtained by expressing wTile in FIG. 3 in units of wUnitTile) and height row_height_in_unit_minus1[i] (a value obtained by expressing hTile in FIG. 3 in units of hUnitTile) of each tile for each tile. The header decoding unit 3020 of the video decoding device decodes the tile sizes wTile[m] and hTile[n] for each tile based on the encoded (column_width_in_unit_minus1[], row_width_in_unit_minus1[]) as follows:

wTile[m] = (column_width_in_unit_minus1[m]+1)*wUnitTile (0<=m<M-1)
hTile[n] = (row_height_in_unit_minus1[m]+1)*hUnitTile (0<=n<N-1)
wTile[M-1] = ceil((wPict-sum_m(wTile[m]))/wUnitTile)*wUnitTile
hTile[N-1] = ceil((hPict-sum_n(hTile[n]))/hUnitTile)*hUnitTile
Here, sum_m(wTile[m]) represents the sum of wTile (0<=m<M-1), and sum_n(hTile[n]) represents the sum of hTile (0<=n<N-1).

Alternatively, if the picture size is not an integer multiple of the smallest unit of a tile, wTile[M-1] and hTile[N-1], which correspond to the edges of the screen, can simply be set as follows:

wTile[M-1] = wPict-sum_m(wTile[m])
hTile[N-1] = hPict-sum_n(hTile[n])
In this case, however, it is desirable to appropriately process the screen edges by padding, cropping, or the like so that the processing unit matches the minimum unit of a tile or the minimum CU size.

The tile at position (m, n) is also represented by an identifier TileId, which may be calculated as follows:

TileId = n*M+m
Alternatively, if the TileId is known, (m, n) indicating the position of the tile may be calculated from the TileId.

m = TileId%M
n = TileId/M
The header decoder 3020 decodes tile unit information indicating the unit of the tile size. If the minimum units of the tile width and height are wUnitTile and hUnitTile, the tile width and height are set to integer multiples of wUnitTile and hUnitTile, respectively.

For example, the header decoding unit 3020 decodes tile_unit_width_idc and tile_unit_height_idc as tile unit identifiers from the encoded data. The unit of the tile size may be a constant multiple (including 1) of the minimum size of the coding unit CU. If the CTU is a square with the same width and height, only tile_unit_size_idc may be decoded and the unit of the tile size may be derived by setting tile_unit_height_idc = tile_unit_width_idc = tile_unit_size_idc. In this case, in the following specification, tile_unit_width_idc and tile_unit_height_idc may be read as tile_unit_size_idc (in this case, wUnitTile = hUnitTile = UnitTileSizeY at all times).

(Setting method 1)
If the tile unit identifier is 0, the header decoding unit 3020 sets the unit of the tile size as a CTU unit. Specifically, if tile_unit_width_idc=0, it sets wUnitTile=ctuWidth, and if tile_unit_width_idc=0, it sets hUnitTile=ctuHeight. Here, ctuWidth is the width of the CTU, and ctuHeight is the height of the CTU.

Furthermore, the header decoding unit 3020 may set the unit of tile size to be a constant multiple of the minimum size of the coding unit CU when the tile unit identifier is other than 0, as follows:

wUnitTile = (tile_unit_width_idc==0) ? ctuWidth : 1<<(log2CbSize+tile_unit_width_idc-1)
hUnitTile = (tile_unit_height_idc==0) ? ctuHeight : 1<<(log2CbSize+tile_unit_height_idc-1)
where log2CbSize=log2(minCU), where minCU is the minimum size of a coding unit CU.

Note that the CTU size ctuSize=ctuWidth=ctuHeight may be derived by decoding the syntax element log2_min_luma_coding_block_size_minus3 that defines the minimum CU size and the syntax element log2_diff_max_min_luma_coding_block_size that indicates the CU size as a logarithm of the difference from the minimum CU size, as follows.

log2CbSize = log2_min_luma_coding_block_size_minus3+3
log2CtuSize = log2CbSize+log2_diff_max_min_luma_coding_block_size
minCU = 1 << log2CbSize
ctuSize = 1 << log2CtuSize
Note that when the tile unit identifier !=0, the tile unit may be set to a constant multiple (an exponential power of 2) of the minimum size of a coding unit CU.

wUnitTile = (tile_unit_width_idc==0) ? ctuWidth : 1<<(log2CbSize+ Log2DiffTileUnitSize)
hUnitTile = (tile_unit_height_idc==0) ? ctuHeight : 1<<(log2CbSize+ Log2DiffTileUnitSize)
Here, Log2DiffTileUnitSize is a constant that determines the tile unit. For example, by setting it to 1, 2, or 3, the tile unit is set to 4, 8, or 16 times the minimum CU size. For example, if the CU size is 4, In some cases, it is set to 8, 16, or 32. Furthermore, the maximum value may be clipped to the CTU size.

(Setting method 2)
The tile information is a tile unit identifier tile_unit_width_idc ranging from 0 to a predetermined number, and the unit of the tile size may be set as 1 << (log2(minCU) + tile unit identifier).
(or tile_unit_size_idc) and set the tile unit as follows:

wUnitTile = 1<<(log2CbSize+tile_unit_width_idc)
hUnitTile = 1<<(log2CbSize+tile_unit_height_idc)
If the CTU is a square with equal width and height, the above is equivalent to decoding tile_unit_size_idc and setting it as follows. The following is self-explanatory, so explanations for squares only will be omitted.

wUnitTile = hUnitTile = ctuSize>>tile_unit_size_idc
Note that the tile unit identifier may not be explicitly coded, and the tile unit may be set to a constant multiple (an exponential power of 2) of the minimum size of a coding unit CU.

wUnitTile = 1<<(log2CbSize+Log2DiffTileUnitSize)
hUnitTile = 1<<(log2CbSize+Log2DiffTileUnitSize)
Here, Log2DiffTileUnitSize is a constant that defines the tile unit. Furthermore, the maximum value is CTU
You may clip it to size.

wUnitTile = min(ctuWidth,1<<(log2CbSize+Log2DiffTileUnitSize))
hUnitTile = min(ctuHeight1<<(log2CbSize+Log2DiffTileUnitSize))
(Setting method 3)
The tile information is a tile unit identifier ranging from 0 to a predetermined number, and the unit of tile size is set to 1/(log2(CTU size)-(CTU size)) so that the unit of tile size is 1/(exponential power of 2) of the CTU size. That is, the header decoding unit 3020 decodes tile_unit_width_idc, tile_unit_height_idc (or tile_unit_size_idc),
The tile units may be set as follows:

wUnitTile = 1<<(log2CtuWidthY-tile_unit_width_idc)
hUnitTile = 1<<(log2CtuHeightY-tile_unit_height_idc)
Here, log2CtuWidthY = log2(ctuWidth) and log2CtuHeightY = log2(ctuHeight).

You can also set the width and height of the CTU size to the same value.

wUnitTile = 1<<(log2CtuSize-tile_unit_width_idc)
hUnitTile = 1<<(log2CtuSize-tile_unit_height_idc)
Here, log2CtuSize = log2(ctuWidth) = log2(ctuHeight).
In this case, since the difference is expressed in logarithm, it is also appropriate to call tile_unit_width_idc log2_diff_luma_tile_unit_size.

Note that the tile unit may be clipped to the minimum size that is a multiple of 8 as follows:

wUnitTile = max(8, ctuWidth>>tile_unit_width_idc)
hUnitTile = max(8, ctuHeight>>tile_unit_height_idc)
This also produces the effect shown in (setting method 5) since the tile width is always a multiple of 8.

You can also clip the unit size to the minCTU size as follows:

wUnitTile = max(minCTU, ctuWidth>>tile_unit_width_idc)
hUnitTile = max(minCTU, ctuHeight>>tile_unit_height_idc)
This also produces the effect shown in (setting method 5) since the tile width is always a multiple of the minimum size of the CTU.

Furthermore, the range of tile_unit_width_idc may be restricted. For example, if the minimum value of ctuWidth>>tile_unit_width_idc is set as minCTU,
ctuWidth>>tile_unit_width_idc >= minCTU
ctuWidth >= minCTU<< tile_unit_width_idc
ctuWidth >= minCTU * (1<<tile_unit_width_idc)
(1<<tile_unit_width_idc) <= ctuWidth/minCTU
tile_unit_width_idc <= log2(ctuWidth)-log2(minCTU)
Therefore, tile_unit_width_idc is limited to be equal to or greater than 0 and equal to or less than log2(ctuWidth)-log2(minCTU). That is, the video decoding device can use a tile unit that is a multiple of minCTU by decoding encoded data that is limited to be equal to or greater than 0 and equal to or less than log2(ctuWidth)-log2(minCTU).

Similarly, by decoding the encoded data restricted to the range 0 to log2(ctuWidth)-log2(8)=log2(ctuWidth)-3, it is possible to use tile units that are multiples of 8. If the number is not a multiple of 8 but a multiple of 16 or 32, simply change the 3(log2(8)) above to 4 or 5.

(Setting method 4)
The tile information is a tile unit identifier ranging from 0 to a predetermined number, and is characterized in that the CTU size >> tile unit identifier is set so that the unit size of the tile is 1/the exponential power of the CTU size of 2. That is, the header decoding unit 3020 may decode tile_unit_width_idc and tile_unit_height_idc (or tile_unit_size_idc) and set the tile unit as follows:

wUnitTile = ctuWidth>>tile_unit_width_idc
hUnitTile = ctuHeight>>tile_unit_height_idc
Since setting method 3 and setting method 4 are equivalent, the method described in setting method 3 can be used to limit tile_unit_width_idc and tile_unit_height_idc (or tile_unit_size_idc).

(Setting method 5)
The header decoding unit 3020 does not decode the tile unit identifiers (tile_unit_width_idc, tile_unit_height_idc), but sets a predetermined value TileUnit to wUnitTile and hUnitTile. The predetermined value TileUnit is preferably a multiple of 8, particularly any of 8, 16, and 32. The tile information tile_info() is notified as shown in FIG. 23(d) instead of FIG. 4(d), and the header decoding unit 3020 decodes this tile information.

wUnitTile = TileUnit
hUnitTile = TileUnit
Here, TileUnit is a multiple of 8, such as 8, 16, or 32. If the tile unit is a multiple of 8, even if the pixel bitDepth is not a multiple of 8, such as 10 bits, the pixel data of the tile width will be 8 bits.
This allows the tile width to be a multiple of 16. As a result, when arranging in memory, the tile boundaries are always byte (8-bit) boundaries, and no memory gaps are required between tiles when arranging them in memory. If there are no gaps, no memory for the gaps is required and data can be transferred continuously, which has the effect of reducing memory size and enabling high-speed transfer. Furthermore, if the tile width is a multiple of 16, the data can always be a multiple of 128 bits (when bitDepth = 8 bits), enabling high-speed access with burst access to many memories (e.g. DDR).

In addition, the values 16 and 32 have the effect of matching the minimum CTU size described below.

(Setting method 6)
The header decoding unit 3020 does not decode the tile unit identifiers (tile_unit_width_idc, tile_unit_height_idc), and sets a predetermined value TileUnit to wUnitTile and hUnitTile. The predetermined value TileUnit is preferably the minimum size of a CTU (minimum CTU size). In this case, the tile information tile_info() is notified as shown in FIG. 23(d) instead of FIG. 4(d), and the header decoding unit 3020 decodes this tile information.

wUnitTile = TileUnit
hUnitTile = TileUnit
TileUnit = 1<<(log2_min_luma_coding_tree_block_size_minus4+4)
Here, log2_min_luma_coding_block_size_minus4 is the power-of-two representation of the minimum CTU size minus 4. For example, if the values are 0, 1, 2, and 3, the minimum CTU sizes are 16, 32, and 4, respectively.
The minimum CTU size is 64, 128. The log2_min_luma_coding_block_size_minus4 that determines the minimum CTU size may be coded in the coded data as a syntax element or may be a constant. For example, if it is a constant, the minimum CTU size is usually 16, 32, which coincides with (setting method 5).

The minimum CTU size may be set by a profile or level that specifies the functions and capabilities of the video decoder. The profile and level are used in negotiation between devices and are transmitted in a relatively high-level parameter set or header, for example, in the syntax elements profile_idc and level_idc of the SPS.

When the tile unit is a multiple of the minimum CTU size, the tile width is always a multiple of the minimum CTU size. Typically, when storing a picture in memory, memory allocation is optimized based on the fact that the range of possible CTU sizes is limited (for example, 16, 32, 64). If the tile width is an integer multiple (an exponential power of 2) of the CTU size, memory allocation can be performed for tiles in the same way as for pictures, resulting in the effect of saving memory and enabling high-speed access to images.

(Setting method 7)
The tile information is a tile unit identifier ranging from 0 to a specified number, and is set as 1<<(log2(minimum CTU size) + tile unit identifier) so that the unit of tile size is an integer multiple (an exponential power of 2) of the minimum CTU size. In (setting method 7), the tile unit identifier is a logarithmic representation of a multiple of the minimum CTU size. That is, the header decoding unit 3020 may decode tile_unit_width_idc and set the tile unit as follows:

wUnitTile = 1<<(log2CtuUnitSize+tile_unit_width_idc)
hUnitTile = 1<<(log2CtuUnitSize+tile_unit_height_idc)
Here, the minimum size of CTU may be minCTU=ctuUnitSize=1<<(log2_min_luma_coding_tree_block_size_minus4+4), or log2CtuUnitSize=log2_min_luma_coding_tree_block_size_minus4+4.

(Setting method 8)
The tile information is a tile unit identifier ranging from 0 to a predetermined number, and the unit of tile size is set as maximum CTU size >> tile unit identifier. In (setting method 8), the tile unit identifier is a logarithmic representation of a multiple of the maximum CTU size (1 times a power of 2). That is, the header decoding unit 3020 may decode tile_unit_width_idc and set the tile unit as follows:

wUnitTile = ctuUnitSize>>tile_unit_width_idc
hUnitTile = ctuUnitSize>>tile_unit_height_idc
(Setting method 9)
In (setting method 9), as shown in Fig. 24(a), tile_parameters() and tile information tile_info() are notified in SPS, not PPS. As shown in Fig. 24(c), is notified by log2_min_unit_tile_size_minus3. The header decoding unit 3020 decodes this tile information and derives the minimum unit of a tile (wUnitTile, hUnitTile).

wUnitTile = 1<<(log2_min_unit_tile_size_minus3+3)
hUnitTile = 1<<(log2_min_unit_tile_size_minus3+3)
Here, log2_min_unit_tile_size_minus3 is the power-of-two representation of the minimum value of the minimum tile unit minus 3, and is an integer equal to or greater than 0. Therefore, the minimum tile unit is 8 or greater.

As another example, as shown in Fig. 25(a), only log2_diff_curr_min_unit_tile_size may be notified as tile information. log2_diff_curr_min_unit_tile_size is the difference between the minimum unit of tile size in the SPS and the minimum size of the CTU. The header decoding unit 3020 decodes this tile information and derives the minimum unit of the tile (wUnitTile, hUnitTile).

wUnitTile = 1<<( log2CtuSize + log2_diff_curr_min_unit_tile_size )
hUnitTile = 1<<( log2CtuSize + log2_diff_curr_min_unit_tile_size )
(Setting method 10)
In (Setting method 10), a constraint is imposed on (Setting method 9) that the minimum unit of tile size in the SPS does not exceed the CTU size in the SPS. Therefore, log2_diff_curr_min_unit_tile_size must be a value that satisfies the following formula: It won't happen.

0<=log2_diff_curr_min_unit_tile_size<=log2_diff_max_min_luma_coding_block_size, where log2_diff_max_min_luma_coding_block_size is the difference between the power-of-two representation of the maximum and minimum sizes of the CTU.

(Setting method 11)
In (setting method 11), as shown in Fig. 26, tile_parameters() and tile information tile_info() are notified in the PPS. As shown in Fig. 26(a) and (d), the minimum unit of a tile is derived using the minimum size of a CTU notified in the SPS and the tile information (log2_diff_curr_min_unit_tile_size) notified in the PPS. log2_diff_curr_min_unit_tile_size is the difference between the minimum unit of the tile size and the minimum size of a CTU. The header decoding unit 3020 decodes this tile information and derives wUnitTile and hUnitTile.

wUnitTile = 1<<(log2CtuSize+log2_diff_curr_min_unit_tile_size)
hUnitTile = 1<<(log2CtuSize+log2_diff_curr_min_unit_tile_size)
(Setting method 12)
In (Setting method 12), wUnitTile and hUnitTile are derived for TileUnit by referring to independent_tiles_flag. When independent_tiles_flag=1, that is, when each tile can be processed independently (without referring to other tiles), the process is more efficient than when it is not. Set the tile unit (minimum unit) of the tile to a smaller value. For example, in (setting method 5), if independent_tiles_flag=1, set the predetermined value TileUnit to 8, and if independent_tiles_flag=0, set the predetermined value TileUnit to
Set to 16.

As another example, in (setting method 6), when independent_tiles_flag=1, a specified value TileUnit is set to a value equal to or less than the minimum size of a CTU (minimum CTU size), and when independent_tiles_flag=0, a specified value TileUnit is set to the minimum size of a CTU (minimum CTU size).

As another example, in (setting method 6), when independent_tiles_flag = 1, the predetermined value TileUnit is set to a value equal to or smaller than the CTU size, and when independent_tiles_flag = 0, the predetermined value TileUnit is set to the CTU size. For example, when independent_tiles_flag = 1, the predetermined value TileUnit is set to any one of 8, 16, or 32, and when it is 0, the CTU size is used.

Alternatively, as another example, in (setting method 6), if independent_tiles_flag=1, a predetermined value TileUnit is set to the minimum size of a CTU (minimum CTU size), and if independent_tiles_flag=0, a predetermined value TileUnit is set to the CTU size.

When independent_tiles_flag is 1, i.e., tiles that are decoded independently without referencing each other, the tiles can be allocated independently in memory (i.e., there is no problem even if memory gaps occur between tiles), so even a small value does not cause problems in memory allocation. This has the effect of allowing flexible determination of tile size in the case of areas that can be decoded independently.

(Setting method 13)
In (setting method 13), a predetermined value TileUnit is set to match the unit of other processing. For example, the unit of prediction (sub-CU) is set as the predetermined value TileUnit. Or, the unit for storing motion vectors for TMVP is set as the predetermined value TileUnit. For example, if the unit for storing motion vectors for TMVP is 8x8, TileUnit=8 is set. Or, the unit for updating quantization parameters is set as the predetermined value TileUnit. For example, if the unit for notifying differential quantization parameters is 16x16, TileUnit=16 is set.

(Setting method 14)
In (setting method 14), a predetermined value TileUnit is set to an integer multiple ctuSize of the minimum CTU size for the horizontal direction of the tile, and to 8 or 16 for the vertical direction. The header decoder 3020 decodes this tile information. For example, the minimum unit of the tile is set using the following formula.

wUnitTile = ctuSize
hUnitTile = 8
(Tile information setting section)
The tile information setting unit sets the top left coordinates (xTile, yTile) of the tile based on the minimum tile unit wUnitTile and hUnitTile. Furthermore, the tile information setting unit sets the width tileWidthInCtus (=TileWidthInCtbsY) and height tileHeightInCtus (=TileHeightInCtbsY) in CTU units of the tile.

(Step 1)
Derive the screen widths wPictInUnitTile and hPictInUnitTile in tile units (wUnitTile, hUnitTile) from the minimum tile units wUnitTile and hUnitTile and the screen widths wPict and hPict.

wPictInUnitTile = divRoundUp(wPict,wUnitTile)
hPictInUnitTile = divRoundUp(hPict,hUnitTile)
where divRoundUp(x,y) = (x+y-1)/y
That is, it returns the smallest integer greater than or equal to the value of x divided by y. In other words, it can be expressed as ceil(x/y).

(Step 2)
The tile information setting unit derives the top left coordinates (xTile, yTile) of a tile in a picture indicated by a tile unit position (col, row). Hereinafter, the coordinates in a picture refer to the coordinates in the picture coordinate system in which the top left of the picture is (0, 0), and the coordinates in a tile refer to the coordinates in the tile coordinate system in which the top left of each tile is (0, 0). The unit loops through the column positions col of the tiles from 0 to the number of columns numTileColumns-1 of the tiles, and derives the X coordinate tx[col] and width tw[col] of each tile in each col using the minimum unit wUnitTile. Similarly, the unit loops through the row positions row of the tiles from 0 to the number of rows numTileRows-1 of the tiles, and derives the Y coordinate ty[row] and height th[row] of each tile in each row using the minimum unit hUnitTile.

When uniform_spacing_flag is 1, the tile information setting unit divides the screen into tiles of numTileColumns x numTileRows (= M x N), and derives the tile size tw[col], th[row], and the X coordinate tx[col] and Y coordinate ty[row] of the tile. Specifically, as shown in the following pseudo code, tx[col] and tw[col] are derived using the picture width wPict and the number of tile columns numTileColumns, and ty[row] and th[row] are derived using the picture width hPict and the number of tile columns numTileRows. For ty and tw, the values previously calculated as the above-mentioned hTile and wTile may be used. Note that min() is applied to wPict-tx[col] and tw[col] to prevent the tiles from exceeding the screen edge.

for(col=0; col <numTileColumns; col++)
{
tx[col] = wUnitTile*(col*wPictInUnitTile/numTileColumns)
tw[col] = wUnitTile*((col+1)*wPictInUnitTile/numTileColumns)-tx[col]
tw[col] = min(wPict-tx[col],tw[col])
}
for(row=0; row <numTileRows; row++)
{
ty[row] = hUnitTile*(row*hPictInUnitTile/numTileRows)
th[row] = hUnitTile*((row+1)*hPictInUnitTile/numTileRows)-ty[row]
th[row] = min(hPict-ty[row],th[row])
}
If uniform_spacing_flag is 0, the tile information setting unit divides the screen using the tile sizes wTile(col) and hTile(row) and derives the X coordinate tx[col] and the Y coordinate ty[row] of the tile. .
Specifically, as shown in the following pseudo code, tx[col] is derived using the picture width wPict and the tile column width wTile[col], and the picture height hPict and the tile row height hTile[col] are calculated. Use [row] to derive ty[row].

for(col=0;col<numTileColumns-1;col++)
{
tw[col] = wTile[col] = (column_width_in_unit_minus1[col]+1)*wUnitTile
tx[col+1] = tx[col]+tw[col]
}
tw[numTileColumns-1] = wPict-tx[numTileColumns-1]
for(row=0;row<numTileRows-1;row++)
{
th[row] = hTile[row] = (column_height_in_unit_minus1[row]+1)*hUnitTile
ty[row+1] = ty[row]+th[row]
}
th[numTileRows-1] = hPict-ty[numTileRows-1]
The tile information setting unit sets the derived X coordinate tx[col], Y coordinate ty[row], and tile size tw[col], th[row] of the tile to the top left coordinate xTile in the picture of the tile indicated by the tile ID. Store in [TileId], yTile[TileId], tile size wTile[TileId], and hTile[TileId].

for(row=0;row<numTileRows;row++)
{
for(col=0;col<numTileColumns;col++)
{
TileId = row*numTileColumns+col
xTile[TileId] = tx[col]
yTile[TileId] = ty[row]
wTile[TileId] = tw[col]
hTile[TileId] = th[row]
tileWidthInCtus[TileId] = divRoundUp(tw[col],ctuWidth)
tileHeightInCtus[TileId] = divRoundUp(th[row],ctuHeight)
}
}
for(PicWidthInCtbsY=0,i=0;i<=num_tile_columns_minus1;i++)
PicWidthInCtbsY += tileWidthInCtus[i]
for(PicHeightInCtbsY=0,j=0;j<=num_tile_rows_minus1;j++)
PicHeightInCtbsY += tileHeightInCtus[j)]
Note that the following expressions have different temporary variables but perform the same operation, so the tile information setting unit may perform the following operation.

if (uniform_spacing_flag) {
for (i=0; i<= num_tile_columns_minus1;i++)
colWidth[i] = ((i+1)*wPictInUnitTile)/(num_tile_columns_minus1+1)-(i*hPictInUnitTile)/(num_tile_columns_minus1+1))
for (j=0; j<= num_tile_rows_minus1;j++)
rowHeight[j] = ((j+1)*hPictInUnitTile)/(num_tile_rows_minus1+1)-(j*hPictInUnitTile)/(num_tile_rows_minus1+1))
else {
colWidth[num_tile_columns_minus1] = wPictInUnitTile
for (i=0; i<= num_tile_columns_minus1;i++) {
colWidth[i] = column_width_minus1[i]+1
colWidth[num_tile_columns_minus1]-=colWidth[i]
}
rowHeight [num_tile_rows_minus1] = hPictInUnitTile
for (j=0; j<= num_tile_rows_minus1;j++) {
rowHeight[j] = row_height_minus1[j]+1
rowHeight[num_tile_rows_minus1]-=rowHeight[j]
}
}
}
for (colX[0]=0,i=0; i <= num_tile_columns_minus1; i++)
colX[i+1] = colX[i]+colWidth[i]*wUnitTile
for (colY[0]=0,j=0; j <= num_tile_rows_minus1; j++)
colY[j+1] = colY[j]+colRow[j]*hUnitTile
for (colBd[0]=0,i=0; i <= num_tile_columns_minus1; i++)
colBd[i+1] = colBd[i]+Ceil(colWidth[i]*wUnitTile÷ctuWidth)
for (rowBd[0]=0,j=0; j <= num_tile_rows_minus1;j++)
rowBd[j+1] = rowBd[j]+Ceil(rowHeight[j]*hUnitTile÷ctuHeight)
for(PicWidthInCtbsY=0,i=0;i<=num_tile_columns_minus1; i++)
PicWidthInCtbsY += Ceil(colWidth[i]*TileUnitSizeY÷ctuSize)
for(PicHeightInCtbsY=0,j=0;j<=num_tile_rows_minus1; j++)
PicHeightInCtbsY += Ceil(rowHeight[j]*TileUnitSizeY÷ctuSize)
It should be noted that the following derivation may be performed for the entire screen.

for(i = 0; i <NumTilesInPic; i++ ) {
col = i % ( num_tile_columns_minus1 + 1 )
row = i / ( num_tile_columns_minus1 + 1 )
TileWidth [ i ] = colWidth[ col ] * TileUnitSizeY
TileHeight [ i ] = rowHeight[ row ] * TileUnitSizeY
TileWidthInCtbsY[ i ] = Ceil( wTile [ i ] ÷ ctuSize )
TileHeightInCtbsY[ i ] = Ceil( hTile [ i ] ÷ ctuSize )
TileX[ i ] = TileColX[ col ]
TileY[ i ] = TileRowY[ row ]
TileXInCtbsY[ i ] = colBd[ col ]
TileYInCtbsY[ i ] = rowBd[ row ]
}
where TileUnitSizeY = wUnitTile = hUnitTile
As described above, the slice data (slice_segment_data()) uses the tile information notified by the PPS shown in FIG. 4(b) to notify the information of the tile starting from the position (xTile, yTile) on the picture in CTU units. Specifically, starting from (xTile, yTile) on the picture, the upper left coordinate of each tile is (0,0), and the tile is divided into CTUs (width ctuWidth, height ctuHeight), and the encoded data of each CTU is coding_quadtree() may also be notified.

The header decoding unit 3020 decodes header information from the coded stream Te that is input from the outside and coded in units of network abstraction layer (NAL) units. The header decoding unit 3020 also derives tiles (TileId) required for display from control information that is input from the outside and indicates an image area to be displayed on a display or the like. The header decoding unit 3020 also extracts coded tiles required for display from the coded stream Te. The coded tiles are divided into CTUs by the CT information decoding unit 3021 according to the syntax in the coded data and processed. Specifically, the tile information includes the number of tiles in the horizontal direction M, the number of tiles in the vertical direction N, the tile width wTile[m] and height hTile[n], etc., calculated from syntax such as num_tile_columns_minus1, num_tile_rows_minus1, uniform_spacing_flag, column_width_in_unit_minus1[i], row_height_in_unit_minus1[i], etc.

Fig. 11 is an example of dividing tiles into sizes other than integer multiples of the CTU size. Fig. 11(a) is a diagram showing the technology of this embodiment, in which a 1920x1080 HD image is divided into 4x3 tiles. In this embodiment, when dividing into 4x3 tiles, all tiles can be divided into equal sizes (divided into 480x360), which has the effect of enabling equal load balancing across multiple processors and hardware. The tile size can be a size other than an integer multiple of the CTU, regardless of picture boundaries.
Fig. 11(b) is a diagram showing how each tile is divided into CTUs. When dividing into CTUs, if the tile size is not an integer multiple of the CTU size, a crop offset area is provided outside the tile. As shown in particular in TILE B, the CTUs are divided based on the top left corner of each tile. Therefore, the top left coordinate of a CTU is not limited to being an integer multiple of the CTU size.

FIG. 12 is a diagram showing a case where each tile is further divided into CTUs. In the figure, a picture is divided into 2x2 tiles, and each tile is further divided into CTUs. In this embodiment, as shown in the figure,
Each tile is divided into CTUs such that the CTU starts at the top left of the tile, so that the tile may end at a position on the right and bottom that is not an integer multiple of the CTU size, but the CTU boundary coincides with the tile boundary on the left and top.

(coding tree unit)
In Fig. 1(e), a set of data that the video decoding device 31 refers to in order to decode the CTU to be processed is specified. The CTU is divided into coding units CU, which are basic units of the coding process, by recursive quad tree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division), or ternary tree division (TT (Ternary Tree) division). BT division and TT division are collectively called multi tree division (MT (Multi Tree) division). A node of a tree structure obtained by recursive quad tree division is called a coding node. The intermediate nodes of the quad tree, binary tree, and ternary tree are coding nodes, and the CTU itself is also specified as the top coding node. QT division, BT division, and TT division are collectively called CT division.

CT includes, as CT information, a QT split flag (cu_split_flag) indicating whether or not to perform QT split, MT
It includes MT split mode (split_mt_mode) indicating the split method, MT split direction (split_mt_dir) indicating the split direction of MT split, and MT split type (split_mt_type) indicating the split type of MT split. cu_split_flag, split_mt_flag, split_mt_dir, and split_mt_type are transmitted for each encoding node.

If cu_split_flag is 1, the coding node is split into four coding nodes (Figure 2(b)).
). When cu_split_flag is 0, and split_mt_flag is 0, the coding node is not split and has one CU as a node (Figure 2(a)). A CU is the terminal node of a coding node and is not split any further. A CU is the basic unit of coding processing.

When split_mt_flag is 1, the coding node is MT-split as follows: When split_mt_type is 0, if split_mt_dir is 1, the coding node is split horizontally into two coding nodes (Fig. 2(d)), and when split_mt_dir is 0, the coding node is split vertically into two coding nodes (Fig. 2(c)). When split_mt_type is 1, if split_mt_dir is 1, the coding node is split horizontally into three coding nodes (Fig. 2(f)), and when split_mt_dir is 0, the coding node is split vertically into three coding nodes (Fig. 2(e)).

Also, when the size of the CTU is 64x64 pixels, the size of the CU is 64x64 pixels, 64x32 pixels, 32x64 pixels, 32x32 pixels, 64x16 pixels, 16x64 pixels, 32x16 pixels, 16x32 pixels, 16x16 pixels, 64x8 pixels, 16x16 ...
Pixels, 8x64 pixels, 32x8 pixels, 8x32 pixels, 16x8 pixels, 8x16 pixels, 8x8 pixels, 64x4 pixels, 4x64
The pixel size can be any of 32x4 pixels, 4x32 pixels, 16x4 pixels, 4x16 pixels, 8x4 pixels, 4x8 pixels, and 4x4 pixels.

(Encoding Unit)
As shown in Fig. 1(f), a set of data to be referenced by the video decoding device 31 in order to decode a coding unit to be processed is defined. Specifically, a CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header defines a prediction mode, etc.

Prediction processing may be performed on a CU basis, or on a sub-CU basis, which is a further division of a CU. If the size of the CU and sub-CU are the same, there is one sub-CU in the CU. If the size of the CU is larger than the size of the sub-CU, the CU is divided into sub-CUs. For example, if the CU is 8x8 and the sub-CU is 4x4, the CU is divided into 2 parts horizontally and 2 parts vertically, into 4 sub-CUs.

There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction is a prediction within the same picture, while inter prediction refers to a prediction process performed between different pictures (e.g., between display times or between layer images).

The transformation and quantization processes are performed in units of CUs, but the quantized transformation coefficients may be entropy coded in units of subblocks such as 4x4.

(Prediction parameters)
The predicted image is derived from prediction parameters associated with the block, which include intra-prediction and inter-prediction parameters.

(Configuration of video decoding device)
The configuration of a video decoding device 31 (FIG. 5) according to this embodiment will be described.

The video decoding device 31 includes a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generating unit (prediction image generating device) 308, an inverse quantization and inverse transform unit 311, and an addition unit 312. Note that, in accordance with the video encoding device 11 described below, the video decoding device 31 may also be configured not to include the loop filter 305. The parameter decoding unit 302 further includes an entropy decoding unit 301, a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024.

(Decoding module)
The operation of each module will be outlined below: The parameter decoding unit 302 performs a process of decoding parameters such as header information, partition information, prediction information, and quantized transform coefficients.

The entropy decoding unit 301 decodes syntax elements from binary data. More specifically, the entropy decoding unit 301 decodes syntax elements from data encoded by an entropy encoding method such as CABAC based on syntax elements supplied from a supply source, and returns the decoded syntax elements to the supply source. In the example shown below, the supply sources of the syntax elements are a CT information decoding unit 3021 and a CU decoding unit 3022.

(Basic flow)
FIG. 6 is a flowchart illustrating a schematic operation of the video decoding device 31.

(S1100: Decoding parameter set information) The header decoding unit 3020 decodes parameter set information such as VPS, SPS, and PPS from the encoded data.

(S1200: Decoding slice information) The header decoding unit 3020 decodes the slice header (slice information) from the encoded data.

Hereinafter, the video decoding device 31 performs steps S1300 to S5000 for each CTU included in the target picture.
By repeating the above process, a decoded image of each CTU is derived.

(S1300: Decoding CTU information) The CT information decoding unit 3021 decodes the CTU from the encoded data.

(S1400: CT information decoding) The CT information decoding unit 3021 decodes the CT from the encoded data.

(S1500: CU decoding) The CU decoding unit 3022 performs S1510 and S1520 to decode the CU from the encoded data.

(S1510: Decoding CU information) The CU decoding unit 3022 decodes CU information, prediction information, the TU split flag split_transform_flag, the CU residual flags cbf_cb, cbf_cr, cbf_luma, and the like from the encoded data.

(S1520: TU information decoding) When a prediction error is included in a TU, the TU decoding unit 3024 decodes QP update information (quantization correction value) and quantization prediction error (residual_coding) from the encoded data. Note that the QP update information is a difference value from the quantization parameter predicted value qPpred, which is a predicted value of the quantization parameter QP.

(S2000: Generation of predicted image) The predicted image generation unit 308 generates a predicted image for each block included in the target CU based on the prediction information.

(S3000: Inverse quantization and inverse transform) The inverse quantization and inverse transform unit 311 performs inverse quantization and inverse transform processing on each TU included in the target CU.

(S4000: Generation of decoded image) The adder 312 adds the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization and inverse transform unit 311 to generate a decoded image of the current CU.

(S5000: Loop filter) The loop filter 305 applies a loop filter such as a deblocking filter, SAO, or ALF to the decoded image to generate a decoded image.

(Decoding CTU in case of tiles)
13 is a diagram showing encoded slice data. The CT information decoding unit 3021 of the parameter decoding unit 302 sequentially processes the CTUs constituting a slice (tile). Furthermore, the CT information decoding unit 3021 uses a target CTU coordinate derivation unit (not shown) to derive the upper left coordinates (xCtb, yCtb) of the target CTU from the CTU raster scan address CtbAddrInRs of the target CTU by referring to the CTU address tables CtbAddrToCtbX[], CtbAddrToCtbY[].

xCtb = CtbAddrToCtbX[CtbAddrInRs]
yCtb = CtbAddrToCtbY[CtbAddrInRs]
The CT information decoding unit 3021 decodes the coding tree coding_quadtree (xCtb, yCtb, log2CbSize, cqtDepth) with the target CTU as the root from the coded data by recursive processing.

Here, CtbAddrToCtbX and CtbAddrToCtbY are tables derived by the CTU address table derivation unit.
The parameter decoding unit 302 decodes the flag end_of_slice_segment_flag indicating whether or not the end of the slice segment is reached, increments the CTU address CtbAddrInTs by 1 (CtbAddrInTs++) in preparation for processing the next CTU, and derives the address of the subsequent CTU. It also derives the CTU raster scan address corresponding to the subsequent CTU (CtbAddrInRs = CtbAddrTsToRs[CtbAddrInTs]). Here, CtbAddrInTs is the tile scan address of the target CTU.

If the data is not the end of the slice segment (!end_of_slice_segment_flag), the tile is enabled (tiles_enabled_flag), and the tile ID of the subsequent CTU (TileId[CtbAddrInTs] or TildIdTbl[CtbAddrInTs]) is different from the tile ID of the CTU of the target CTU ([CtbAddrInTs-1]), the parameter decoding unit 302 decodes the end_of_subset_one_bit and further decodes the bit for byte alignment (byte_alignment()).

Furthermore, in addition to tiles, the parameter decoding unit 302 supports a wavefront that performs parallel processing on a CTU line basis, and when the wavefront flag entropy_coding_sync_enabled_flag is 1 and the subsequent CTU is at the beginning of a CTU line (CtbAddrInRs%PicWidthInCtbsY==0), it may decode end_of_subset_one_bit and also decode a bit for byte alignment (byte_alignment()).

The condition for decoding end_of_subset_one_bit and byte_alignment() may be the condition shown in Fig. 27(a). That is, when the end of the slice segment is not reached (!end_of_slice_segment_flag) and the tile is enabled (tiles_enabled_flag), the parameter decoding unit 302 decodes end_of_subset_one_bit and byte_alignment().

Here, in this embodiment, a feature is the method of deriving PicWidthInCtbsY in the case of tiles with sizes not limited to integer multiples of the CTU size. PicWidthInCtbsY means how many CTUs are included in the picture width, but in the case where the size of the effective area of the tile is not limited to an integer multiple of the CTU size as in this embodiment, there may be a plurality of CTUs whose effective area is smaller than the CTU size, so the value of PicWidthInCtbsY is not necessarily equal to ceil(wPict/ctuWidth). As described above, the parameter decoding unit 302 in this embodiment derives PicWidthInCtbsY in the following manner.

for(PicWidthInCtbsY=0,i=0;i<=num_tile_columns_minus1;i++)
PicWidthInCtbsY += tileWidthInCtus[i]
The same applies to PicHeightInCtbsY, which is derived in the following way.

for(PicHeightInCtbsY=0,j=0;j<=num_tile_rows_minus1;j++)
PicHeightInCtbsY += tileHeightInCtus[j]
When tiles are limited to integer multiples of the CTU size, they may be derived in the following manner.

PicWidthInCtbsY = Ceil(wPict/ctuWidth)
PicHeightInCtbsY = Ceil(hPict/ctuHeight)
The division (/) here is with decimal precision.

(CTU address table derivation part)
The CTU address table derivation unit derives the CTU address tables CtbAddrToCtbX[], CtbAddrToCtbY[] that derive the CTU coordinates (xCtb, yCtb) from the CTU raster scan address (CtbAddrInRs) by the following steps.

(Step 1)
The tile ID (TileId) is derived from the CTU raster scan address CtbAddrInRs of the target CTU.

TileId = CtbAddrRsToTileID[CtbAddrInRs]
Here, CtbAddrRsToTileID is a table derived by the tile ID table derivation unit.

(Step 2)
The CTU address table derivation unit derives the first CTU raster scan address firstCtbAddrInRs[TileId] of the tile that contains the target CTU, and derives the intra-tile coordinates (xCtbInCtus, yCtbInCtus) in units of CTUs using the following formula, using the number of CTUs included in the picture width PicWidthInCtbsY.
xCtbInCtus = (CtbAddrInRs-firstCtbAddrInRs[TileId])%PicWidthInCtbsY
yCtbInCtus = (CtbAddrInRs-firstCtbAddrInRs[TileId])/PicWidthInCtbsY
Here, firstCtbAddrInRs is a table for deriving the first CTU raster scan address of the tile indicated by the tile ID, and is derived by the first tile address derivation unit.

The CTU address table derivation unit multiplies the CTU size (ctuWidth, ctuHeight) to derive the pixel-unit coordinates within the tile (xCtbInTile, yCtbInTile) using the following formula.

xCtbInTile = xCtbInCtus*ctuWidth
yCtbInTile = yCtbInCtus*ctuHeight
Alternatively, the logarithmic representation of the CTU size may be used to derive it as follows:

xCtbInTile = xCtbInCtus<<CtbLog2SizeY
yCtbInTile = yCtbInCtus<<CtbLog2SizeY
(Step 3)
The CTU address table derivation unit calculates the top left coordinates (xCtb, yCtb) of the CTU in the picture from the sum of the coordinates in the tile (xCtbInTile, yCtbInTile) and the coordinates in the picture of the top left of the tile (xTile[TileId], yTile[TileId]). Derive.

xCtb = xTile[TileId]+xCtbInTile
yCtb = yTile[TileId]+yCtbInTile
(Step 4)
The CTU address table derivation unit stores the top left coordinates (xCtb, yCtb) of the CTU related to the last derived CtbAddrInRs in the table.
CtbAddrToCtbX[CtbAddrInRs] = xCtb
CtbAddrToCtbY[CtbAddrInRs] = yCtb
The above process can also be expressed as follows, using the position of the top left CTU of the tile (xTileInCtus, yTileInCtu):

xTileInCtus = firstCtbAddrInRs[TileId] % PicWidthInCtbsY
yTileInCtus = firstCtbAddrInRs[TileId] / PicWidthInCtbsY
xCtb = (((CtbAddrInRs-xTileInCtus)%tileWidthInCtus[TileId])<<log2CtuWidthY)+xTile[TileId]
yCtb = (((CtbAddrInRs-yTileInCtus)/tileWidthInCtus[TileId])<<log2CtuHeightY)+yTile[TileId]
Here, log2CtuWidthY = log2(ctuWidth), log2CtuHeightY = log2(ctuHeight). The position of the top left CTU of the tile (xTileInCtu, yTileInCtu) and the top left coordinates of the CTU (xCtb, yCtb) are often referenced, so we use the derived values. The above may be stored in a table for use, or the above derivation may be performed each time the table is referenced.

The entire process above is shown in pseudocode below.

for (CtbAddrInRs=0;CtbAddrInRs<numCtusInFrame;CtbAddrInRs++)
{
TileId = CtbAddrRsToTileID[CtbAddrInRs]
xCtb = xTile[TileId]+((CtbAddrInRs-firstCtbAddrInRs[TileId])%PicWidthInCtbsY)* ctuUWidth
yCtb = yTile[TileId]+((CtbAddrInRs-firstCtbAddrInRs[TileId])/PicWidthInCtbsY) * ctuHeight
CtbAddrToCtbX[CtbAddrInRs] = xCtb
CtbAddrToCtbY[CtbAddrInRs] = yCtb
}
Here, numCtusInFrame is the number of CTUs in one picture, and numCtusInFrame=numTileColumns*numTileRows.

(Decoding CTU in case of tiles - modified version)
Regarding the above-described method of decoding the CTU of a tile, a different method may be used to derive xCtb and yCtb instead of using the CTU address tables CtbAddrToCtbX[] and CtbAddrToCtbY[]. An example of such a method is shown below.

The CT information decoding unit 3021 uses a target CTU coordinate derivation unit (not shown) to derive rx and ry, which are the X coordinate (col position of the CTU) and the Y coordinate (row position of the CTU) of the target CTU, from the CTU raster scan address CtbAddrInRs of the target CTU, and derives the top left coordinate (xCtb, yCtb) of the target CTU by referring to the CTU coordinate conversion tables CtbColToCtbX[] and CtbRowToCtbY[].

rx = CtbAddrInRs % PicWidthInCtbsY
ry = CtbAddrInRs / PicWidthInCtbsY
xCtb = CtbColToCtbX[rx]
yCtb = CtbRowToCtbY[ry]
The coding_tree_unit() in Fig. 30(a) is an example of syntax using the derivation method of this modified example. The upper left coordinate (xCtb, yCtb) of the target CT is derived using the CTU coordinate conversion table, and the coding_quadtree The upper left coordinate (xCtb, yCtb) is passed to (). As shown in Figure 30(b), the derivation method may be changed depending on whether the tile is available (tile_enabled_flag is true) or not. .

(CTU coordinate conversion table derivation part)
The CTU coordinate conversion table derivation unit derives the CTU coordinate conversion tables CtbColToCtbX[] and CtbRowToCtbY[] that derive the pixel-unit coordinates (xCtb, yCtb) of the CTU from the X coordinate in CTU unit and the Y coordinate in CTU unit (rx, ry) by the following steps.

The CTU coordinate conversion table derivation unit loops the X coordinate rx in CTU units from 0 to num_tile_columns_minus1, and executes the following steps.

(Step 1x)
The CTU coordinate conversion table derivation unit derives a tile ID from the X coordinate rx in the CTU unit. At this time, since this table is the same for all Y coordinates, the Y coordinate in the CTU unit (row position of the CTU) is set to 0. Therefore,
TileId = CtbAddrRsToTileID[rx]
Here, CtbAddrRsToTileID is a table derived by the tile ID table derivation unit.

(Step 2x)
The CTU coordinate conversion table derivation unit derives the first CTU raster scan address firstCtbAddrInRs[TileId] of the tile including the target CTU, and derives the intra-tile coordinate xCtbInCtus in units of CTU using the following formula.

xCtbInCtus = rx - firstCtbAddrInRs[TileId] % PicWidthInCtbsY
Here, firstCtbAddrInRs is a table for deriving the first CTU raster scan address of the tile indicated by the tile ID, and is derived by the first tile address derivation unit.

Alternatively, it can be derived as follows using the table firstCtbCol[], which derives the CTU-unit X coordinate (col position) of the first CTB of the tile indicated by the tile ID.

xCtbInCtus = rx - firstCtbCol[TileId]
The table firstCtbCol[] is derived in the first tile address derivation unit as follows.

firstCtbCol[TileId] = firstCtbAddrInRs[TileId] % PicWidthInCtbsY
Next, the CTU coordinate conversion table derivation unit multiplies xCtbInCtus by the CTU size ctuWidth to derive the in-tile coordinate xCtbInTile in pixel units using the following formula.

xCtbInTile = xCtbInCtus*ctuWidth
(Step 3x)
The CTU coordinate conversion table derivation unit derives the top-left X coordinate xCtb of the CTU in the picture from the sum of the X coordinate in the tile xCtbInTile and the in-picture coordinate xTile[TileId] of the top-left X coordinate of the tile.

xCtb = xTile[TileId]+xCtbInTile
(Step 4x)
Finally, the CTU coordinate conversion table derivation unit stores the derived upper left X coordinate xCtb of the CTU related to rx in a table.

CtbColToCtbX[rx] = xCtb
The above process can also be expressed as follows, using the X coordinate xTileInCtus of the top left CTU of the tile:

xTileInCtus = firstCtbAddrInRs[TileId] % PicWidthInCtbsY
xCtb = ((rx-xTileInCtus)<<log2CtuWidthY)+xTile[TileId]
Here, log2CtuWidthY = log2(ctuWidth). The X coordinate of the top left corner of the tile, CTU, is xTileInCtus or CTU
Since the upper left X coordinate xCtb is frequently referenced, the derived value may be stored in a table and used, or the above derivation may be performed each time the value is referenced.

The entire process above is shown in pseudocode below.

for ( col=0; col<=num_tile_columns_minus1; col++ )
{
TileId = CtbAddrRsToTileID[col]
xCtb = xTile[TileId]+((rx-firstCtbAddrInRs[TileId])%PicWidthInCtbsY)* ctuWidth
CtbColToCtbX[col] = xCtb
}
As with the table for the X coordinate, the CTU coordinate conversion table derivation unit converts the Y coordinate ry in CTU units into
Loop through num_tile_rows_minus1 and perform the following steps:

(Step 1y)
The CTU coordinate conversion table derivation unit derives a tile ID from the Y coordinate ry in CTU units. At this time, since this table is the same for all X coordinates, the X coordinate in CTU units (col position of the CTU) is set to 0. Therefore,
TileId = CtbAddrRsToTileID[ry*PicWidthInCtbsY]
Here, CtbAddrRsToTileID is a table derived by the tile ID table derivation unit.

(Step 2y)
The CTU coordinate conversion table derivation unit derives the first CTU raster scan address firstCtbAddrInRs[TileId] of the tile including the target CTU, and derives the intra-tile coordinate yCtbInCtus in units of CTU using the following formula.

yCtbInCtus = ry - firstCtbAddrInRs[TileId] / PicWidthInCtbsY
Here, firstCtbAddrInRs is a table for deriving the first CTU raster scan address of the tile indicated by the tile ID, and is derived by the first tile address derivation unit.

Alternatively, it can be derived as follows using the table firstCtbRow[], which derives the CTU-unit Y coordinate (row position) of the first CTB of the tile indicated by the tile ID.

yCtbInCtus = ry - firstCtbRow[TileId]
The table firstCtbRow[] is derived in the first tile address derivation section as follows.

firstCtbRow[TileId] = firstCtbAddrInRs[TileId] / PicWidthInCtbsY
Next, the CTU coordinate conversion table derivation unit multiplies yCtbInCtus by the CTU size ctuHeight to derive the in-tile coordinate yCtbInTile in pixel units using the following formula.

yCtbInTile = yCtbInCtus*ctuHeight
(Step 3y)
The CTU coordinate conversion table derivation unit derives the top left Y coordinate yCtb of the CTU in the picture from the sum of the in-tile Y coordinate yCtbInTile and the in-picture coordinate yTile[TileId] of the top left Y coordinate of the tile.

yCtb = yTile[TileId]+yCtbInTile
(Step 4y)
Finally, the CTU coordinate conversion table derivation unit stores the derived top left Y coordinate yCtb of the CTU related to ry in the table.

CtbRowToCtbY[ry] = yCtb
The above process can also be expressed as follows, using the y position yTileInCtus of the top left CTU of the tile:

yTileInCtus = firstCtbAddrInRs[TileId] / PicWidthInCtbsY
yCtb = ((ry-yTileInCtus)<<log2CtuHeightY)+yTile[TileId]
Here, log2CtuHeightY = log2(ctuHeight). Since the Y coordinate yTileInCtus of the upper left corner of the tile CTU and the Y coordinate yCtb of the upper left corner of the CTU are frequently referenced, it is acceptable to store the derived values in a table and use them. , the above derivation may be performed each time a reference is made.

The entire process above is shown in pseudocode below.

for ( row=0; row<=num_tile_rows_minus1; row++ )
{
TileId = CtbAddrRsToTileID[row*PicWidthInCtbsY]
yCtb = yTile[TileId]+((ry-firstCtbAddrInRs[TileId])/PicWidthInCtbsY)* ctuHeight
CtbRowToCtbY[row] = yCtb
}
In addition, since CtbAddrInTs and CtbAddrInRs can be converted into each other, the table in the above embodiment with the raster scan order CTU position CtbAddrInRs as an index is derived with the tile scan order CTU position CtbAddrInTs as an index, and the index when referring is also the tile scan order. Similarly, a table with the tile scan order CTU position CtbAddrInTs as an index can be derived with the raster scan order CTU position CtbAddrInRs as an index, and the index at the time of reference can be It is also possible to use the CTU positions converted into raster scan order.

An example of derivation of the table CtbAddrRsToTs[] that derives a tile scan order CTU address from a raster scan order CTU address is shown below in pseudo code: CtbAddrRsToTileId[] is a tile ID table that derives a tile ID from a raster scan order CTU address derived by a tile ID table derivation unit.

for( ctbAddrRs = 0; ctbAddrRs <numCtusInFrame; ctbAddrRs++ ) {
tileId = CtbAddrRsToTileId[ ctbAddrRs ]
tbX = ctbAddrRs % PicWidthInCtbsY
tbY = ctbAddrRs / PicWidthInCtbsY
CtbAddrRsToTs[ ctbAddrRs ] = 0
for( t = 0; t <tileId; t++ ) {
CtbAddrRsToTs[ctbAddrRs] += TileWidthInCtbsY[tileId]*TileHeightInCtbsY[tileId]
}
CtbAddrRsToTs[ctbAddrRs] += (tbY-TileYInCtbsY[tileId])*TileWidthInCtbsY[tileId]+
tbX-TileXInCtbsY[tileId]
}
With the above configuration, a video decoding device that divides an image into CTUs and decodes a video in units of CTUs includes a CT information decoding unit that divides tiles into CTUs and recursively divides them, and performs raster scanning of a target CTU. The header decoding unit (target CTU coordinate derivation unit) derives the top left coordinate of the target CTU on the picture by referring to the size of one or more tiles from the address, and the CT information decoding unit Based on the top left coordinate of the CTU on the picture,
The method includes means for dividing a TU into coding trees, and processing the coding tree when the coordinates in the tile of the bottom right position of a divided CT are within the tile size.

A video decoding device that divides an image into tiles and decodes video on a tile-by-tile basis is provided with a header decoding unit (target CTU coordinate derivation unit) that references a CTU address table that derives the top left coordinate of a target CTU from the raster scan address of the target CTU, and a header decoding unit (CTU address table derivation unit) that derives the CTU address table from the size of one or more tiles.

(Tile ID table derivation unit)
The tile ID table derivation unit derives the tile ID table CtbAddrRsToTileID[] in the following steps.

(Step 1)
The tile coordinates (columnIdx, rowIdx) of the CTU corresponding to the on-screen CTU address CtbAddrInRs are derived.

(Step 2)
Derive the in-screen tile ID from the tile coordinates (columnIdx, rowIdx).

TileId = rowIdx*numTileColumns+columnIdx
where numTileColumns is the number of tiles horizontally in the picture.

(Step 3)
The derived tile ID is stored in the tile ID table CtbAddrRsToTileID[].

CtbAddrRsToTileID[CtbAddrInRs] = TileId
The pseudocode for the above process is shown below.

for(CtbAddrInRs=0;CtbAddrInRs<numCtusInFrame;CtbAddrInRs++)
{
for(col=0;col<numTileColumns;col++)
{
if(CtbAddrInRs%PicWidthInCtbsY<=rightEdgePosInCtus[col])
{
columnIdx = col
break
}
}
for(row=0;row<numTileRows;row++)
{
if(CtbAddrInRs/PicWidthInCtbsY<=bottomEdgePosInCtus[row*numTileColumns])
{
rowIdx = row
break
}
}
CtbAddrRsToTileID[CtbAddrInRs] = rowIdx*numTileColumns+columnIdx
}
(First tile address derivation part)
The first tile address derivation unit derives the first tile address table firstCtbAddrInRs[] in the following steps.

(Step 1)
For TileId corresponding to (col, row) in tile coordinates, derive the right edge position in CTU units of the tile indicated by TileId, rightEdgePosInCtus[TileId], by referencing the width in CTU units of each tile i (0<=i<col) up to the col column, tileWidthInCtus[i]. Similarly, derive the bottom edge position in CTU units of the tile indicated by TileId, bottomEdgePosInCtus[TileId], by referencing the height in CTU units of each tile j (0<=j<row) up to the row column.

(Step 2)
The left edge position xCtbInCtus of the tile indicated by TileId in CTU units is derived from the right edge position rightEdgePosInCtus[TileId] of the tile indicated by TileId in CTU units and the width tileWidthInCtus[TileId] of the tile indicated by TileId in CTU units.

The top edge position yCtbInCtus in CTU units of the tile indicated by TileId is derived from the bottom edge position bottomEdgePosInCtus[TileId] in CTU units of the tile indicated by TileId and the height tileHeightInCtus[TileId] in CTU units of the tile indicated by TileId.

xCtbInCtus = rightEdgePosInCtus[TileId]-tileWidthInCtus[TileId]+1
yCtbInCtus = bottomEdgePosInCtus[TileId]-tileHeightInCtus[TileId]+1
The CTU raster scan address CtbAddrInRs is derived from the on-screen position (xCtbInCtus, yCtbInCtus) of the top left pixel of the CTU tile indicated by TileId.

CtbAddrInRs = yCtbInCtus*PicWidthInCtbsY+xCtbInCtus
Here, tileWidthInCtus[] and tileHeightInCtus[] are derived by the tile information setting unit.

(Step 3)
Store the derived CTU address in the first tile address table firstCtbAddrInRs[]
firstCtbAddrInRs[TileId] = CtbAddrInRs
The pseudocode for the above process is shown below.

for(row=0; row <numTileRows; row++)
{
for(col=0; col <numTileColumns; col++)
{
TileIdx = row*numTileColumns+col;
rightEdgePosInCTU = 0;
for(i=0; i <= col; i++)
{
rightEdgePosInCTU += tileWidthInCtus[row*numTileColumns+i]
}
rightEdgePosInCtus[TileId] = rightEdgePosInCTU-1
bottomEdgePosInCTU = 0;
for(i=0 ; i <= row ; i++)
{
bottomEdgePosInCTU += tileHeightInCtus[i*numTileColumns+col]
}
bottomEdgePosInCtus[tileId] = bottomEdgePosInCTU-1
xCtbInCtus = rightEdgePosInCtus[TileId]-tileWidthInCtus[TileId]+1
yCtbInCtus = bottomEdgePosInCtus[TileId]-tileHeightInCtus[TileId]+1
CtbAddrInRs = yCtbInCtus*PicWidthInCtbsY+xCtbInCtus
firstCtbAddrInRs[TileId] = CtbAddrInRs
}
}
(CT information decoding process)
The CT information decoding process will be described below with reference to Fig. 7, Fig. 8, and Fig. 9. Fig. 7 is a flowchart illustrating the operation of the CT information decoding unit 3021 according to one embodiment of the present invention. Also, Figure 8 shows
FIG. 1 is a diagram showing an example of a syntax table of CTU and QT information according to an embodiment of the present invention;
Fig. 9 is a diagram showing an example of the structure of a syntax table of MT division information according to an embodiment of the present invention. Other examples of Figs. 7 to 9 are shown in Figs. 27 to 29.

In the decoding of the coding_tree_unit() syntax in Fig. 8(a), the CT information decoding derives the top left coordinate (xCtb, yCtb) of the target CTU, and recursively decodes the coding_quadtree() syntax. The initial value of the top left coordinate (xCtb, yCb) of the CT of coding_quadtree() is the top left coordinate (xCtb, yCtb) of the CTU.
As another method of deriving (xCtb, yCtb), as shown in FIG. 27(b), the derivation method may be changed depending on whether tiles are used (tile_enabled_flag is true) or not.
That is, when tiles are used, the CTU top-left coordinate (xCtb, yCtb) is derived from the sum of the top-left coordinate of the tile on the screen (xTile[TileId], yTile[TileId]) and the top-left coordinate of the CTU in the tile. In other words, the top-left coordinate of the CTU in a tile is derived from the difference between the in-screen coordinate (rx, ry) of the CTU in CTU unit and the in-screen coordinate (TileXInCtbY, TileYInCtbY) of the tile in CTU unit, and then multiplied by the CTU size (shifted left by CtbLog2SizeY).

CurrTileId = TileIdTbl[ CtbAddrInRs ]
xCtb = xTile[TileId]+(rx-TileXInCtbsY[CurrTileId]) <<CtbLog2SizeY
yCtb = yTile[TileId]+(ry-TileYInCtbsY[CurrTileId]) <<CtbLog2SizeY
Conversely, when tiles are not used, the top left coordinates of the CTU (xCtb, yCtb) are derived from the screen coordinates of the CTU in the CTU unit. In other words, the top left coordinates of the CTU in the tile are calculated from the screen coordinates of the CTU in the CTU unit. Multiply the coordinates (rx, ry) by the CTU size (shift left by CtbLog2SizeY).

rx = CtbAddrInRs % PicWidthInCtbsY
ry = CtbAddrInRs / PicWidthInCtbsY
xCtb = rx << CtbLog2SizeY
yCtb = ry << CtbLog2SizeY
Even when tiles are not used, the entire screen may be regarded as one tile and processing may be performed regardless of whether tiles are used or not.

(Decryption below CTU)
(CT division of flexible tiles using intra-picture coordinate system)
The CT information decoding unit 3021 decodes the CT information from the coded data, and recursively decodes the coding tree CT (coding_quadtree). Specifically, the CT information decoding unit 3021 decodes the QT information, and decodes the target CT coding_quadtree (x0, y0, log2CbSize, cqtDepth). Note that (x0, y0) is the upper left coordinate of the target CT, log2CbSize is the logarithm of the CT size, which is the size of the CT, and cqtDepth is the CT depth (QT depth) indicating the hierarchy of the CT.

(S1411) The CT information decoding unit 3021 determines whether or not the decoded CT information has a QT division flag. If the QT division flag is present, the process proceeds to S1421, and otherwise the process proceeds to S1422.

(S1421) If it is determined that the logarithmic CT size log2CbSize is larger than MinCbLog2SizeY, the CT information decoding unit 3021 decodes the QT split flag (split_cu_flag).
Here, when tiles are used, as shown in the formula below, split_cu_flag is notified, indicating whether or not to perform further quadtree division, taking into consideration the upper left coordinates (xCtb, yCtb) of the CTU and the tile size.

if (x0+(1<<log2CbSize)-xTile<=wT &&y0+(1<<log2CbSize)-yTile<=hT&&log2CbSize>MinCbLog2SizeY)
split_cu_flag[x0][y0]
Here, (x0, y0) is the upper left coordinate of the block, (xTile, yTile) is the upper left coordinate of the tile, log2CbSize is the logarithmic value of the block size, wT and hT are the widths of the tile effective area (or tile coding area), The height, MinCbLog2SizeY, is the logarithm of the minimum size of a block.

If the coordinate x0 + (1 << log2CbSize) of the right edge of the block and the coordinate y0 + (1 << log2CbSize) of the bottom edge are smaller than the coordinate xTile + wTile of the right edge of the tile valid area and the coordinate yTile + hTile of the bottom edge of the tile valid area, the target block exists within the tile valid area. If the block exists within the tile and the block size is larger than the minimum value (log2CbSize > MinCbLog2SizeY), a flag split_cu_flag is notified indicating whether the block should be further divided. If the block should be further quadtree divided, split_cu_flag is set to 1, and if the block should not be quadtree divided, split_cu_flag is set to 0.

(S1422) In other cases, the CT information decoding unit 3021 omits decoding of the QT split flag split_cu_flag from the encoded data, and sets the QT split flag split_cu_flag to 0.

(S1450) If the QT split flag split_cu_flag is other than 0, the process transitions to S1451; otherwise, the process transitions to S1471.

(S1451) The CT information decoding unit 3021 performs QT division. Specifically, the CT information decoding unit 3021 decodes four CTs with a logarithmic CT size log2CbSize-1 at positions (x0, y0), (x1, y0), (x0, y1), and (x1, y1) of the CT depth cqtDepth+1. The CTs whose bottom right coordinates (xi-xTile[TileId], yi-yTile[TileId]) of each CT in the tile are less than the tile size (wTile[TileId], hTile[TileId]) are decoded. Conversely, the CTs whose bottom right coordinates (xi-xTile[TileId], yi+yTile[TileId]) of each CT are equal to or greater than the tile size (wTile[TileId], hTile[TileId]) are not decoded. That is, the CT information decoding unit 3021 recursively divides the target CTU into a coding tree, and processes the coding tree if the top left coordinates (x1, y0), (x0, y1), (x1, y1) of the divided CT are within the target tile.

coding_quadtree(x0,y0,log2CbSize-1,cqtDepth+1)
if (x1-xTile[TileId] < wTile[TileId])
coding_quadtree(x1,y0,log2CbSize-1,cqtDepth+1)
if (y1-yTile[TileId] < hTile[TileId])
coding_quadtree(x0,y1,log2CbSize-1,cqtDepth+1)
if (x1-xTile[TileId] < wTile[TileId] && y1-yTile[TileId] < hTile[TileId])
coding_quadtree(x1,y1,log2CbSize-1,cqtDepth+1)
In the above example, the block located at (x1, y0) obtained by quadtree division, coding_quadtree(x1, y0, log2CbSize-1, cqtDepth+1, wTile, hTile, xTile, yTile) is , is encoded or decoded if x1 is located within the tile as follows:

if (x1-xTile[TileId]<wTile[TileId])
coding_quadtree(x1,y0,log2CbSize-1,cqtDepth+1)
Here, (x0, y0) is the upper left coordinate of the target CT, and (x1, y1) is calculated by adding 1/2 of the CT size (1<<log2CbSize) to (x0, y0) as shown in the following formula. do.

x1 = x0 + (1<<(log2CbSize-1))
y1 = y0 + (1 << (log2CbSize-1))
1<<N is equivalent to 2 to the Nth power, and so on.

Then, the CT information decoding unit 3021 updates the CT depth cqtDepth and the logarithmic CT size log2CbSize, which indicate the CT hierarchy, as shown in the following formula.

cqtDepth = cqtDepth+1
log2CbSize = log2CbSize-1
The CT information decoding unit 3021 continues the QT information decoding started from S1411 using the updated upper left coordinate, logarithmic CT size, and CT depth for the lower CTs as well.

After QT division is completed, the CT information decoding unit 3021 decodes the CT information from the encoded data, and recursively decodes the coding tree CT (MT, coding_multitree (Figure 9) or multi_type_tree (Figure 29)). Specifically, the CT information decoding unit 3021 decodes the MT division information, and decodes the target CT coding_multitree(x0, y0, log2CbWidth, log2CbHeight, cbtDepth) or multi_type_tree(x0, y0, cbWidth, cbHeight, mttDepth, depthOffset, partIdx, treeType). Note that log2CbWidth is the logarithmic value of the CT width, log2CbHeight is the logarithmic value of the CT height, and cbtDepth is the CT depth (MT depth) that indicates the hierarchy of the multitree. Below, coding_multitree is explained.

(S1471) The CT information decoding unit 3021 determines whether or not the decoded CT information has an MT division flag (division information). If the MT division flag is present, the process proceeds to S1481. Otherwise, the process proceeds to S1482.

(S1481) The CT information decoding unit 3021 decodes the MT split flag split_mt_flag.

(S1482) The CT information decoding unit 3021 does not decode the MT split flag split_mt_flag from the encoded data, but sets it to 0.

(S1490) The CT information decoding unit 3021 transitions to S1491 if the MT split flag split_mt_flag is other than 0. In other cases, the CT information decoding unit 3021 does not split the target CT and ends the process (transitions to decoding of the CU).

(S1491) The CT information decoding unit 3021 performs MT division. The flag split_mt_dir indicating the direction of MT division is set to
and decodes a syntax element split_mt_type indicating whether the MT division is a binary tree or a ternary tree. The CT information decoding unit 3021 processes the coding tree when the top left coordinate of the divided CT is within the target tile. The top left coordinate of the CT is (x0, y1) or (x1, y0) in a binary tree, and (x0, y1), (x0, y2) or (x1, y0), (x2, y0) in a ternary tree. Here, if the top left coordinate (x0, y0) of the leading CT is outside the tile, the division itself is not performed, so it is assumed that (x0, y0) is always within the tile.

When the MT split type split_mt_type is 0 (two splits) and the MT split direction split_dir_flag is 1 (horizontal split), the CT information decoding unit 3021 decodes the following two CTs (BT split information decoding). It decodes the CT whose bottom coordinate y1-yTile[TileId] of CT(x0, y1) in the tile is less than the tile size hTile[TileId]. Conversely, it does not decode the CT whose bottom coordinate y1-yTile[TileId] of each CT in the tile is equal to or greater than the tile size hTile[TileId].

coding_multitree(x0,y0,log2CbWidth,log2CbHeight-1,cbtDepth+1)
if (y1-yTile[TileId] < hTile[TileId])
coding_multitree(x0,y1,log2CbWidth,log2CbHeight-1,cbtDepth+1)
On the other hand, if the MT split direction split_dir_flag is 0 (vertical split), the following two CTs are decoded (
Decode the CT whose right coordinate x1-xTile[TileId] of CT(x1, y0) in the tile is less than the tile size wTile[TileId]. Conversely, A CT whose right coordinate x1-xTile[TileId] is equal to or larger than the tile size wTile[TileId] is not decoded.

coding_multitree(x0,y0,log2CbWidth-1,log2CbHeight,cbtDepth+1)
if (x1-xTile[TileId] < wTile[TileId])
coding_multitree(x1,y0,log2CbWidth-1,log2CbHeight,cbtDepth+1)
Here, (x1, y1) is derived using the following formula:

x1 = x0 + (1<<(log2CbWidth-1))
y1 = y0 + (1 << (log2CbHeight - 1))
Furthermore, log2CbWidth or log2CbHeight is updated according to the following formula.

log2CbWidth = log2CbWidth-1
log2CbHeight = log2CbHeight-1
When the MT division type split_mt_type indicates 1 (3 divisions), the CT information decoding unit 3021
Decode one CT (TT division information decode).

When the MT split direction split_dir_flag is 1 (horizontal split), the following three CTs are decoded. CTs whose bottom coordinate yi-yTile[TileId] of each CT(x0, yi) for i=0, 1 in the tile is less than the tile size hTile[TileId] are decoded. Conversely, CTs whose bottom coordinate yi-yTile[TileId] of each CT in the tile is equal to or greater than the tile size hTile[TileId] are not decoded.

coding_multitree(x0,y0,log2CbWidth,log2CbHeight-2,cbtDepth+1)
if (y1-yTile[TileId] < hTile[TileId])
coding_multitree(x0,y1,log2CbWidth,log2CbHeight-1,cbtDepth+1)
if (y2-yTile[TileId] < hTile[TileId])
coding_multitree(x0,y2,log2CbWidth,log2CbHeight-2,cbtDepth+1)
On the other hand, if the MT split direction split_dir_flag is 1 (vertical split), the following three CTs are decoded (TT split information decoded). Decode CTs whose coordinates xi-xTile[TileId] are less than the tile size wTile[TileId]. Conversely, CTs whose right coordinates xi-xTile[TileId] of each CT in the tile are equal to or greater than the tile size wTile[TileId] are decoded. Do not decrypt.

coding_multitree(x0,y0,log2CbWidth-2,log2CbHeight,cbtDepth+1)
if (x1-xTile[TileId] < wTile[TileId])
coding_multitree(x1,y0,log2CbWidth-1,log2CbHeight,cbtDepth+1)
if (x2-xTile[TileId] < wTile[TileId])
coding_multitree(x2,y0,log2CbWidth-2,log2CbHeight,cbtDepth+1)
Here, (x1, y1) and (x2, y2) are derived as follows:

x1 = x0 + (1<<(log2CbWidth-2))
y1 = y0 + (1 << (log2CbHeight - 2))
x2 = x0 + (3<<(log2CbWidth-2))
y2 = y0 + (3 << (log2CbHeight - 2))
The CT information decoding unit 3021 continues the BT partition information decoding started from S1471 or the TT partition information decoding, using the updated top left coordinate, CT width and height, and MT depth, even for the lower CT.

In addition, when the MT split flag split_mt_flag is 0, that is, when neither QT splitting nor MT splitting is performed, the CT information decoding unit 3021 decodes a CU (coding_unit(x0, y0, log2CbSize, cq
tDepth)

In the following, the in-screen coordinate system refers to using the luminance top-left coordinate of the target block based on the top-left coordinate of the target screen, and the in-tile coordinate system may refer to using the luminance top-left coordinate of the target block based on the top-left coordinate of the target tile.

As described above, the top left coordinate of the tile (xTile, yTile) is subtracted from the top left coordinate (xi, yi) of each CU expressed in the on-screen coordinate system to derive the top left coordinate in the tile coordinate system, and this is compared with the tile size (wTile, hTile) to decode the split flag and decode each CT. This has the effect of making it possible to efficiently split CUs without decoding unnecessary split flags and unnecessary CTs, even when the tile size is not an integer multiple of the CTU size.

(Another example of tiling)
Another example of tile division will be described using the syntax table in FIG. 31. In this example, the screen is divided into rectangular tiles, and encoded data is constructed using a unit in which tiles are groups. The upper left coordinates of the tile are xTile, yTile, and the tile width and height are wTile, hTile. The CTU address ctbAddrInTs of the first tile is derived from the table FirstCtbAddrTs that stores the start position of the tile and the tile group position tile_group_address. Tiles from 0 to num_tiles_in_tile_group_minus1 are decoded. num_tiles_in_tile_group_minus1 is a value obtained by subtracting 1 from the number of tiles in the picture. CTUs from 0 to NumCtusInTile[tileIdx] are decoded for each tile indicated by TileId. At the end of the tile, end_of_tile_one_bit and the byte alignment bit byte_alignment() are decoded. The CTU is decoded using the CTU position j in the tile as an argument to coding_tree_unit, where j is set as the value of CtbAddrInTile.

Fig. 31B shows an example of decoding each CTU using the CTU position CtbAddrInTile in the tile coordinate system.
The position (rx, ry) of the CTU unit within the tile and the position (xCtb, yCtb) of the pixel unit within the tile are derived from the unit size TileWidthInCtus. The CTU is decoded using the derived CTU position in the tile coordinate system as follows: Note that the raster position CtbAddrInTile of the CTU unit within the tile may also be derived by subtracting the start position of the tile FirstCtbAddrTs from the CTU position CtbAddrInTs in screen units.

CtbAddrInTile = CtbAddrInTs - FirstCtbAddrTs[tile_group_id]
(Decryption below CTU)
(CT division of flexible tile using intra-tile coordinate system)
The operation of the CT information decoding unit 3021 when decoding the syntax tables of Figs. 7, 32, and 33 will be described. Using the intra-tile coordinate system with the top left corner of the tile as the origin, the CTU coordinates ( The CT information is decoded by deriving the CTU coordinates (xCtb, yCtb) and recursively decoding the CT information and MT division information based on the CTU coordinates and the tile size (wTile, hTile).

First, the CT information decoding unit 3021 decodes the CT information from the coded data, and recursively decodes the coding tree CT (coding_quadtree). Specifically, the CT information decoding unit 3021 decodes the QT information, and decodes the target CT coding_quadtree (x0, y0, log2CbSize, cqtDepth, treeType).

(S1411) The CT information decoding unit 3021 determines whether or not the decoded CT information has a QT division flag. If the QT division flag is present, the process proceeds to S1421, and otherwise the process proceeds to S1422.

(S1421) If it is determined that the logarithmic CT size log2CbSize is larger than MinCbLog2SizeY, the CT information decoding unit 3021 decodes the QT split flag (qt_split_cu_flag).
Here, as shown in the formula below, when tiles are used, the qt_split function is used to determine whether or not to perform further quadtree division, taking into account the upper left coordinate (x0, y0) of CT in the tile coordinate system and the tile size.
_cu_flag is notified.

if (((x0+(1<<log2CbSize) <= wTile[TileId])?1:0)+
((y0+(1<<log2CbSize) <= hTile[TileId])?1:0)+
(((1<<log2CbSize) <= MaxBtSizeY)?1:0)) >= 2 &&
log2CbSize > MinQtLog2SizeY)
qt_split_cu_flag[x0][y0]
In other words, if the coordinate x0+(1<<log2CbSize) of the right edge of the block and the coordinate y0+(1<<log2CbSize) of the bottom edge are both less than the tile width wTile and tile height hTile,
(x0+(1<<log2CbSize) <= wTile[TileId])?1:0)+
((y0+(1<<log2CbSize) <= hTile[TileId])?1:0)>=2
Or, if the coordinate x0+(1<<log2CbSize) of the right edge of the block is less than or equal to the tile width wTile and the CT size is greater than or equal to the maximum BT size MaxBtSizeY,
(x0+(1<<log2CbSize) <= wTile[TileId])?1:0)+
(((1<<log2CbSize) <= MaxBtSizeY)?1:0)) >= 2
Or, if the bottom coordinate y0+(1<<log2CbSize) of the block is less than or equal to the tile height hTile and the CT size is greater than or equal to the maximum BT size MaxBtSizeY,
(y0+(1<<log2CbSize) <= hTile[TileId])?1:0)+
(((1<<log2CbSize) <= MaxBtSizeY)?1:0)) >= 2
If so, the QT split flag qt_split_cu_flag is decoded from the encoded data.

(S1422) In other cases, the CT information decoding unit 3021 omits decoding of the QT split flag qt_split_cu_flag from the encoded data, and derives the QT split flag qt_split_cu_flag using the tile size (wTile, hTile) as follows.

If any of the following is true, qt_split_cu_flag is set to 1.

x0+(1<<log2CbSize) > wTile[TileId] &&(1<<log2CbSize)> MaxBtSizeY
y0+(1<<log2CbSize) > hTile[TileId] &&(1<<log2CbSize)> MaxBtSizeY
If all of the following are true, set qt_split_cu_flag to 1.

x0+(1<<log2CbSize) > wTile[TileId]
y0+(1<<log2CbSize) > hTile[TileId]
(1<<log2CbSize) > MinQtSizeY
In all other cases, 1 is derived for qt_split_cu_flag.

In other words, if the CT right coordinate exceeds the tile size width and the CT size exceeds the maximum BT split size MaxBtSizeY, it will always be split (1 is derived for qt_split_cu_flag).

If the CT bottom coordinate exceeds the tile size height and the CT size exceeds the maximum BT split size MaxBtSizeY, it will always be split (1 is derived for qt_split_cu_flag).

Split is always performed if the bottom right coordinate of the CT exceeds the tile size (1 is derived for qt_split_cu_flag).

(S1450) If the QT split flag split_cu_flag is other than 0, the process transitions to S1451; otherwise, the process transitions to S1471.

(S1451) The CT information decoding unit 3021 performs QT division. Specifically, the CT information decoding unit 3021 performs QT division on the logarithmic CT size log2
Four CTs of CbSize-1 are decoded. CTs whose bottom right coordinates (xi-xTile[TileId], yi+yTile[TileId]) of each CT in the tile are equal to or smaller than the tile size (wTile[TileId], hTile[TileId]) are decoded. Conversely, CTs whose bottom right coordinates (xi, yi) of each CT are larger than the tile size (wTile[TileId], hTile[TileId]) are not decoded. That is, the CT information decoding unit 3021 recursively divides the target CTU into coding trees, and processes the coding trees when the top left coordinates (x1, y0), (x0, y1), and (x1, y1) of the divided CTs are within the target tile.

coding_quadtree(x0,y0,log2CbSize-1,cqtDepth+1)
if (x1 < wTile[TileId])
coding_quadtree(x1,y0,log2CbSize-1,cqtDepth+1)
if (y1 < hTile[TileId])
coding_quadtree(x0,y1,log2CbSize-1,cqtDepth+1)
if (x1 < wTile[TileId] && y1 < hTile[TileId])
coding_quadtree(x1,y1,log2CbSize-1,cqtDepth+1)
In the above, for example, the block located at (x1, y0) obtained by quadtree division, coding_quadtree(x1, y0, log2CbSize-1, cqtDepth+1), is determined as follows: The signal is encoded or decoded when it is located.

if (x1<wTile[TileId])
coding_quadtree(x1,y0,log2CbSize-1,cqtDepth+1)
Here, (x0, y0) is the upper left coordinate of the target CT, and (x1, y1) is calculated by adding 1/2 the CT size (1<<log2CbSize) to (x0, y0) as shown in the following formula.

x1 = x0 + (1<<(log2CbSize-1))
y1 = y0 + (1 << (log2CbSize-1))
1<<N is equivalent to 2 to the Nth power, and so on.

Then, the CT information decoding unit 3021 updates the CT depth cqtDepth and the logarithmic CT size log2CbSize, which indicate the CT hierarchy, as shown in the following formula.

cqtDepth = cqtDepth+1
log2CbSize = log2CbSize-1
The CT information decoding unit 3021 continues the QT information decoding started from S1411 using the updated upper left coordinate, logarithmic CT size, and CT depth for the lower CTs as well.

After QT division is completed, the CT information decoding unit 3021 decodes CT information from the encoded data and recursively decodes the coding tree CT (MT, multi_type_tree). Specifically, the CT information decoding unit 3021 decodes MT division information using the coordinates in the tile and the tile size, and decodes the target CT multi_type_tree (x0, y0, CbWidth, CbHeight, mttDepth, depthOffset, partIdx, treeType). Note that CbWidth is the width of the CT, CbHeight is the height of the CT, and mttDepth is the CT depth (MT depth) indicating the hierarchy of the multi-tree.

(S1471) The CT information decoding unit 3021 determines whether or not the decoded CT information has an MT split flag (split information, i.e., mtt_split_cu_flag, mtt_split_cu_vertical_flag, mtt_split_cu_binary_flag). If the MT split flag is present as determined below, the process transitions to S1481. Otherwise, the process transitions to S1482.

(allowSplitBtVer || allowSplitBtHor || allowSplitTtVer || allowSplitTtHor)
&&(x0+cbWidth<=wTile[TileId])&&(y0+cbHeight<=hTile[TileId])
Here, allowSplitBtVer and allowSplitBtHor are derived by the following process.

If any of the following is true, set allowBtSplit to false:
・cbSize <= MinBtSizeY
・cbWidth > MaxBtSizeY
・cbHeight > MaxBtSizeY
・mttDepth >= MaxMttDepth + depthOffset
If all of the following are true except for the above, set allowBtSplit to false.
・btSplit = SPLIT_BT_VER
・y0 + cbHeight > hTile[TileId]
If all of the following are true except for the above, set allowBtSplit to false.
・btSplit = SPLIT_BT_HOR
・x0 + cbWidth > wTile[TileId]
If all of the following are true except for the above, set allowBtSplit to false.
・mttDepth > 0
・partIdx = 1
・MttSplitMode[ x0 ][ y0 ][ mttDepth - 1 ] = parallelTtSplit
Otherwise, set allowBtSplit to true.
・allowBtSplit = TRUE
Here, cbSize and parallelTtSplit are set as follows:

If SPLIT_BT_VER(allowSplitBtVer), cbSize=cbWidth, parallelTtSplit= SPLIT_TT_VER
If SPLIT_BT_HOR(allowSplitBtHor), cbSize=cbHeight, parallelTtSplit=SPLIT_TT_HOR
Note that allowSplitTtVer and allowSplitTtHor are derived by the following process.

If any of the following is true, set allowTtSplit(allowTtSplirHor, allowTtSplirVer) to false.
・cbSize <= 2 * MinTtSizeY
・cbWidth > MaxTtSizeY
・cbHeight > MaxTtSizeY
・mttDepth >= MaxMttDepth + depthOffset
・x0 + cbWidth > wTile[TileId]
・y0 + cbHeight > hTile[TileId]
In all other cases, allowTtSplit(allowTtSplirHor, allowTtSplirVer) is set to true.
・allowTtSplit = TRUE
Here, cbSize is set as follows:

If SPLIT_TT_VER(allowSplitTtVer), cbSize=cbWidth
If SPLIT_TT_HOR(allowSplitTtHor), cbSize=cbHeight
That is, whether or not to code the MT split flag is determined depending on whether the CT right coordinate exceeds the tile size width or the CT bottom coordinate exceeds the tile size height.

(S1481) The CT information decoding unit 3021 decodes the MT split flag mtt_split_cu_flag.

(S1482) The CT information decoding unit 3021 does not decode the MT split flag mtt_split_cu_flag from the encoded data, but derives it as follows.

If any of the following is true, derive 1 for mtt_split_cu_flag.
・x0 + cbWidth > wTile[TileId]
・x0 + cbHeight > hTile[TileId]
In other cases, 0 is derived for mtt_split_cu_flag. In other words, if the right and bottom coordinates of the CT in the tile coordinate system exceed the tile size, the MT split flag is not decoded.

In other words, whether or not to encode the TT split flag is determined depending on whether the CT right coordinate exceeds the tile size width or the CT bottom coordinate exceeds the tile size height.

(S1490) The CT information decoding unit 3021 transitions to S1491 if the MT split flag split_mt_flag is other than 0. In other cases, the CT information decoding unit 3021 does not split the target CT and ends the process (transitions to decoding of the CU).

(S1491) The CT information decoding unit 3021 performs MT division. If BT division or TT division is possible (allowSplitBtHor ∥ allowSplitTtHor ∥ ( allowSplitBtVer ∥ allowSplitTtVer )), the flag mtt_split_cu_vertical_flag indicating the MT division direction is decoded. In the decoded division method mtt_split_cu_vertical_flag, if both BT division and TT division are possible ((allowSplitBtVer && allowSplitTtVer && mtt_split_cu_vertical_flag) || (allowSplitBtHor&& allowSplitTtHor && !mtt_split_cu_vertical_flag), the syntax element mtt_split_cu_binary_flag indicating whether the MT division is a binary tree or a ternary tree is decoded. The CT information decoding unit 3021 processes the coding tree when the upper left coordinate of the divided CT is within the target tile. The upper left coordinate of the CT is (x0, y1) or (x1, y0) in a binary tree, and (x0, y1), (x0,
The coordinates are (x1, y0), (x2, y0). If the top left coordinate (x0, y0) of the first CT is outside the tile, the division is not performed, so it is assumed that (x0, y0) is always inside the tile.

When the MT split type split_mt_type is 0 (two splits) and the MT split direction split_dir_flag is 1 (horizontal split), the CT information decoding unit 3021 decodes the following two CTs (BT split information decoding). It decodes the CT whose bottom coordinate y1 of CT(x0,y1) in the tile is less than the tile size hTile[TileId]. Conversely, it does not decode the CT whose bottom coordinate y1 of each CT in the tile is equal to or greater than the tile size hTile[TileId].

multi_type_tree(x0,y0,cbWidth,cbHeight/2,mttDepth,depthOffset,0,treeType)
if (y1 < hTile[TileId])
multi_type_tree(x0,y1,cbWidth,cbHeight/2,mttDepth+1,depthOffset,1,treeType)
On the other hand, if the MT split direction split_dir_flag is 0 (vertical split), the following two CTs are decoded (BT split information decoded). The right coordinate x1 of CT(x1, y0) in the tile is the tile size wTile[ Conversely, the right coordinate x1 of each CT in the tile is decoded if it is smaller than the tile size wTile[TileId].
The above CTs will not be decoded.

multi_type_tree(x0,y0,cbWidth/2,cbHeight,mttDepth+1,depthOffset,0,treeType)
if (x1 < wTile[TileId])
multi_type_tree(x1, y0, cbWidth/2, cbHeightY, mttDepth+1, depthOffset, 1, treeType) Here, (x1, y1) is derived by the following formula.

x1 = x0 + (cbWidth/2)
y1 = y0 + (cbHeight/2)
Furthermore, log2CbWidth or log2CbHeight is updated according to the following formula.

CbWidth = CbWidth/2
CbHeight = CbHeight/2
When the MT division type split_mt_type indicates 1 (3 divisions), the CT information decoding unit 3021
Decode one CT (TT division information decode).

When the MT split direction split_dir_flag is 1 (horizontal split), the following three CTs are decoded. CTs whose bottom coordinate yi of each CT(x0, yi) for i=0, 1 in the tile is less than the tile size hTile[TileId] are decoded. Conversely, CTs whose bottom coordinate yi of each CT in the tile is equal to or greater than the tile size hTile[TileId] are not decoded.

multi_type_tree(x0,y0,cbWidth,cbHeight/4,mttDepth+1,depthOffset,0,treeType)
if (y1 < hTile[TileId])
multi_type_tree(x0,y1,cbWidth,cbHeight/2,mttDepth+1,depthOffset,1,treeType)
if (y2 < hTile[TileId])
multi_type_tree(x0,y2,cbWidth,cbHeight/4,mttDepth+1,depthOffset,2,treeType)
On the other hand, if the MT split direction split_dir_flag is 1 (vertical split), the following three CTs are decoded (TT split information decoded). Decode CTs whose coordinate xi is less than the tile size wTile[TileId]. Conversely, do not decode CTs whose right coordinate xi of each CT in the tile is equal to or greater than the tile size wTile[TileId].

multi_type_tree(x0,y0,cbWidth/4,cbHeight,mttDepth+1,depthOffset,0,treeType)
if (x1 < wTile[TileId])
multi_type_tree(x1,y0,cbWidth/2,cbHeight,mttDepth+1,depthOffset,1,treeType)
if (x2 < wTile[TileId])
multi_type_tree(x2,y0,cbWidth/4,cbHeight,mttDepth+1,depthOffset,2,treeType)
Here, (x1, y1) and (x2, y2) are derived as follows:

x1 = x0 + (cbHeight/4)
y1 = y0 + (3 * cbHeight / 4)
x2 = x0 + (cbWidth/4)
y2 = y0 + (3 * cbWidth / 4)
The CT information decoding unit 3021 continues the BT partition information decoding started from S1471 or the TT partition information decoding, using the updated top left coordinate, CT width and height, and MT depth, even for the lower CT.

Furthermore, when the MT split flag split_mt_flag is 0, that is, when neither QT splitting nor MT splitting is performed, the CT information decoding unit 3021 decodes the CU (coding_unit(x0, y0, cbWidth, cbHeight, treeType)) in the CU decoding unit 3022.

As described above, the split flag is decoded and each CT is decoded by comparing the top left coordinates (xi, yi) of each CU represented in a tile with the tile size (wTile, hTile). This provides the advantage that, even when the tile size is not an integer multiple of the CTU size, it is possible to perform CU splitting efficiently without decoding unnecessary split flags and unnecessary CTs. Furthermore, by performing CU splitting using the intra-tile coordinate system, it is possible to perform CU splitting with simple processing without converting from the intra-screen coordinate system to the intra-tile coordinate system.

The parameter decoding unit 302 includes an inter-prediction parameter decoding unit 303 and an intra-prediction parameter decoding unit 304, both of which are not shown. The predicted image generating unit 308 includes an inter-prediction image generating unit 309 and an intra-prediction image generating unit 310, both of which are not shown.

(Multi-line intra prediction in flexible tiles)
FIG. 35 is a diagram showing multi-line intra-prediction. The intra-prediction parameter decoding unit 304
The intra prediction parameter coding unit 113 decodes or codes the multi-line intra prediction parameters. The intra prediction image generation unit 310 selects which line to reference from among the reference pixels adjacent to the current block boundary according to the syntax element intra_luma_ref_idx. For example, as shown in FIG. 35(a), when intra_luma_ref_idx is 0, 1, or 2, the index IntraLumaRefLineIdx (refIdx) indicating the line of the reference picture is set to 0, 1, or 3. That is, lines 0, 1, and 3 are referenced in order of proximity from the current block.

35(b) shows a case where the Y coordinate of the current block abuts on the CTU boundary. In this case, the syntax element intra_luma_ref_idx is not decoded and 0 is inferred.

Fig. 36 is a flowchart showing a process of multi-line intra prediction in a case where a picture is divided into tiles of a size not limited to an integer multiple of a CTU. Fig. 41(a) is an example of a syntax table.

S4001: The header decoding unit 3020 and the header encoding unit 1110 decode or encode the tile size in units equal to or smaller than the CTU size (tile size not limited to integer multiples of the CTU size). For example, as already explained, the tile_unit_size_idc may be decoded to derive the tile unit size wUnitTile or hUnitTile, and a tile size that is a multiple of wUnitTile or hUnitTile may be decoded.

S4002: The CT information decoding unit 3021 and the CT information encoding unit 1111 (CT division unit) perform CT division using the coordinate system in the tile. For example, as already described, CT division is performed using the syntax configuration shown in Figs. 31 to 33. At this time, the upper left coordinate (x0, y0) of CT, that is, the argument (x0, y0) of QT division coding_quadtree(x0, y0, ...) and MTT division multi_type_tree(x0, y0, ...) uses coordinates with the upper left coordinate of the tile as the origin. In addition, the division syntax qt_split_cu_flag and mtt_split_cu_flag refer to the coordinate (x0, y0) in the tile and the tile sizes wTile and hTile to determine whether or not to perform decoding or encoding.

S4004: The intra prediction parameter decoding unit 304 and the intra prediction parameter coding unit 113 use the coordinate values (x0, y0) of the current block in the intra-tile coordinate system to determine whether the current block is below the CTU boundary. If (y0 % ctuSize)>0, it is determined that the current block is below the CTU boundary. If the current block is below the CTU boundary, the process proceeds to S4005, where the intra reference line index intra_luma_ref_idx is decoded or coded. Otherwise, if the current block is in contact with the CTU boundary, intra_luma_ref_idx is not decoded and is inferred as 0.

S4005: The intra prediction parameter decoding unit 304 and the intra prediction parameter encoding unit 113 decode or encode the intra reference line index intra_luma_ref_idx.

In addition, in intra prediction, in addition to the intra reference line index intra_luma_ref_idx, a flag intra_luma_mpm_flag indicating whether the mode is one of the estimated intra prediction modes Most Probable Modes (MPM), an index intra_luma_mpm_idx selecting an MPM, and an index intra_luma_mpm_remainder indicating the intra prediction mode when the mode is not MPM may also be decoded or encoded.

The intra-prediction image generation unit 310 uses the derived IntraLumaRefLineIdx(refIdx) to derive reference picture samples p[x][y] in the following range:

p[x][y] with x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y =
-1-refIdx
Here, refW and refH are derived using the following formula.

refW = (nTbH>nTbW) ? (nTbW+(nTbH>>whRatio)+Ceil(nTbH/32)) : (nTbW*2)
refH = (nTbW>nTbH) ? (nTbH+(nTbW>>whRatio)+Ceil(nTbW/32)) : (nTbH*2)
Here, nTbW and nTbH are the size of the target block, and whRatio is a value calculated by Min(Abs(Log2(nTbW/nTbH), 2)). The larger the aspect ratio of the target block size, the larger the value becomes. The value is 0 if the shape is a square, 1 if nTbH:nTbW is 1:2, and 2 if nTbH:nTbW is 1:4.

The intra-prediction image generator 310 generates a prediction image predSamples using the reference picture samples p[x][y]. For example, the planar mode may be derived by the following formula:
predV[x][y] = ((nTbH-1-y)*p[x][-1]+(y+1)*p[-1][nTbH])<<Log2(nTbW)
predH[x][y] = ((nTbW-1-x)*p[-1][y]+(x+1)*p[nTbW][-1])<<Log2(nTbH)
predSamples[x][y] = (predV[x][y]+predH[x][y]+nTbW*nTbH)>>(Log2(nTbW)+Log2(nTbH)+1)
The intra-prediction parameter decoding unit 304 and intra-prediction parameter encoding unit 113 configured as above decode or encode a tile size that is an integer multiple of the tile unit size, perform multi-tree partitioning into quadtrees, binary trees, and ternary trees using the intra-tile coordinates and the tile size, and determine CTU boundaries using the intra-tile coordinates (the upper left coordinate of the target block based on the upper left coordinate of the target tile). This has the effect of making it possible to easily perform multi-line intra prediction at CTU boundaries without using additional line memory, even when the tile size is not an integer multiple of the CTU.

Fig. 37 is a flowchart showing a process of multi-line intra prediction in a case where a picture is divided into tiles of a size not limited to an integer multiple of a CTU. Fig. 41(b) is an example of a syntax table.

S4101: The header decoding unit 3020 and the header encoding unit 1110 decode or encode the tile size in units equal to or smaller than the CTU size (tile size not limited to integer multiples of the CTU size). For example, as already explained, the tile_unit_size_idc may be decoded to derive the tile unit size wUnitTile or hUnitTile, and a tile size that is a multiple of wUnitTile or hUnitTile may be decoded.

S4102: The CT information decoding unit 3021 and the CT information encoding unit 1111 (CT division unit) perform CT division using the intra-picture coordinate system. For example, as already explained, CT division is performed using the syntax configurations shown in Figs. 27 to 29. At this time, the upper left coordinates (x0, y0) of CT, that is, the arguments (x0, y0) of QT division coding_quadtree(x0, y0, ...) and MTT division multi_type_tree(x0, y0, ...) use coordinates with the upper left coordinates of the picture as the origin. In addition, the division syntax qt_split_cu_flag and mtt_split_cu_flag are determined by referring to the intra-picture coordinates (x0, y0), the upper left coordinates of the tile xTile, yTile, and the tile sizes wTile and hTile. For example,
if (x1-xTile[TileId] < wTile[TileId])
coding_quadtree( x1, y0, log2CbSize-1, cqtDepth+1, treeType)
if (y1-yTile[TileId] < hTile[TileId])
coding_quadtree( x0, y1, log2CbSize-1, cqtDepth+1, treeType)
In this way, CT division may be performed based on the coordinates in the tile derived by subtracting the coordinates of the top left corner of the tile from the coordinates in the picture.

S4103: The intra-prediction parameter decoding unit 304 and the intra-prediction parameter encoding unit 113 convert from intra-picture coordinates to intra-tile coordinates. The intra-tile coordinates (x0-xTile, y0-yTile) are derived by subtracting the upper left tile coordinates (xTile, yTile) in the intra-picture coordinates from the intra-picture coordinates (x0, y0). At this time, only the Y coordinate may be derived.

S4104: The intra prediction parameter decoding unit 304 and the intra prediction parameter coding unit 113 use the coordinate values (x0-xTile, y0-yTile) of the current block in the intra-tile coordinate system to determine whether the current block is below the CTU boundary. If ((y0-yTile)%ctuSize)>0, it is determined that the current block is below the CTU boundary. If the current block is below the CTU boundary, the process proceeds to S4105, where the intra reference line index intra_luma_ref_idx is decoded or coded. Otherwise, if the current block is in contact with the CTU boundary, intra_luma_ref_idx is not decoded and is derived (inferred) as 0.
In other words, the CTU boundary determination may be performed when (yCb%ctuSize)>0 using the top left coordinates (xCb, yCb) of the top left luminance block relative to the top left coordinates of the target tile, which are the coordinate values of the target block in the tile coordinate system.

S4105: The intra prediction parameter decoding unit 304 and the intra prediction parameter encoding unit 113 decode or encode the intra reference line index intra_luma_ref_idx.

In addition, in intra prediction, in addition to the intra reference line index intra_luma_ref_idx, a flag intra_luma_mpm_flag indicating whether or not it is MPM, an index intra_luma_mpm_idx for selecting MPM, and an index intra_luma_mpm_remainder indicating the intra prediction mode when it is not MPM may be further decoded or encoded. That is, the CTU boundary determination may be performed with (yCb%ctuSize)>0 using the upper left coordinate (xCb, yCb) of the luminance block relative to the upper left coordinate of the target tile, which is the coordinate value of the target block in the intra-tile coordinate system.

The intra-prediction image generation unit 310 uses IntraLumaRefLineIdx (refIdx) to derive a reference picture sample p[x][y], and derives a prediction image from the derived reference picture sample p[x][y].

The intra prediction parameter decoding unit 304 and intra prediction parameter encoding unit 113 configured as above decode or encode a tile size that is an integer multiple of the tile unit size, perform multi-tree division into quad trees, binary trees, and ternary trees using the intra-picture coordinates, the top left coordinate of the tile, and the tile size, and perform CTU boundary determination by converting from intra-picture coordinates (the top left coordinate of the target block based on the top left coordinate of the target picture) to intra-tile coordinates (the top left coordinate of the target block based on the top left coordinate of the target tile). This has the effect of enabling accurate CTU boundary determination even when the tile size is not an integer multiple of the CTU, and enabling multi-line intra prediction without using additional line memory.

(MPM derived)
The intra-prediction parameter decoding unit 304 and the intra-prediction parameter encoding unit 113 derive the positions (xNbA, yNbA) and (xNbB, yNbB) of adjacent blocks A and B, respectively, to the left and above of the target block with top left coordinates (xCb, yCb) and size (cbWidth, cbHeight), as follows.

(xNbA, yNbA) = (xCb-1, yCb+cbHeight-1)
(xNbA, yNbA) = (xCb+cbWidth-1, yCb-1)
The intra prediction parameter decoding unit 304 and the intra prediction parameter encoding unit 113 derive adjacent intra prediction modes candIntraPredModeA and candIntraPredModeB, which are the intra prediction modes of blocks A and B. Specifically, when X=A or B, If any one of the above conditions is true, the adjacent intra prediction modes (candIntraPredModeA, candIntraPredModeB) are set to the planar mode.
・The X position (xNbX, yNbX) is outside the screen or tile and is not available.
The prediction mode CuPredMode at the X position (xNbX, yNbX) is not intra (MODE_INTRA).
・X is the upper adjacent block B, and the Y coordinate of X (yCb) is above the CTU boundary.

In cases other than the above, the intra prediction mode at the position (xNbX, yNbX) of X is set as the adjacent prediction mode. Note that the CTU boundary determination in the case of flexible tiles will be described later.

candIntraPredModeX = IntraPredModeY[xNbX][yNbX]
The intra prediction parameter decoding unit 304 and the intra prediction parameter encoding unit 113 may derive the MPM list candModeList using the following formula.
First, derive an initial list using the following formula:
candModeList[ 0 ] = candIntraPredModeA
candModeList[ 1 ] = !candIntraPredModeA
candModeList[ 2 ] = INTRA_ANGULAR50
candModeList[ 3 ] = INTRA_ANGULAR18
candModeList[ 4 ] = INTRA_ANGULAR46
candModeList[ 5 ] = INTRA_ANGULAR54
If candIntraPredModeB is equal to candIntraPredModeA, then do the following.

If candIntraPredModeA is greater than 1, the list is updated as follows: Entries 3, 4, and 5 store intra prediction modes of −1, +1, and −2 for candIntraPredModeA.
candModeList[ 0 ] = candIntraPredModeA
candModeList[ 1 ] = INTRA_PLANAR
candModeList[ 2 ] = INTRA_DC
candModeList[ 3 ] = 2+((candIntraPredModeA+62)%65)
candModeList[ 4 ] = 2+((candIntraPredModeA-1)%65)
candModeList[ 5 ] = 2+((candIntraPredModeA+61)%65)
In any other case (if candIntraPredModeB and candIntraPredModeA are not equal), the two adjacent prediction modes are stored in entries 0 and 1.
candModeList[ 0 ] = candIntraPredModeA
candModeList[ 1 ] = candIntraPredModeB
biggerIdx = candModeList[0]>candModeList[1] ? 0 : 1
Furthermore, if candIntraPredModeA and candIntraPredModeB are both greater than 1 (if both adjacent prediction modes are angular prediction (directional prediction)), the list is updated as follows.
candModeList[ 2 ] = INTRA_PLANAR
candModeList[ 3 ] = INTRA_DC
Furthermore, if candModeList[biggerIdx]-candModeList[!biggerIdx] is 64 or 1 (
That is, when the difference between the two adjacent prediction modes is −1 or 1), the following is performed and the intra prediction modes of −2 and +2 of the adjacent prediction modes are stored in entries 4 and 5.
candModeList[ 4 ] = 2+((candModeList[biggerIdx]+62)%65)
candModeList[ 5 ] = 2+((candModeList[biggerIdx]-1)%65)
If candModeList[biggerIdx]-candModeList[!biggerIdx] is not 64 or 1, the following is performed and intra prediction modes that are −1 and +1 of the adjacent prediction modes are stored in entries 4 and 5.
candModeList[ 4 ] = 2+((candModeList[biggerIdx]+61)%65)
candModeList[ 5 ] = 2+(candModeList[biggerIdx]%65)
In the case other than the above, if the sum of candIntraPredModeA and candIntraPredMode is 2 or more (if the two adjacent prediction modes are not the DC mode and the planar mode), the following is performed: In the entries 3, 4, and 5, the intra prediction modes of −1, +1, and −2 of the adjacent prediction modes are stored.
candModeList[ 2 ] = !candModeList[!biggerIdx]
candModeList[ 3 ] = 2+((candModeList[biggerIdx]+62)%65)
candModeList[ 4 ] = 2+((candModeList[biggerIdx]-1)%65)
candModeList[ 5 ] = 2+((candModeList[biggerIdx]+61)%65)
(Derivation of adjacent intra prediction modes in MPM derivation in flexible tiles)
42 is a flowchart showing the MPM derivation process when a picture is divided into tiles with a size not limited to an integer multiple of the CTU.
Since this is the same processing as S4002, the explanation will be omitted.

S5004: The intra-prediction parameter decoding unit 304 and the intra-prediction parameter encoding unit 113 use the coordinate values (xCb, yCb) of the target block in the intra-tile coordinate system to determine whether the Y coordinate yCb of the adjacent block X is above the CTU boundary.

(yCb-1)<((yCb>>CtbLog2SizeY)<<CtbLog2SizeY)
In addition to the above determination formula, a case in which any of the following determination formulas is true may be added.
- The X position is off-screen or off-tile and is not available.
The prediction mode CuPredMode at the X position (xNbX, yNbX) is not intra (MODE_INTRA).

S5005: Set the intra prediction mode for X position (xNbX, yNbX) to adjacent prediction mode.

candIntraPredModeX = IntraPredModeY[xNbX][yNbX]
When the following condition is true, the intra prediction parameter decoding unit 304 and the intra prediction parameter encoding unit 113 set the adjacent intra prediction modes (candIntraPredModeA, candIntraPredModeB) to the planar mode.
- X is the upper adjacent block B, and the Y coordinate of X (yCb) is above the CTU boundary.

The intra prediction parameter decoding unit 304 and intra prediction parameter encoding unit 113 configured as above decode or encode a tile size that is an integer multiple of the tile unit size, perform multi-tree partitioning into quad trees, binary trees, and ternary trees using the intra-tile coordinates and tile size, and determine CTU boundaries using the intra-tile coordinates (the top left coordinate of the target block based on the top left coordinate of the target tile), thereby achieving the effect of making it possible to derive an MPM without using line memory, even when the tile size is not an integer multiple of the CTU size.

43 is a flowchart showing the MPM derivation process when a picture is divided into tiles with a size that is not limited to an integer multiple of the CTU. S5101, S5102, and S5103 are the same processes as S4101, S4102, and S4103, and therefore description thereof will be omitted.

S5104: The intra-prediction parameter decoding unit 304 and the intra-prediction parameter encoding unit 113 use the coordinate values (x0-xTile, y0-yTile) of the target block in the intra-tile coordinate system to determine whether the Y coordinate (yCb-yTile) of adjacent block X is above the CTU boundary.

(yCb-yTile-1) <((yCb-yTile)>>CtbLog2SizeY)<<CtbLog2SizeY
In addition to the above determination formula, a case in which any of the following determination formulas is true may be added.
- The X position is off-screen or off-tile and is not available.
The prediction mode CuPredMode at the X position (xNbX, yNbX) is not intra (MODE_INTRA).

S5105: The intra prediction mode at the X position (xNbX, yNbX) is set to the adjacent prediction mode.

candIntraPredModeX = IntraPredModeY[xNbX][yNbX]
When the following condition is true, the intra-prediction parameter decoding unit 304 and the intra-prediction parameter encoding unit 113 set the adjacent intra-prediction modes (candIntraPredModeA, candIntraPredModeB) to the planar mode.
X is the upper neighbor block B, and the Y coordinate of X (yCb) is above the CTU boundary.

The intra prediction parameter decoding unit 304 and intra prediction parameter encoding unit 113 configured as above decode or encode a tile size that is an integer multiple of the tile unit size, perform multi-tree partitioning into quad trees, binary trees, and ternary trees using the intra-picture coordinates, the top left tile coordinates, and the tile size, and convert from the intra-picture coordinates to intra-tile coordinates to determine the CTU boundaries. This has the effect of making it possible to derive MPM without using line memory, even when the tile size is not an integer multiple of the CTU.

The entropy decoding unit 301 receives inter prediction parameters (prediction mode predMode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc,
The entropy decoding unit 301 outputs the quantized transform coefficients to the inverse quantization and inverse transform unit 311. ...

(CCLM forecast)
The intra-prediction image generating unit 310 (CCLM prediction unit) may perform CCLM prediction (Chroma Component Linear Model) that linearly predicts chrominance from luminance. In CCLM prediction, a linear prediction relationship (linear prediction parameters a, b) between luminance and chrominance is derived for adjacent pixels (adjacent above and to the left) of the target block, and a chrominance image is derived from the luminance image of the target block using the relational expression y=a*x+b. More specifically, a chrominance prediction image predSamples[][] is derived from the pixel value pDsY[x][y] obtained by downsampling luminance, prediction parameters a, b, and a predetermined shift value k using the following expression.

predSamples[x][y] = Clip1C(((pDsY[x][y]*a)>>k)+b)
(CCLM forecast for flexible tiles)
As described above, the size of the tile is decoded or encoded as an integer multiple of the tile unit size TileUnitSizeY. In addition, the CT division unit uses the intra-picture coordinate system to calculate the intra-picture coordinates, the upper left coordinate of the tile, and the tile Using the size, perform multi-tree division into quad trees, binary trees, and ternary trees. Alternatively, using the intra-tile coordinate system, perform multi-tree division into quad trees, binary trees, and ternary trees using the intra-tile coordinates and tile size. Perform multi-tree partitioning of the 1-ary tree.

The CCLM prediction unit performs CTU boundary determination using the top left coordinates (xCbC, yCbC) of the chrominance block relative to the top left coordinates of the target tile, which are the coordinate values of the target block in the tile coordinate system.

bCTUboudary = yCbC &((1<<(CtbLog2SizeY-1)-1))
In addition, the following can be derived considering samples other than 4:2:0 where the luma and chroma sampling is 2:1. If SubHeightC is the luma and chroma sampling ratio,
bCTUboudary = (SubHeightC==2) ? (yCbC &((1<<(CtbLog2SizeY)-1)-1)) : ((1<<(CtbLog2SizeY)-1))
bCTUboudary = yCbC &((1<<(CtbLog2SizeY-1)-(SubHeightC-1)))
You can also switch using chroma_format_idc.

You can also use the luminance top left coordinate (xCb, yCb) in the tile to derive the value using the following formula:

bCTUboudary = yCb &((1<<(CtbLog2SizeY))-1)
Here, the top left coordinates (xCbC, yCbC) of the top left chrominance block may be derived from the top left luminance block coordinates (xCb, yCb) with reference to the top left coordinates of the tile.

It is also possible to perform multi-tree division into a quadtree, binary tree, or ternary tree using the coordinates within the picture, the top-left coordinate of the tile, and the tile size, and derive the top-left luminance block coordinates (xCb, yCb) within the tile from the top-left luminance coordinate (xCbInPic, yCbInPic) of the target block based on the top-left coordinate of the target screen, and the top-left luminance coordinate (xTile, yTile) of the target tile based on the top-left coordinate of the target screen.

xCb = xCbInPic - xTile[TileId]
yCb = yCbInPic - yTile[TileId]
When performing multi-tree division into quad trees, binary trees, or ternary trees using the intra-tile coordinate system and the intra-tile coordinates and tile size, the upper left coordinates of the target block are used as is as (xCb, yCb).

The CCLM prediction unit derives the adjacent luminance image pTopDsY as follows: If the upper pixel is available (availT = TRUE) and is not a CTU boundary (bCTUboundary = FALSE), the CCLM prediction unit derives it using the following formula for x = 1..nTbW-1. In other words, if bCTUboundary = FALSE, two lines of pY[x][y], pY[x][-1] and pY[x][-2], are used.
pTopDsY[x] = (pY[2*x-1][-2]+pY[2*x-1][-1]+2*pY[2*x][-2]+2*pY[2*x][-1]+pY[2*x+1][-2]+pY[2*x+1][-1]+4)>>3
Here, pY[][] is a luminance image adjacent to the target block.

If the top left coordinate is available, it may be further derived using the following formula:
pTopDsY[0]=(pY[-1][-2]+pY[-1][-1]+2*pY[0][-2]+2*pY[0][-1]+pY[1][-2]+pY[1][-1]+4)>>3
If the upper left adjacent coordinate (coordinate with negative subscripts x and y) is not available, it may be derived using the following formula:
pTopDsY[0] = (pY[0][-2]+pY[0][-1]+1)>>1
The CCLM prediction unit derives the adjacent luminance image pTopDsY in the following manner. When the upper pixel is available for reference (availT = TRUE) and is a CTU boundary (bCTUboundary = TRUW), the CCLM prediction unit derives the adjacent luminance image pTopDsY for x = 1..nTbW-1 using the following formula:
pTopDsY[x] = (pY[2*x-1][-1]+2*pY[2*x][-1]+pY[2*x+1][-1]+2)>>2
In other words, when bCTUboundary = TRUE, only one line, pY[x][-1], of pY[x][y] is used.

If the upper left adjacent coordinate is available, it may be further derived using the following formula:
pTopDsY[0] = (pY[-1][-1]+2*pY[0][-1]+pY[1][-1]+2)>>2
If the top left neighbor coordinate is not available, it may be further derived using the following formula:
pTopDsY[0] = pY[0][-1]
The CCLM prediction unit further derives the adjacent luminance image pLeftDsY[ ].

The CCLM prediction unit derives the minimum value MinLuma and maximum value MaxLuma of the adjacent luminance image pTopDsY and the chrominance pixel values ChromaForMinLuma and ChromaForMaxLuma at those positions.

The CCLM prediction unit derives the slope a by dividing the difference between ChromaForMinLuma and ChromaForMaxLuma by the difference between MinLuma and MaxLuma. More specifically, for integer arithmetic, the CCLM parameter a is derived using the following formula:

shift = (BitDepthC>8) ? BitDepthC-9 : 0
add = shift ? 1<<(shift-1) : 0
diff = (MaxLuma-MinLuma+add)>>shift
k = 16
If diff is positive, the CCLM prediction unit derives a using the following formula, otherwise a = 0.

div = ((ChromaForMaxLuma-ChromaForMinLuma)*(Floor((65536*65536)/diff)-Floor(65536/diff)*65536)+32768)>>16
a = (((ChromaForMaxLuma-ChromaForMinLuma)*Floor(65536/diff)+div+add)>>shift)
Furthermore, the CCLM prediction unit derives b.
b = ChromaForMinLuma-((a*MinLuma)>>k)
According to the CCLM prediction unit having the above configuration, a tile size that is an integer multiple of the tile unit size is decoded or encoded, and a quad tree, a binary tree, or a ternary tree is generated using the coordinates in the picture, the coordinates of the upper left corner of the tile, and the tile size. The CTU boundary is determined by converting the coordinates in the picture to coordinates in the tile (the upper left coordinate of the target block based on the upper left coordinate of the target tile), or by using the coordinates in the tile, Multi-tree partitioning is performed using 1-, 2-, and 3-way trees, and CTU boundaries are determined using coordinates within the tile. This makes it possible to use line memory even when the tile size is not an integer multiple of the CTU. This has the effect of making CCLM predictions possible without any need for calculations.

(Merge prediction)
38(a) is a schematic diagram showing a configuration of a merge prediction parameter derivation unit 3036 according to this embodiment. The merge prediction parameter derivation unit 3036 includes a merge candidate derivation unit 30361 and a merge candidate selection unit 30362. Note that a merge candidate includes a prediction list use flag predFlagLX, a motion vector mvLX, and a reference picture index refIdxLX, and is stored in a merge candidate list. An index is assigned to the merge candidates stored in the merge candidate list according to a predetermined rule.

The merge candidate derivation unit 30361 derives merge candidates by directly using the motion vectors of the decoded neighboring blocks and the reference picture indexes refIdxLX. In addition, the merge candidate derivation unit 30361 may apply a spatial merge candidate derivation process, a temporal merge candidate derivation process, a joint merge candidate derivation process, a zero merge candidate derivation process, and a spatio-temporal merge candidate derivation process, which will be described later.

(Spatial merge candidate derivation process)
As a spatial merge candidate derivation process, the merge candidate derivation unit 30361 reads out prediction parameters (prediction list use flag predFlagLX, motion vector mvLX, reference picture index refIdxLX) stored in the prediction parameter memory 307 according to a predetermined rule, and sets them as merge candidates. The method of specifying the reference picture is, for example, prediction parameters related to each of adjacent blocks within a predetermined range from the target block (for example, all or part of blocks adjacent to the left end L, the lower left end BL, the upper left end AL, the upper end A, and the upper right end AR of the target block). Each merge candidate is called L, BL, AL, A, and AR.

(Temporal merge candidate derivation process, temporal motion vector derivation process)
As a temporal merge derivation process, the merge candidate derivation unit 30361 reads prediction parameters of the lower right CBR of the target block or block C in the reference image including the center coordinates from the prediction parameter memory 307, sets it as a merge candidate, and stores it in the merge candidate list mergeCandList[ ].

FIG. 18 is a diagram showing blocks C and CBR in a reference image. Block CBR (block BR) is added to the merge candidate list mergeCandList[] with priority, and when block CBR does not have a motion vector (for example, an intra-prediction block) or when block CBR is located outside the picture, the motion vector of block C is added to the prediction vector candidate. By adding the motion vector of a collocated block that is likely to have different motion as a prediction candidate, the number of prediction vector options increases and the coding efficiency increases. The method of specifying the reference image may be, for example, the reference picture index refIdxLX specified in the slice header, or the smallest one of the reference picture indexes refIdxLX of adjacent blocks.

For example, the merge candidate derivation unit 30361 may derive the position of block C (xColCtr, yColCtr) and the position of block CBR (xColBr, yColBr) using the following formula.

xColCtr = xCb + (bW>>1)
yColCtr = yCb + (bH>>1)
xColBr = xCb+bW
yColBr = yCb + bH
Here, (xCb, yCb) are the upper left coordinates of the target block, and (bW, bH) are the width and height of the target block. Note that bW and bH are also written as cbWidth and cbHeight.
If the block CBR is available, the motion vector of the block CBR is used to derive the merge candidate COL. If the block CBR is not available, the block C is used to derive COL.

(Configuration of tile boundary time merge candidate derivation process)
The merge candidate derivation unit 30361 (temporal motion prediction unit, temporal motion vector derivation unit) preferably manages the motion vectors in the reference memory in units of CTU lines, and fetches only the range required for the target CTU to the internal memory. At this time, the merge candidate derivation unit 30361 checks whether the position (xColBr, yColBr) of the block CBR is within the range of the target CTU so as not to access areas other than the fetched area.
When the line is crossed, block BR is not used, and block C is used according to the following process.

Below is a processing example in which the upper left coordinate of the target block is processed using the coordinates in the tile. In the following processing, if it is determined that the CTU line is the same because (yCb>>CtbLog2SizeY)==(yColBr>>CtbLog2SizeY), (xColBr, yColBr) is used as the reference position (xRef, yRef), and (xColCtr, yColCtr) is used otherwise. Furthermore, as in the example below, access may be restricted to 8*8 units by shifting 3 bits to the right and then shifting 3 bits to the left. When referring to a motion vector in the reference picture memory, the coordinates of the upper left tile are added to the reference position to derive the reference position in the coordinates in the screen, but this operation may be performed before or after quantization (shifting) of the reference position.

xColBr = xCbInTile + cbWidth
yColBr = yCbInTile + cbHeight
xColCtr = xCbInTile + (cbWidth>>1)
yColCtr = yCbInTile + (cbHeight>>1)
if (((yCb>>CtbLog2SizeY)==(yColBr>>CtbLog2SizeY)) &&(yColBr<hPict)&&(xColBr<wPict)) {
// Use (xColBr, yColBr) to look up the motion vector of the reference picture. For example, see (xRef, yRef) below.

xRef = ((xColBr>>3)<<3)+xTile[TileId]
yRef = ((yColBr>>3)<<3)+yTile[TileId]
}
else {
// Use (xColCtr, yColCtr) to look up the motion vector of the reference picture. For example,
xRef = ((xColCtr>>3)<<3)+xTile[TileId]
yRef = ((yColCtr>>3)<<3)+yTile[TileId]
}
FIG. 40 shows a case where the tile division of the reference picture and the tile division of the target picture are equal. In this case, the CTU coordinates in the screen are derived from the CTU coordinates in the target picture and the CTU coordinates in the reference picture. In addition, when managing the MV memory for each CTU line in a tile, it is difficult to refer to the MV memory of a CTU line that is not in the tile even if it is in the screen. The range is limited to within a tile, as shown in Figure 39(a).

xColBr = xCbInTile+cbWidth
yColBr = yCbInTile+cbHeight
xColCtr = xCbInTile+(cbWidth>>1)
yColCtr = yCbInTile+(cbHeight>>1)
if (((yCb>>CtbLog2SizeY)==(yColBr>>CtbLog2SizeY)) &&(yColBr<hTile[TileId])&&(xColBr<wTile[TileId])) {
// Use (xColBr, yColBr) to look up the motion vector of the reference picture. For example, see (xRef, yRef) below.

xRef = ((xColBr>>3)<<3)+xTile[TileId]
yRef = ((yColBr>>3)<<3)+yTile[TileId]
}
else {
// Use (xColCtr, yColCtr) to look up the motion vector of the reference picture. For example,
xRef = ((xColCtr>>3)<<3)+xTile[TileId]
yRef = ((yColCtr>>3)<<3)+yTile[TileId]
}
The above process may also be performed in the case of independent tiles (independent_tiles_flag=1).

To avoid referencing ranges beyond tile boundaries, it is appropriate to use intra-tile coordinates (xCbInTile, yCbInTile) with the top left corner of the tile as the coordinate system for the prediction block, and not reference each reference position if it exceeds the tile size (wTile, hTile). Furthermore, when referencing a motion vector in the reference picture memory, the upper left coordinate of the tile is added to the reference position to derive the reference position in intra-screen coordinates. As shown in Figures 31 to 34, by performing CT division in intra-tile coordinates, the upper left coordinate of the prediction block can be derived in intra-tile coordinates, and the position to be referenced in temporal prediction can be determined, which has the effect of easily determining whether or not it crosses a CTU boundary.

Here is an example of processing the top left coordinate of the target block using in-screen coordinates. Unlike processing using the in-tile coordinate system, the top left coordinate of the tile is subtracted and converted to in-tile coordinates (yColBr-yTile, xColBr-xTile), which are then compared with the tile size (wTile, hTile) to determine whether or not they exceed the tile size. The tile boundary is also determined by right-shifting the in-tile coordinates of the target block position (yCb-yTile[TileId]) and the reference block position (yColBr-yTile[TileId]) by the logarithm of the CTU size, CtbLog2SizeY.

xColBr = xCbInPic+cbWidth
yColBr = yCbInPic+cbHeight
xColCtr = xCbInPic+(cbWidth>>1)
yColCtr = yCbInPic+(cbHeight>>1)
if (((yCb-yTile[TileId])>>CtbLog2SizeY)==((yColBr-yTile[TileId])>>CtbLog2SizeY)) &&(yColBr<hPict)&&(xColBr<wPict)) {
xRef = ((xColBr>>3)<<3)
yRef = ((yColBr>>3)<<3)
}
else {
xRef = ((xColCtr>>3)<<3)
yRef = ((yColCtr>>3)<<3)
}
Fig. 40 shows a case where the tile division of the reference picture and the tile division of the target picture are equal. In this case, in this embodiment, the reference range is limited to within the tile. Fig. 39(b) shows an example of processing.

xColBr = xCbInPic+cbWidth
yColBr = yCbInPic+cbHeight
xColCtr = xCbInPic+(cbWidth>>1)
yColCtr = yCbInPic+(cbHeight>>1)
if (((yCb-yTile[TileId])>>CtbLog2SizeY)==((yColBr-yTile[TileId])>>CtbLog2SizeY)) &&((yColBr-yTile[TileId])<hTile[TileId])&&((xColBr-xTile[TileId])<wTile[TileId])) {
xRef = ((xColBr>>3)<<3)
yRef = ((yColBr>>3)<<3)
}
else {
xRef = ((xColCtr>>3)<<3)
yRef = ((yColCtr>>3)<<3)
}
The above process may also be performed in the case of independent tiles.

By converting the coordinates of a predicted block from in-screen coordinates to in-tile coordinates, it becomes possible to easily determine whether the block is within the tile range and whether it crosses a CTU boundary.

(Method using virtual CTU line)
14 is a diagram showing a reference range when a virtual CTU line is set and a motion vector of a reference picture is referred to. In this configuration, a reference picture is divided into virtual CTU lines, and a motion vector of the virtual CTU line corresponding to a target block is referred to.

One configuration is to divide the reference picture into fixed CTU size and set virtual CTU lines.

FIG. 15 is a flowchart showing the operation of the temporal motion prediction unit when an integer pixel picture is divided into tiles (flexible tiles) with a size not limited to an integer multiple of the CTU.

S3001: Derive in-screen coordinates The temporal motion prediction unit derives the CTU coordinates (xCtbInPic, yCtbInPic), bottom right coordinate (xColBr, yColBr), and center coordinates (xColCtr, yColCtr) of the in-screen coordinates from (xCbInTile, yCbInTile) in the in-tile coordinate system. Note that when processing is performed in the in-tile coordinate system, such as when CT division (multi-tree division such as QT, BT, TT) is not performed in the in-tile coordinate system, conversion from the in-tile coordinate system to the in-screen coordinate system can be omitted.
xCtbInPic = xCtbInTile+xTile[currTileId]
yCtbInPic = yCtbInTile+yTile[currTileId]
xColBrInPic = xCbInTile+cbWidth+xTile[currTileId]
yColBrInPic = yCbInTile+cbHeight+yTile[currTileId]
xColCtrInPic = xCbInTile+(cbWidth>>1)+xTile[currTileId]
yColCtrInPic = yCbInTile+(cbHeight>>1)+yTile[currTileId]
S3003: Judgment within virtual CTU line It is judged whether the bottom right coordinate (xColBrInPic, yColBrInPic) of the screen is within the virtual CTU line.
Determine whether it is within the U line and the screen.

Judgment formula = ((yColBrInPic>>CtbLog2SizeY) == (yVirCtb>>CtbLog2SizeY)) &&(yColBr<hPict)&&(xColBr<wPict)
S3004: Referencing the motion vector of the reference picture When the judgment formula is true, the temporal motion prediction unit refers to the motion vector of the lower right coordinate. Here, the coordinates may be further thinned to a unit such as 8x8 for reference. .

((xColBrInPic>>3)<<3, (yColBrInPic>>3)<<3)
S3005: If the determination formula is false, the motion vector at the center coordinates (xColCtrInPic, yColCtrInPic) is referenced. Here, the coordinates may be thinned out to units such as 8x8 before reference.

((xColCtrInPic>>3)<<3, (yColCtrInPic>>3)<<3)
Furthermore, before referring to the motion vector of the center coordinates, it may be determined whether the reference coordinates are within the virtual CTU line using the following determination formula.

Judgment formula = (yColCtrInPic>yVirCtbB)
16 is a diagram showing a reference range when a virtual CTU line is set and a motion vector of a reference picture is referred to. In this configuration, the reference picture is set in VBSize×VBSize size units (Log2VBSize=log2(VBSize)). From the perspective of the target CTU, this separator position is set to the upper coordinate yCtbInPic of the reference CTU line, and a predetermined range from this upper coordinate yCtbInPic is set as the range of motion vectors that can be referenced from the reference picture.

FIG. 17 is a flowchart showing the operation of the temporal motion prediction unit when an integer pixel picture is divided into tiles (flexible tiles) with a size not limited to an integer multiple of the CTU.

S3001: Derive in-screen coordinates The temporal motion prediction unit derives the CTU coordinates (xCtbInPic, yCtbInPic), bottom right coordinate (xColBr, yColBr), and center coordinates (xColCtr, yColCtr) of the in-screen coordinates from (xCbInTile, yCbInTile) in the in-tile coordinate system. Note that if processing is performed in the in-screen coordinate system, such as when CT division (multi-tree division such as QT, BT, TT) is not performed in the in-tile coordinate system, conversion from the in-tile coordinate system to the in-screen coordinate system can be omitted.
xCtbInTile = (xCbInTile>>Log2VBSize)<<Log2VBSize
yCtbInTile = (yCbInTile>>Log2VBSize)<<Log2VBSize
xCtbInPic = xCtbInTile+xTile[currTileId]
yCtbInPic = yCtbInTile+yTile[currTileId]
xColBrInPic = xCbInTile+cbWidth+xTile[currTileId]
yColBrInPic = yCbInTile+cbHeight+yTile[currTileId]
xColCtrInPic = xCbInTile+(cbWidth>>1)+xTile[currTileId]
yColCtrInPic = yCbInTile+(cbHeight>>1)+yTile[currTileId]
S3002: Derive Virtual CTU Line Next, the temporal motion prediction unit converts yCtbInPic into coordinates in VBSize units, and derives the upper coordinate yVirCtbT of the virtual CTU line. It also sets the lower coordinate yVirCtbB, which is a predetermined size (here, the CTU size ctuSize) below.
yVirCtbT = ((yCtbInPic>>VBLog2Size)<<VBLog2Size)
yVirCtbB = ((yCtbInPic>>VBLog2Size)<<VBLog2Size)+ctuSize
Note that when VBLog2Size is set to CtbLog2SizeY, the processing is the same as when the reference picture is divided into fixed CTU size units.
In the above, the reference range is limited to the CTU size, but it may be a larger size. For example, when referring to a motion vector in a range of the target CTU size + M pixels on the reference picture, the following formula is used.
yVirCtbB = ((yCtbInPic>>VBLog2Size)<<VBLog2Size)+ctuSize+M
For example, M=16.

S3003: Virtual CTU In-line Determination The time motion prediction unit determines whether the bottom right coordinate (xColBrInPic, yColBrInPic) is within the virtual CTU
It is determined whether the bottom right coordinate (xColBrInPic, yColBrInPic) is within the virtual CTU line and within the screen. If it is within the virtual CTU line, the motion vector of the reference position (xColBrInPic, yColBrInPic) is referenced.

Judgment formula = (yColBrInPic<yVirCtbB) &&(yColBr<hPict)&&(xColBr<wPict)
S3004: Referencing the motion vector of the reference picture When the judgment formula is true, the temporal motion prediction unit refers to the motion vector of the lower right coordinate of the reference picture. Here, the coordinates are further thinned out to a unit of 8x8, etc. That's fine.

((xColBrInPic>>3)<<3, (yColBrInPic>>3)<<3)
Although not shown, the following process may be carried out.

S3005: Motion vector reference 2 of reference picture
If the judgment formula is false, the temporal motion prediction unit refers to the motion vector at the center coordinates (xColCtrInPic, yColCtrInPic) of the reference picture. Here, the coordinates may be thinned out to a unit of 8x8 or the like.

((xColCtrInPic>>3)<<3, (yColCtrInPic>>3)<<3)
Furthermore, before referring to the motion vector of the center coordinates, it may be determined whether the reference coordinates are within the virtual CTU line using the following determination formula.

Judgment formula = (yColCtrInPic<yVirCtbB)
The pseudo code for the above process is as follows:
if ((yColBrInPic<yVirCtbB) &&(yColBr<hPict)&&(xColBr<wPict)) {
Look up the motion vector using (xColBrInPic, yColBrInPic)
// For example, the following coordinates rounded to 8x8 units are acceptable:

// ((xColBrInPic>>3)<<3, (yColBrInPic>>3)<<3)
}
else if (yColCtrInPic<yVirCtbB) { // Determine the virtual CTU line boundary at the block center
Look up the motion vector using (xColCtr, yColCtr)
// For example, the following coordinates rounded to 8x8 units are acceptable.

// ((xColCtr>>3)<<3, (yColCtr>>3)<<3)
}
According to the temporal motion prediction unit having the above configuration, the in-picture coordinate yColBrInPic of the reference block is compared with the virtual CTU coordinate yVirCtbB derived from the in-picture coordinate of the target picture to determine the referenceability of the reference block. In the derivation of the virtual CTU coordinate yVirCtbB, the virtual CTU coordinate is shifted right and left by VBLog2Size to make it an integer multiple of VBSize. If the virtual CTU coordinate yVirCtbB derived from the above is smaller than the virtual CTU coordinate yVirCtbB, the motion vector of the reference picture is referenced. This allows the range of motion vectors in the reference picture to be referenced to be determined by the virtual CTU coordinate yVirCtbB even if the CTU sizes of the target picture and the reference picture are different. Since it is limited to the range of the CTU line, it is possible to pre-fetch and process the motion vector on the reference picture, which has the effect of enabling high-speed processing. In addition, when setting a virtual CTU line, which is the range for referencing a motion vector in a reference picture, the CTU line can be set while maintaining the same position as the target CTU as much as possible.

(Modification)
In the above, the virtual CTU line was set with a granularity of VBSize x VBSize, but if the granularity of the virtual CTU line (for example, 8 x 8, i.e. VBSize = 8) is less than or equal to the granularity of the CTU size (ctuUnitSize), the shift processing related to VBSize can be omitted when deriving the virtual CTU line.

18 is a flowchart showing the operation of the temporal motion prediction unit when dividing an integer pixel picture into tiles (flexible tiles) of a size not limited to an integer multiple of a CTU. In this example, shift processing is omitted. In this case, the virtual CTU line derivation in S3002 may be the following operation.

The upper coordinate yVirCtbT of the virtual CTU line is derived from the in-screen coordinate of the target CTU coordinate. If the target CTU coordinate is a coordinate in a tile, the in-tile coordinate of the target CTU (the target CTU coordinate in the tile coordinate system) and the in-screen coordinate of the tile are added to derive and process the in-screen coordinate of the target CTU. In addition, the lower coordinate yVirCtbB is set at a position lower by a predetermined size (here, the CTU size ctuSize). Here, the shift processing related to VBSize is omitted.
yVirCtbT = yCtbInPic
yVirCtbB = yCtbInPic+ctuSize
The pseudocode for the process that follows is as follows:
if ((yColBrInPic<yVirCtbB) &&(yColBr<hPict)&&(xColBr<wPict)) {
Look up the motion vector using (xColBrInPic, yColBrInPic)
// For example, the following coordinates rounded to 8x8 units are acceptable:

// ((xColBrInPic>>3)<<3, (yColBrInPic>>3)<<3)
}
else if (yColCtrInPic<yVirCtbB) { // Determine the virtual CTU line boundary at the block center
Look up the motion vector using (xColCtr, yColCtr)
// For example, the following coordinates rounded to 8x8 units are acceptable.

// ((xColCtr>>3)<<3, (yColCtr>>3)<<3)
The temporal motion prediction unit configured as above compares the intra-picture coordinate yColBrInPic of the reference block with the virtual CTU coordinate yVirCtbB derived from the intra-picture coordinate of the target picture to determine the referenceability of the reference block.

The loop filter 305 is a filter provided in the encoding loop, which removes block distortion and ringing distortion to improve image quality. The loop filter 305 applies filters such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312.

The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 in a predetermined location for each target picture and target CU.

The prediction parameter memory 307 stores prediction parameters at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

The prediction image generating unit 308 receives a prediction mode predMode, prediction parameters, and the like. The prediction image generating unit 308 also reads a reference picture from the reference picture memory 306. The prediction image generating unit 308 generates a prediction image of a block or sub-block using the prediction parameters and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block is a set of pixels on the reference picture (usually rectangular, so called a block), and is an area to be referenced for generating a prediction image.

The inverse quantization and inverse transform unit 311 inverse quantizes the quantized transform coefficients input from the entropy decoding unit 301 to obtain transform coefficients. In the encoding process, the quantized transform coefficients are transformed using a discrete cosine transform (DCT), a discrete sine transform (DST), a Karyhnen Loeve transform (KLT), etc.
The inverse quantization and inverse transform unit 311 performs inverse frequency transform such as inverse DCT, inverse DST, and inverse KLT on the obtained transform coefficients to calculate a prediction error. The inverse quantization and inverse transform unit 311 outputs the prediction error to the adder 312.

The adder 312 adds, for each pixel, the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization and inverse transform unit 311 to generate a decoded image of the block.
The adder 312 stores the decoded image of the block in the reference picture memory 306 , and also outputs it to the loop filter 305 .

(Configuration of the video encoding device)
Next, the configuration of the video encoding device 11 according to this embodiment will be described. Fig. 19 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a prediction image generating unit 101, a subtraction unit 102, a transformation/quantization unit 103, an inverse quantization/inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determining unit 110, a parameter encoding unit 111, and an entropy encoding unit 104.

The predicted image generating unit 101 generates a predicted image for each CU, which is an area obtained by dividing each picture of the image T. The predicted image generating unit 101 operates in the same manner as the predicted image generating unit 308 already described, and therefore a description thereof will be omitted.

The subtraction unit 102 subtracts the pixel values of the predicted image of the block input from the predicted image generation unit 101 from the pixel values of image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the transformation and quantization unit 103.

The transform/quantization unit 103 calculates transform coefficients by frequency transforming the prediction error input from the subtraction unit 102, and derives quantized transform coefficients by quantizing the prediction error.
The quantized transform coefficients are output to the entropy coding unit 104 and the inverse quantization and inverse transform unit 105 .

The inverse quantization and inverse transform unit 105 is a unit similar to the inverse quantization and inverse transform unit 311 (FIG. 5) in the video decoding device 31.
The calculated prediction error is output to the adder 106.

The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, a CU coding unit 1112 (prediction mode coding unit), an entropy coding unit 104, and an inter-prediction parameter coding unit 112 and an intra-prediction parameter coding unit 113 (not shown). The CU coding unit 1112 further includes a TU coding unit 1114.

The operation of each module will be outlined below: The parameter coding unit 111 performs coding processing of parameters such as header information, division information, prediction information, and quantized transformation coefficients.

The CT information encoding unit 1111 encodes QT, MT (BT, TT) division information, etc.

The CU encoding unit 1112 encodes CU information, prediction information, the TU split flag split_transform_flag, the CU residual flags cbf_cb, cbf_cr, cbf_luma, etc.

When a TU contains a prediction error, the TU encoding unit 1114 encodes the QP update information (quantization correction value) and the quantized prediction error (residual_coding).

The entropy encoding unit 104 converts the syntax elements supplied from the supply source into binary data, generates encoded data by an entropy encoding method such as CABAC, and outputs the encoded data. In the example shown in FIG. 19, the syntax elements are supplied from a CT information encoding unit 1111 and a CU encoding unit 1112. The syntax elements include inter prediction parameters (prediction mode predMode, merge flag merge_flag, merge index merge_idx, inter prediction identifier inter_pred_idc, etc.) and the like.
, reference picture index refIdxLX, predicted vector index mvp_LX_idx, difference vector mvdLX), intra prediction parameters (prev_intra_luma_pred_flag, mpm_idx, rem_selected_mode_flag, rem_selected_mode, rem_non_selected_mode), quantized transform coefficients, etc.

The adder 106 generates a decoded image by adding, for each pixel, the pixel value of the predicted image of the block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization and inverse transform unit 105. The adder 106 stores the generated decoded image in the reference picture memory 109.

The loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the adder 106. Note that the loop filter 107 does not necessarily have to include the above three types of filters, and may be configured, for example, as only a deblocking filter.

The prediction parameter memory 108 stores the prediction parameters generated by the encoding parameter determination unit 110 in a predetermined location for each target picture and CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 in a predetermined location for each target picture and CU.

The coding parameter determination unit 110 selects one set from among a plurality of sets of coding parameters. The coding parameters are the above-mentioned QT, BT or TT division information, prediction parameters, or parameters to be coded that are generated in relation to these. The predicted image generation unit 101 generates a predicted image using these coding parameters.

The coding parameter determination unit 110 calculates an RD cost value indicating the magnitude of the information amount and the coding error for each of the multiple sets. The RD cost value is, for example, the sum of the code amount and the value obtained by multiplying the squared error by a coefficient λ. The code amount is the information amount of the coding stream Te obtained by entropy coding the quantization error and the coding parameters. The squared error is the sum of the squares of the prediction errors calculated in the subtraction unit 102. The coefficient λ is a real number greater than zero that is set in advance. The coding parameter determination unit 110 selects a set of coding parameters that minimizes the calculated cost value. As a result, the entropy coding unit 104 outputs the selected set of coding parameters as the coding stream Te. The coding parameter determination unit 110 stores the determined coding parameters in the prediction parameter memory 108.

In addition, a part of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generating unit 308, the inverse quantization and inverse transform unit 311, the addition unit 312, the predicted image generating unit 101, the subtraction unit 102, the transform and quantization unit 103, the entropy encoding unit 104, the inverse quantization and inverse transform unit 105, the loop filter 107, the encoding parameter determination unit 110, and the parameter encoding unit 111 may be realized by a computer. In this case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into and executed by a computer system. In addition, the "computer system" referred to here is a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. In addition, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and devices that hold a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above-mentioned program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

In addition, part or all of the video encoding device 11 and video decoding device 31 in the above-mentioned embodiments may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the video encoding device 11 and video decoding device 31 may be individually processed, or part or all of them may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit technology that can replace LSI appears due to advances in semiconductor technology, an integrated circuit based on that technology may be used.

One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the present invention.

A video decoding device according to one embodiment of the present invention is a video decoding device that divides an image into tiles, divides the tiles into CTUs, and decodes video in CTU units, and is characterized in that it includes a header decoding unit that sets the smallest unit of tile size to a predetermined value, and a CT information decoding unit that decodes the CTUs from the encoded stream, and the header decoding unit derives the top left position of the tile and the tile size using the smallest unit of tile size.

In the video decoding device according to one aspect of the present invention, the predetermined value is a minimum CTU size.

In the video decoding device according to one aspect of the present invention, the predetermined value is an integer multiple of a minimum CTU size.

In the video decoding device according to one aspect of the present invention, the predetermined value is 8 or 16.

In the video decoding device according to an aspect of the present invention, the predetermined value is a minimum CTU in the horizontal direction.
It is characterized in that it is an integer multiple of the size, and 8 or 16 in the vertical direction.

In a video decoding device according to one aspect of the present invention, the minimum unit of the tile size is derived using a minimum CTU size and a difference value between the minimum unit of the tile size and the minimum CTU size.

In the video decoding device according to one aspect of the present invention, the differential value is decoded from SPS encoded data.

In the video decoding device according to one aspect of the present invention, the difference value is decoded from PPS encoded data.

In a video decoding device according to one aspect of the present invention, the minimum unit of the tile size is not larger than the CTU size.

In a video decoding device according to one aspect of the present invention, the predetermined value is set to a value less than the minimum CTU size if each tile can be processed independently, and is set to the minimum CTU size if not.

In a video decoding device according to one aspect of the present invention, the predetermined value is set to a smaller minimum unit of a tile when each tile can be processed independently than when it cannot.

In a video decoding device according to one aspect of the present invention, the predetermined value is set according to one of a sub-CU, a motion vector storage unit for TMVP, and a unit for notifying a differential quantization parameter.

A video encoding device according to one embodiment of the present invention divides an image into tiles, divides the tiles into CTUs, and encodes video in units of CTUs.The video encoding device includes a header encoding unit that sets the smallest unit of tile size to a predetermined value, and a CT information encoding unit that encodes the CTUs to generate an encoded stream, and the header encoding unit derives the top left position of the tile and the tile size using the smallest unit of tile size.

[Application example]
The above-mentioned video encoding device 11 and video decoding device 31 can be mounted on various devices that transmit, receive, record, and play videos. The video may be a natural video captured by a camera or the like, or an artificial video (including CG and GUI) generated by a computer or the like.

First, with reference to FIG. 20, it will be explained that the above-mentioned video encoding device 11 and video decoding device 31 can be used to transmit and receive videos.

Fig. 20(a) is a block diagram showing the configuration of a transmitting device PROD_A equipped with a video encoding device 11. As shown in Fig. 20(a), the transmitting device PROD_A includes an encoding unit PROD_A1 that obtains encoded data by encoding a video, a modulation unit PROD_A2 that obtains a modulated signal by modulating a carrier wave with the encoded data obtained by the encoding unit PROD_A1, and a transmitting unit PROD_A3 that transmits the modulated signal obtained by the modulation unit PROD_A2. The above-mentioned video encoding device 11 is used as this encoding unit PROD_A1.

The transmitting device PROD_A may further include a camera PROD_A4 for capturing moving images, a recording medium PROD_A5 for recording moving images, an input terminal PROD_A6 for inputting moving images from the outside, and an image processing unit A7 for generating or processing images, as a supply source of moving images to be input to the encoding unit PROD_A1. In Fig. 20(a), a configuration in which the transmitting device PROD_A includes all of these components is illustrated, but some of them may be omitted.

The recording medium PROD_A5 may record unencoded moving images, or may record moving images encoded by an encoding method for recording that is different from the encoding method for transmission. In the latter case, a decoding unit (
It is preferable to use a conductor (not shown) between the two.

Fig. 20(b) is a block diagram showing the configuration of a receiving device PROD_B equipped with a video decoding device 31. As shown in Fig. 20(b), the receiving device PROD_B includes a receiving unit PROD_B1 that receives a modulated signal, a demodulation unit PROD_B2 that obtains coded data by demodulating the modulated signal received by the receiving unit PROD_B1, and a decoding unit PROD_B3 that obtains a video by decoding the coded data obtained by the demodulation unit PROD_B2. The above-mentioned video decoding device 31 is used as this decoding unit PROD_B3.

The receiving device PROD_B may further include, as destinations of the moving image output by the decoding unit PROD_B3, a display PROD_B4 for displaying the moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal PROD_B6 for outputting the moving image to the outside. In Fig. 20(b), a configuration in which the receiving device PROD_B includes all of these components is illustrated, but some of them may be omitted.

The recording medium PROD_B5 may be for recording unencoded video, or may be encoded by an encoding method for recording that is different from the encoding method for transmission. In the latter case, it is preferable to interpose an encoding unit (not shown) between the decoding unit PROD_B3 and the recording medium PROD_B5, which encodes the video acquired from the decoding unit PROD_B3 according to the encoding method for recording.

The transmission medium for transmitting the modulated signal may be wireless or wired. The transmission mode for transmitting the modulated signal may be broadcast (here, this refers to a transmission mode in which the destination is not specified in advance) or communication (here, this refers to a transmission mode in which the destination is specified in advance). In other words, the transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a broadcasting station (such as broadcasting equipment)/receiving station (such as a television receiver) for terrestrial digital broadcasting is an example of a transmitting device PROD_A/receiving device PROD_B that transmits and receives modulated signals via wireless broadcasting. Also, a broadcasting station (such as broadcasting equipment)/receiving station (such as a television receiver) for cable television broadcasting is an example of a transmitting device PROD_A/receiving device PROD_B that transmits and receives modulated signals via cable broadcasting.

In addition, a server (such as a workstation)/client (such as a television receiver, personal computer, smartphone, etc.) of an Internet-based VOD (Video On Demand) service or video sharing service is an example of a transmitting device PROD_A/receiving device PROD_B that transmits and receives modulated signals by communication (usually, in a LAN, either wireless or wired is used as the transmission medium, and in a WAN, wired is used as the transmission medium). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multi-function mobile phone terminals.

The client of the video hosting service has a function to decode the encoded data downloaded from the server and display it on a display, as well as a function to encode the video captured by the camera and upload it to the server. That is, the client of the video hosting service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, with reference to FIG. 21, it will be explained that the above-mentioned video encoding device 11 and video decoding device 31 can be used for recording and playing back video.

Fig. 21(a) is a block diagram showing the configuration of a recording device PROD_C equipped with the above-mentioned video encoding device 11. As shown in Fig. 21(a), the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a video, and a writing unit PROD_C2 that writes the encoded data obtained by the encoding unit PROD_C1 onto a recording medium PROD_M.
is used as this encoding unit PROD_C1.

The recording medium PROD_M may be (1) a type built into the recording device PROD_C, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), (2) a type connected to the recording device PROD_C, such as an SD memory card or USB (Universal Serial Bus) flash memory, or (3) a type loaded into a drive device (not shown) built into the recording device PROD_C, such as a DVD (Digital Versatile Disc: registered trademark) or BD (Blu-ray Disc: registered trademark).

The recording device PROD_C may further include a camera PROD_C3 for capturing moving images, an input terminal PROD_C4 for inputting moving images from the outside, a receiving unit PROD_C5 for receiving moving images, and an image processing unit PROD_C6 for generating or processing images, as a supply source of moving images to be input to the encoding unit PROD_C1. Although Fig. 21(a) illustrates a configuration in which the recording device PROD_C includes all of these, some of them may be omitted.

The receiving unit PROD_C5 may receive unencoded video, or may receive encoded data encoded by an encoding method for transmission different from the encoding method for recording. In the latter case, a transmission decoding unit (not shown) for decoding the encoded data encoded by the encoding method for transmission may be interposed between the receiving unit PROD_C5 and the encoding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, and an HDD (Hard Disk Drive) recorder (in this case, the input terminal PROD_C4 or the receiving unit PROD_C5 is the main source of the moving image). Also, there are camcorders (in this case, the camera PROD_C3 is the main source of the moving image), personal computers (in this case, the receiving unit PROD_C5 or the image processing unit PROD_C6 is the main source of the moving image), smartphones (in this case, the camera PROD_C3
or the receiving unit PROD_C5 is the main source of video images), such a recording device PROD_C
This is an example.

Fig. 21(B) is a block diagram showing the configuration of a playback device PROD_D equipped with the above-mentioned video decoding device 31. As shown in Fig. 21(b), the playback device PROD_D includes a reading unit PROD_D1 that reads out coded data written to a recording medium PROD_M, and a decoding unit PROD_D2 that obtains video by decoding the coded data read by the reading unit PROD_D1. The above-mentioned video decoding device 31 is used as this decoding unit PROD_D2.

The recording medium PROD_M may be (1) a type that is built into the playback device PROD_D, such as an HDD or SSD, (2) a type that is connected to the playback device PROD_D, such as an SD memory card or USB flash memory, or (3) a type that is loaded into a drive device (not shown) built into the playback device PROD_D, such as a DVD or BD.

The playback device PROD_D may further include, as destinations of the video output by the decoding unit PROD_D2, a display PROD_D3 for displaying the video, an output terminal PROD_D4 for outputting the video to the outside, and a transmission unit PROD_D5 for transmitting the video. In Fig. 21(b), a configuration in which the playback device PROD_D includes all of these components is illustrated, but some of them may be omitted.

The transmission unit PROD_D5 may transmit unencoded video, or may transmit encoded data encoded by an encoding method for transmission different from the encoding method for recording. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes the video by the encoding method for transmission between the decoding unit PROD_D2 and the transmission unit PROD_D5.

Examples of such a playback device PROD_D include a DVD player, a BD player, and an HDD player (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is the main destination of the video image). Other examples of such a playback device PROD_D include a television receiver (in this case, the display PROD_D3 is the main destination of the video image), a digital signage (also called an electronic billboard or electronic bulletin board, and the display PROD_D3 or the transmission unit PROD_D5 is the main destination of the video image), a desktop PC (in this case, the output terminal PROD_D4 or the transmission unit PROD_D5 is the main destination of the video image), a laptop or tablet PC (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main destination of the video image), and a smartphone (in this case, the display PROD_D3 or the transmission unit PROD_D5 is the main destination of the video image).

(Hardware and Software Realizations)
Furthermore, each block of the above-mentioned video decoding device 31 and video encoding device 11 may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be realized by a CPU.
This may be realized in software using a Central Processing Unit (Central Processing Unit).

In the latter case, each of the above devices is composed of a CPU that executes the commands of a program that realizes each function, a ROM (Read Only Memory) that stores the above program, and a RAM (Random Access Memory) that expands the above program.
The present invention also includes a storage device (recording medium) such as a memory for storing the above-mentioned programs and various data, and the like. The object of the embodiment of the present invention can also be achieved by supplying each of the above-mentioned devices with a recording medium on which the program code (executable program, intermediate code program, source program) of the control program of each of the above-mentioned devices, which is software for realizing the above-mentioned functions, is recorded in a computer-readable manner, and the computer (or CPU or MPU) reads and executes the program code recorded on the recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy disks (registered trademark) and hard disks, disks including optical disks such as CD-ROMs (Compact Disc Read-Only Memory), MO disks (Magneto-Optical discs), MDs (Mini Discs), DVDs (Digital Versatile Discs), CD-Rs (CD Recordable), and Blu-ray Discs (registered trademark), cards such as IC cards (including memory cards) and optical cards, semiconductor memories such as mask ROMs, EPROMs (Erasable Programmable Read-Only Memory), EEPROMs (Electrically Erasable and Programmable Read-Only Memory (registered trademark)), and flash ROMs, and logic circuits such as PLDs (Programmable logic devices) and FPGAs (Field Programmable Gate Arrays).

Moreover, each of the above devices may be configured to be connectable to a communication network, and the above program code may be supplied via the communication network. This communication network is not particularly limited as long as it is capable of transmitting the program code. For example, the Internet, an intranet, an extranet, a LAN (Local Area Network), or an ISDN (Integrated Services Digital Network) may be used.
network), VAN (Value-Added Network), CATV (Community Antenna television/Cable Television) communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, etc. can be used. In addition, the transmission medium constituting this communication network is not limited to a specific configuration or type as long as it is a medium capable of transmitting the program code. For example, it can be wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared such as IrDA (Infrared Data Association) or remote control, BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone network, satellite line, terrestrial digital broadcasting network, etc. It should be noted that the embodiment of the present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means that are appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

Embodiments of the present invention can be suitably applied to a video decoding device that decodes coded data in which image data is coded, and a video coding device that generates coded data in which image data is coded. They can also be suitably applied to the data structure of coded data that is generated by a video coding device and referenced by the video decoding device.

11 Video encoding device 31 Video decoding device 101, 308 Prediction image generating unit 104 Entropy encoding unit (encoding unit)
107, 305 Loop filter 111 Parameter coding unit 1111 CT information coding unit (CT division unit)
112 Inter prediction parameter encoding unit 113 Intra prediction parameter encoding unit (intra prediction unit)
301 Entropy decoding unit 302 Parameter decoding unit (division unit)
3021 CT information decoding unit (CT division unit)
303 Inter prediction parameter decoding unit 304 Intra prediction parameter decoding unit (intra prediction unit)
308: predicted image generation unit 309: inter predicted image generation unit 310: intra predicted image generation unit 3036: merge prediction parameter derivation unit 30361: merge candidate derivation unit (temporal motion prediction unit)
30362 Merge candidate selection unit

Claims (4)

a merge candidate derivation unit that stores, in a merge candidate list, temporal motion vectors that are merge candidates for the current block;
deriving a Y coordinate of a block at a first position of the target block by adding a height of the target block to the Y coordinate of the target block;
deriving an X-coordinate of the block at the first position by adding a width of the target block to the X-coordinate of the target block;
deriving a Y coordinate of a block at a second position of the target block by adding 1/2 the height of the target block to the Y coordinate of the target block;
a temporal motion prediction unit configured to derive an X coordinate of the block at the second position by adding 1/2 the width of the target block to the X coordinate of the target block, wherein when the Y coordinate of the target block and the Y coordinate of the block at the first position are in the same CTU, and a value obtained by subtracting the Y coordinate of the target tile from the Y coordinate of the block at the first position is smaller than a height of the target tile, and a value obtained by subtracting the X coordinate of the target tile from the X coordinate of the block at the first position is smaller than a width of the target tile, the temporal motion prediction unit derives the temporal motion vector using a motion vector of a reference picture identified using the Y coordinate and the X coordinate of the block at the first position,
If not, the temporal motion prediction unit derives the temporal motion vector using a motion vector of a reference picture identified using the Y coordinate and the X coordinate of the block at the second position. 2. The video decoding device according to claim 1, wherein whether or not the Y coordinate of the target block and the Y coordinate of the block at the first position are within the same CTU is determined by right-shifting the Y coordinate of the target block by a logarithmic value of the CTU size. The video decoding device according to claim 1 or 2, characterized in that the coordinates of the block at the first position or the block at the second position are multiples of 8x8. a merge candidate derivation unit that stores, in a merge candidate list, temporal motion vectors that are merge candidates for the current block;
deriving a Y coordinate of a block at a first position of the target block by adding a height of the target block to the Y coordinate of the target block;
deriving an X-coordinate of the block at the first position by adding a width of the target block to the X-coordinate of the target block;
deriving a Y coordinate of a block at a second position of the target block by adding 1/2 the height of the target block to the Y coordinate of the target block;
a temporal motion prediction unit configured to derive an X coordinate of the block at the second position by adding 1/2 the width of the target block to the X coordinate of the target block, wherein when the Y coordinate of the target block and the Y coordinate of the block at the first position are in the same CTU, and a value obtained by subtracting the Y coordinate of the target tile from the Y coordinate of the block at the first position is smaller than a height of the target tile, and a value obtained by subtracting the X coordinate of the target tile from the X coordinate of the block at the first position is smaller than a width of the target tile, the temporal motion prediction unit derives the temporal motion vector using a motion vector of a reference picture identified using the Y coordinate and the X coordinate of the block at the first position,
If not, the temporal motion prediction unit derives the temporal motion vector using a motion vector of a reference picture identified using the Y coordinate and the X coordinate of the block at the second position.

Images