CN112954352B

CN112954352B - Video decoding method and video encoding method

Info

Publication number: CN112954352B
Application number: CN202110289850.8A
Authority: CN
Inventors: 朴银姬
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2015-11-24
Filing date: 2016-11-23
Publication date: 2024-08-09
Anticipated expiration: 2036-11-23
Also published as: US20220272343A1; CN112954351B; JP7069379B2; KR102488976B1; EP3349458A1; KR20180075553A; US20200260086A1; KR102449251B1; EP4398578A2; US11812023B2; US11089306B2; US20210211671A1; CN108293129B; CN108293129A; US10979713B2; JP6866369B2; WO2017090967A1; CN112954351A; JP2018535601A; EP4398578A3

Abstract

A video decoding method and a video encoding method are provided. The video decoding method comprises the following steps: obtaining coding order change permission information for an upper data unit including a first block and a second block; when the coding order change permission information indicates that the coding order of the block is permitted to change, obtaining coding order information, and determining the coding order of the first block and the second block based on the coding order information; determining an encoding order of the first block and the second block based on a first encoding order when the encoding order change permission information indicates that the encoding order of the blocks is not permitted to change; and decoding the first block and the second block according to the determined coding order.

Description

Video decoding method and video encoding method

The application is a divisional application of an application patent application with the application number of '2016680068542. X' and the title of 'coding sequence coding method and equipment and decoding method and equipment', wherein the application date is 2016, 11 and 23.

Technical Field

The present disclosure relates to a method of encoding and decoding video, and more particularly, to an intra or inter prediction technique for a method and apparatus for determining an encoding and decoding order with respect to an image.

Background

When high quality video is encoded, a large amount of data is required. However, since the bandwidth allowed for transmission of video data is limited, the data rate applied to the transmission of video data may be limited. Therefore, for efficient transmission of video data, video data encoding and decoding methods having minimal image quality degradation and increased compression rate are required.

Video data may be compressed by removing spatial redundancy and temporal redundancy between pixels. Because adjacent pixels generally have a common characteristic, encoded information of a data unit composed of pixels is transmitted to remove redundancy between adjacent pixels.

The pixel values of the pixels included in the data unit are not directly transmitted, but information about a method of obtaining the pixel values is transmitted. A prediction method for predicting a pixel value similar to the original value is determined for each data unit, and encoding information about the prediction method is transmitted from the encoder to the decoder. Because the predicted value is not exactly equal to the original value, residual data of the difference between the original value and the predicted value is transmitted from the encoder to the decoder.

When prediction is accurate, the data amount of encoding information for specifying a prediction method increases, but the size of residual data decreases. Accordingly, the prediction method is determined by considering the size of the encoded information and the residual data. In particular, data units divided from a picture have various sizes, and in this regard, when the size of a data unit increases, the probability of the accuracy of prediction decreasing increases, however, the data amount of encoded information decreases. Therefore, the size of the block is determined according to the characteristics of the picture.

Prediction methods include intra prediction (intra prediction) and inter prediction (inter prediction). Intra-prediction involves predicting a pixel of a block from neighboring pixels of the block. Inter prediction involves predicting pixels by referencing pixels of different pictures referenced by a picture comprising a block. Thus, spatial redundancy is removed by intra prediction, and temporal redundancy is removed by inter prediction.

When the number of prediction methods increases, the amount of encoding information for indicating the prediction methods increases. Accordingly, when the coding information to be applied to the block is predicted from different blocks, the amount of coding information can be reduced.

Since loss of video data is allowed to the extent that the human eye cannot recognize the loss, residual data can be lossy-compressed according to the transform and quantization process, and by doing so, the amount of residual data can be reduced.

Disclosure of Invention

Technical problem

Provided is a video encoding method for encoding video by determining an optimal encoding order of blocks. Provided is a video decoding method for decoding a video according to a determined coding order. Provided is a computer-readable recording medium having recorded thereon a program for executing a video encoding method and a video decoding method by using a computer.

Solution scheme

Provided is a video decoding method, the video decoding method including: obtaining coding order information indicating whether the coding order of a first block and a second block adjacent to each other is changed; determining an encoding order of the first block and the second block based on the encoding order information; and decoding the first block and the second block according to the determined coding order.

Provided is a video encoding method, the video encoding method including: determining whether the coding order of the first block and the second block adjacent to each other is changed; encoding the first block and the second block according to whether the encoding order is changed; and outputting a bitstream including encoding order information indicating whether the encoding order is changed and encoding information of the first block and the second block.

Provided is a video decoding device including: a coding order information acquirer configured to acquire coding order information indicating whether a coding order of a first block and a second block adjacent to each other is changed; an encoding order determiner configured to determine an encoding order of the first block and the second block based on the encoding order information; and a decoder configured to decode the first block and the second block according to the determined coding order.

Provided is a video encoding device including: a coding order determiner configured to determine whether a coding order of a first block and a second block adjacent to each other is changed; an encoder configured to encode the first block and the second block according to whether the encoding order is changed; and an output unit configured to output a bitstream including encoding order information indicating whether the encoding order is changed and encoding information of the first block and the second block.

The technical problems of the present disclosure are not limited to the foregoing technical features, and other technical problems not stated may be inferred from the following embodiments.

The disclosed advantageous effects

The prediction efficiency for a block can be improved by adjusting the coding order of the block. Therefore, the efficiency of encoding and decoding video can be improved.

Drawings

Fig. 1a illustrates a block diagram of an image encoding apparatus based on an encoding unit according to a tree structure according to an embodiment of the present disclosure.

Fig. 1b illustrates a block diagram of an image decoding apparatus based on an encoding unit according to a tree structure according to an embodiment.

Fig. 2 illustrates a process of determining at least one coding unit when a current coding unit is partitioned, according to an embodiment.

Fig. 3 illustrates a process of determining at least one coding unit when a coding unit having a non-square shape is divided according to an embodiment.

Fig. 4 illustrates a process of dividing an encoding unit based on at least one of block shape information and division shape information according to an embodiment.

Fig. 5 illustrates a method of determining a predetermined coding unit from among an odd number of coding units according to an embodiment.

Fig. 6 illustrates an order of processing a plurality of coding units when the plurality of coding units are determined while the current coding unit is divided, according to an embodiment.

Fig. 7 illustrates a process of determining that a current coding unit is divided into an odd number of coding units when the coding units cannot be processed in a predetermined order according to an embodiment.

Fig. 8 illustrates a process of determining at least one coding unit when a first coding unit is partitioned, according to an embodiment.

Fig. 9 illustrates that the shape of a second coding unit, which is partitionable when a second coding unit having a non-square shape determined when the first coding unit is partitioned, satisfies a predetermined condition, is limited according to an embodiment.

Fig. 10 illustrates a process of dividing an encoding unit having a square shape when division shape information does not indicate division of the encoding unit into four encoding units having a square shape according to an embodiment.

Fig. 11 illustrates that the processing order between a plurality of coding units may be changed according to the division process of the coding units according to an embodiment.

Fig. 12 illustrates a process of determining a depth of an encoding unit when a shape and a size of the encoding unit are changed in a case where a plurality of encoding units are determined when the encoding unit is recursively divided according to an embodiment.

Fig. 13 illustrates a depth determinable according to the shape and size of an encoding unit and a Partial Index (PID) for distinguishing between encoding units according to an embodiment.

Fig. 14 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to an embodiment.

Fig. 15 illustrates a processing block as a criterion when determining the determination order of the reference coding units included in the picture according to the embodiment.

Fig. 16 illustrates a video decoding apparatus related to determining an encoding order of blocks according to an embodiment.

Fig. 17 illustrates a video encoding apparatus involving determining an encoding order of blocks according to an embodiment.

Fig. 18 is a diagram for describing the necessity of changing the coding order of blocks.

Fig. 19a and 19b illustrate an embodiment of a method of determining the coding order of blocks.

Fig. 20 illustrates a method of comparing coding efficiency to determine whether to change the coding order of blocks.

Fig. 21 illustrates reference sampling points to be used when predicting a block according to an intra mode.

Fig. 22a to 22e illustrate an intra prediction method to be performed on a current block when reconstructing a right block of the current block by changing the coding order of the blocks.

Fig. 23 illustrates a reference block to be used when predicting a block according to an inter mode.

Fig. 24 illustrates a video decoding method performed by a video decoding apparatus according to an embodiment.

Fig. 25 illustrates a video encoding method performed by a video encoding apparatus according to an embodiment.

Best mode

Provided is a video decoding method, the video decoding method including: obtaining coding order information indicating whether the coding order of a first block and a second block adjacent to each other is changed; determining the coding order of the first block and the second block based on the coding order information; and decoding the first block and the second block according to the determined coding order.

Detailed Description

Advantages and features of one or more embodiments of the present disclosure and methods of accomplishing the same may be understood more readily by reference to the following detailed description of the embodiments and the accompanying drawings. In this regard, the present disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the embodiments to those skilled in the art.

Hereinafter, terms used in the present specification will be briefly defined, and embodiments will be described in detail.

All terms including descriptive or technical terms used herein should be interpreted to have meanings apparent to those of ordinary skill in the art. However, these terms may have different meanings according to the intention, precedent, or appearance of new technologies of those of ordinary skill in the art. In addition, some terms may be arbitrarily selected by the present inventors, and in this case, meanings of the selected terms will be described in detail in the detailed description of the present disclosure. Accordingly, the terms used herein must be defined based on the meanings of the terms, and the descriptions throughout the specification.

The use of the singular encompasses the plural unless the context otherwise has a distinct meaning.

When a portion "comprises" or "comprising" an element, unless a specific description to the contrary exists, the portion may further include other elements without excluding the other elements. In addition, the term "unit" in the embodiments of the present disclosure means a software component or a hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and performs a specific function. However, the term "unit" is not limited to software or hardware. The "units" may be formed so as to be located in an addressable storage medium or may be formed so as to operate one or more processors. Thus, for example, the term "unit" may refer to a component, such as a software component, an object-oriented software component, a class component, and a task component, and may include a process, a function, an attribute, a program, a subroutine, a program code segment, a driver, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, variables, and the like. The functionality provided for by the components and "units" may be associated with a fewer number of components and "units" or may be divided into additional components and "units".

The term "current block" refers to one of a coding unit, a prediction unit, and a transform unit that is currently to be encoded or decoded. In addition, the term "lower block" refers to a data unit divided from the "current block". The term "upper level block" refers to a data unit that includes a "current block".

Hereinafter, the "sampling point" is data assigned to a sampling position of an image and may mean data as a processing target. For example, pixel values in an image in the spatial domain or transform coefficients on the transform domain may be sampling points. A unit including at least one sampling point may be defined as a block.

The present disclosure will now be described more fully with reference to the accompanying drawings so that one of ordinary skill in the art can perform the disclosure without difficulty. In the following description, well-known functions or constructions are not described in detail to avoid obscuring the embodiments in unnecessary detail.

Fig. 1a illustrates a block diagram of an image encoding apparatus 100 based on an encoding unit according to a tree structure according to an embodiment of the present disclosure.

The image encoding apparatus 100 includes a maximum encoding unit determiner 110, an encoding unit determiner 120, and an output unit 130.

The maximum coding unit determiner 110 divides a picture or slices (slices) included in the picture into a plurality of maximum coding units according to the size of the maximum coding unit. The maximum coding unit may be a data unit of size 32×32, 64×64, 128×128, 256×256, etc., wherein the shape of the data unit is a square having a width and a length in several squares of 2. The maximum coding unit determiner 110 may provide the maximum coding unit size information indicating the size of the maximum coding unit to the output unit 130. The output unit 130 may include maximum coding unit size information in the bitstream.

The coding unit determiner 120 determines the coding unit by dividing the maximum coding unit. The coding unit may be determined by its maximum size and depth. Depth may be defined as the number of times a coding unit is spatially partitioned from a maximum coding unit. When the depth increases by 1, the coding unit is divided into at least two coding units. Therefore, as the depth increases, the size of the coding unit according to the depth decreases. Whether to partition the coding unit is determined by whether the partition coding unit is efficient based on rate distortion optimization. Then, partition information indicating whether the coding unit has been partitioned may be generated. The segmentation information may be represented in the form of a flag.

The coding units may be partitioned by using various methods. For example, a square coding unit may be divided into four square coding units having a width and a height that are half the width and the height of the square coding unit. The square coding unit may be divided into two rectangular coding units having half width. The square coding unit may be divided into two rectangular coding units having half height. Square coding units may be partitioned into three coding units with their width or height being partitioned 1:2:1.

Rectangular coding units having a width twice the height may be divided into two square coding units. A rectangular coding unit having a width twice as large as a height may be divided into two rectangular coding units having a width four times as large as a height. Rectangular coding units having a width twice as large as a height may be divided into two rectangular coding units and one square coding unit in such a manner that the width is divided by 1:2:1.

Likewise, rectangular coding units having a height twice the width may be divided into two square coding units. A rectangular coding unit having a height twice the width may be divided into two rectangular coding units having a height four times the width. Likewise, rectangular coding units having a height twice the width may be divided into two rectangular coding units and one square coding unit in such a manner that the height is divided by 1:2:1.

When the image encoding apparatus 100 is capable of using two or more kinds of division methods, information on division methods available for an encoding unit from among the division methods available to the image encoding apparatus 100 may be determined for each picture. Thus, only a specific segmentation method can be used for each picture. When the image encoding apparatus 100 uses only one division method, information about the division methods available for the encoding units is not determined separately.

When the division information of the encoding unit indicates that the encoding unit is divided, division shape information indicating a division method with respect to the encoding unit may be generated. When only one segmentation method is available in a picture including an encoding unit, segmentation shape information may not be generated. When the segmentation method is determined to be suitable for the coding information adjacent to the coding unit, the segmentation shape information may not be generated.

The maximum coding unit may be divided into minimum coding units according to the minimum coding unit size information. The depth of the largest coding unit may be defined as the highest depth and the depth of the smallest coding unit may be defined as the lowest depth. Thus, a coding unit having a higher depth may include a plurality of coding units having a lower depth.

According to the maximum size of the coding unit as described above, the image data of the current picture is divided into the maximum coding units. The maximum coding unit may include a coding unit divided according to depth. Since the maximum coding unit is segmented according to depth, image data of a spatial domain included in the maximum coding unit can be hierarchically segmented according to depth.

A maximum depth limiting the maximum number of hierarchically split maximum coding units or a minimum size of coding units may be preset.

The coding unit determiner 120 compares the coding efficiency of the hierarchically partitioned coding units with the coding efficiency of the non-partitioned coding units. Then, the coding unit determiner 120 determines whether to divide the coding unit according to the result of the comparison. When the coding unit determiner 120 determines that dividing the coding unit is more efficient, the coding unit determiner 120 hierarchically divides the coding unit. However, according to the result of the comparison, when the coding unit determiner 120 determines that it is more efficient to not divide the coding unit, the coding unit determiner 120 does not divide the coding unit. Whether to partition the coding unit may be determined independently of whether neighboring different coding units are partitioned.

According to an embodiment, it may be determined whether to partition the coding unit from the coding unit having a large depth during the encoding procedure. For example, the coding efficiency of the coding unit having the maximum depth is compared with the coding efficiency of the coding unit having a depth smaller than the maximum depth by 1, and it is determined which of the coding unit having the maximum depth and the coding unit having a depth smaller than the maximum depth by 1 is efficiently coded in each region of the maximum coding unit. According to the result of the determination, it is determined whether to divide the coding unit having a depth smaller than the maximum depth by 1 in each region of the maximum coding unit. Thereafter, it is determined which of the coding unit having the depth less than the maximum depth of 2 and one of the coding unit having the depth less than the maximum depth of 1, which one has been selected according to the result of the determination, is further efficiently encoded in each region of the maximum coding unit. The same determination process is performed for each of the coding units having smaller depths, and finally, whether to partition the maximum coding unit is determined according to which of the maximum coding unit and the hierarchical structure generated by hierarchically partitioning the maximum coding unit is further efficiently encoded.

It may be determined whether to partition the coding unit from the coding unit having a small depth during the encoding procedure. For example, the coding efficiency of the maximum coding unit is compared with the coding efficiency of the coding unit having a depth greater than 1 than the maximum coding unit, and it is determined which of the maximum coding unit and the coding unit having a depth greater than 1 than the maximum coding unit is efficiently coded. When the coding efficiency of the maximum coding unit is better, the maximum coding unit is not divided. When the coding efficiency of the coding unit whose depth is 1 greater than the maximum coding unit is better, the maximum coding unit is divided, and the comparison process is equally applied to the divided coding units.

When the coding efficiency is checked from the coding unit having a large depth, a tree structure having a large but high coding efficiency is calculated. In contrast, when the coding efficiency is checked from the coding unit having a small depth, a tree structure that is small but has a low coding efficiency is calculated. Therefore, in consideration of coding efficiency and computation, an algorithm for obtaining a hierarchical tree structure of the maximum coding unit can be designed by using various methods.

In order to determine the efficiency of the coding unit according to each depth, the coding unit determiner 120 determines the most efficient prediction and transformation method for the coding unit. To determine the most efficient prediction and transformation method, the coding unit may be partitioned into predetermined data units. The data unit may have one of various shapes according to a method of dividing the coding unit. The method performed to determine the partitioned coded units of a data unit may be defined as a partition mode. For example, when a coding unit of 2n×2n (where N is a positive integer) is no longer divided, the size of a prediction unit included in the coding unit is 2n×2n. When a 2n×2n coding unit is divided, the size of a prediction unit included in the coding unit may be 2n× N, N ×2n or n×n according to a partition mode. The partition mode according to the present embodiment can generate symmetrical data units obtained by dividing the height or width of the coding unit symmetrically, data units obtained by dividing the height or width of the coding unit asymmetrically (such as 1: n or n: 1), data units obtained by dividing the coding unit diagonally, data units obtained by dividing the coding unit geometrically, partitions having arbitrary shapes, and the like.

The coding unit may be predicted and transformed based on the data units included in the coding unit. However, according to the present embodiment, the data unit for prediction and the data unit for transformation may be separately determined. The data unit for prediction may be defined as a prediction unit and the data unit for transformation may be defined as a transformation unit. The partition mode applied to the prediction unit and the partition mode applied to the transform unit may be different from each other, and prediction of the prediction unit and transformation of the transform unit may be performed in the encoding unit in a parallel and independent manner.

In order to determine an efficient prediction method, the coding unit may be partitioned into at least one prediction unit. Likewise, in order to determine an efficient transformation method, the coding unit may be partitioned into at least one transformation unit. The division into prediction units and the division into transform units may be performed independently of each other. However, when reconstructed sampling points in the coding unit are used in intra prediction, a dependency is formed between the prediction units or the transform units included in the coding unit such that the division into the prediction units and the transform units may affect each other.

The prediction unit included in the encoding unit may be predicted by intra prediction or inter prediction. Intra prediction involves predicting a prediction unit sampling point by using a reference sampling point adjacent to the prediction unit. Inter prediction involves predicting a prediction unit sampling point by obtaining a reference sampling point from a reference picture referenced by a current picture.

For intra prediction, the coding unit determiner 120 may apply a plurality of intra prediction methods to the prediction unit, thereby selecting the most efficient intra prediction method. The intra prediction method includes a direction mode such as a DC mode, a plane mode, a vertical mode, a horizontal mode, and the like.

When a reconstructed sampling point adjacent to the coding unit is used as a reference sampling point, intra prediction may be performed for each prediction unit. However, when a reconstructed sampling point in the coding unit is used as a reference sampling point, reconstruction in the coding unit with respect to the reference sampling point must precede prediction in the coding unit with respect to the reference sampling point, so that the prediction order of the prediction unit may depend on the transform order of the transform unit. Therefore, when a reconstructed sampling point in an encoding unit is used as a reference sampling point, an intra prediction method and actual intra prediction for only a transform unit corresponding to a prediction unit can be performed for each transform unit.

The coding unit determiner 120 may determine the optimal motion vector and the reference picture, thereby selecting the most efficient inter prediction method. For inter prediction, the coding unit determiner 120 may determine a plurality of motion vector candidates from coding units spatially and temporally adjacent to the current coding unit, and may determine the most efficient motion vector from them as a motion vector. Likewise, the coding unit determiner 120 may determine a plurality of reference picture candidates from coding units spatially and temporally adjacent to the current coding unit, and may determine the most efficient reference picture from among them. In another embodiment, the reference picture may be determined from a list of reference pictures that are predetermined with respect to the current picture. In another embodiment, for the accuracy of prediction, the most efficient motion vector from among the plurality of motion vector candidates may be determined as the predicted motion vector, and the motion vector may be determined by compensating the predicted motion vector. Inter prediction may be performed in parallel for each prediction unit in the coding unit.

The coding unit determiner 120 may reconstruct the coding unit by obtaining only information indicating the motion vector and the reference picture according to the skip mode. According to the skip mode, all encoded information including a residual signal is skipped except for information indicating a motion vector and a reference picture. Since the residual signal is skipped, the skip mode may be used when the accuracy of prediction is very high.

The partition mode to be used may be limited according to a prediction method for the prediction unit. For example, only a partition mode of a prediction unit having a size of 2n×2n or n×n may be applied to intra prediction, whereas a partition mode of a prediction unit having a size of 2n×2n, 2n× N, N ×2n or n×n may be applied to inter prediction. In addition, the partition mode of only the prediction unit having a size of 2n×2n may be applied to the skip mode of the inter prediction. The image encoding apparatus 100 may change the partition mode for each prediction method according to the encoding efficiency.

The image encoding apparatus 100 may perform the transformation based on an encoding unit or a transformation unit included in the encoding unit. The image encoding apparatus 100 may transform residual data, which is a difference between an original value and a predicted value with respect to a pixel included in the encoding unit. For example, the image encoding apparatus 100 may perform lossy compression on residual data by quantization and Discrete Cosine Transform (DCT)/Discrete Sine Transform (DST). Or the image encoding apparatus 100 may perform lossless compression on the residual data without quantization.

The image encoding apparatus 100 can determine a transform unit that is most efficient for quantization and transform. According to an embodiment, transform units in a coding unit may be recursively partitioned into smaller sized regions in a similar manner as the coding unit is partitioned according to a tree structure. Accordingly, residual data in the encoding unit may be divided according to a transform unit having a tree structure according to a transform depth. The image encoding apparatus 100 may generate transform partition information about the partition encoding unit and the transform unit according to the determined tree structure of the transform unit.

The image encoding device 100 may be provided with a conversion depth indicating the number of times of division of the conversion unit by dividing the height and width of the encoding unit. For example, in a current coding unit of 2n×2n, the transform depth may be 0 when the size of the transform unit is 2n×2n, 1 when the size of the transform unit is n×n, and 2 when the size of the transform unit is N/2×n/2. That is, the transformation unit according to the tree structure may be set according to the transformation depth.

In summary, the coding unit determiner 120 determines a prediction method that is most efficient for the current prediction unit and is from among a plurality of intra prediction methods and inter prediction methods. Then, the coding unit determiner 120 determines a prediction unit determination scheme according to the coding efficiency according to the prediction result. Likewise, the coding unit determiner 120 determines a transform unit determination scheme according to coding efficiency from the transform result. The coding efficiency of the coding unit is finally determined according to the most efficient prediction unit and transform unit determination scheme. The coding unit determiner 120 finalizes the hierarchical structure of the maximum coding unit according to the coding efficiency of the coding unit according to each depth.

The coding unit determiner 120 may measure coding efficiency of the coding unit according to depth, prediction efficiency of the prediction method, etc. by using rate distortion optimization based on the lagrangian multiplier.

The coding unit determiner 120 may generate partition information indicating whether to partition the coding unit according to each depth according to the determined hierarchical structure of the maximum coding unit. Then, the coding unit determiner 120 may generate partition mode information to be used to determine a prediction unit and transform unit partition information to be used in determining a transform unit for the partitioned coding unit. In addition, when the encoding unit may be divided by using at least two division methods, the encoding unit determiner 120 may generate both division information and division shape information indicating the division methods. The coding unit determiner 120 may generate information about the prediction method and the transformation method used in the prediction unit and the transformation unit.

The output unit 130 may output pieces of information generated by the maximum coding unit determiner 110 and the coding unit determiner 120 according to the hierarchical structure of the maximum coding unit in the bitstream.

A method of determining a coding unit, a prediction unit, and a transform unit according to a tree structure of a maximum coding unit will be described below with reference to fig. 3 to 12.

Fig. 1b illustrates a block diagram of an image decoding apparatus 150 based on an encoding unit according to a tree structure according to an embodiment.

The image decoding apparatus 150 includes a receiver 160, an encoded information extractor 170, and an image data decoder 180.

The definitions of terms including coding units, depths, prediction units, transform units, various partition information, etc. for the decoding operation performed by the image decoding apparatus 150 are equal to those described above with reference to fig. 1a and the image encoding apparatus 100. Since the image decoding apparatus 150 is designed to reconstruct image data, various encoding methods used by the image encoding apparatus 100 may also be applied to the image decoding apparatus 150.

The receiver 160 receives and parses a bitstream regarding the encoded video. The encoding information extractor 170 extracts pieces of information to be used in decoding the maximum encoding unit from the parsed bitstream and supplies them to the image data decoder 180. The encoding information extractor 170 may extract information about the maximum size of the encoding unit of the current picture from the header, the sequence parameter set, or the picture parameter set of the current picture.

The encoding information extractor 170 extracts final depth and partition information about the encoding units according to the tree structure from the parsed bitstream according to each maximum encoding unit. The extracted final depth and segmentation information is output to the image data decoder 180. The image data decoder 180 may divide the maximum coding unit according to the extracted final depth and division information, thereby determining a tree structure of the maximum coding unit.

The partition information extracted by the encoding information extractor 170 is partition information about a tree structure determined to generate a minimum encoding error, the determination being performed by the image encoding apparatus 100. Accordingly, the image decoding apparatus 150 may reconstruct an image by decoding data according to a decoding scheme that generates a minimum coding error.

The encoding information extractor 170 may extract partition information about data units such as a prediction unit and a transform unit included in the encoding unit. For example, the encoding information extractor 170 may extract partition mode information regarding a partition mode that is most efficient for the prediction unit. The encoding information extractor 170 may extract transform partition information about a tree structure that is most efficient for the transform unit.

The encoding information extractor 170 may obtain information about the most efficient prediction method for the prediction unit divided from the encoding unit. Then, the encoded information extractor 170 may obtain information about the most efficient transform method for the transform unit divided from the encoding unit.

The encoding information extractor 170 extracts information from the bitstream according to a method of configuring the bitstream, which is performed by the output unit 130 of the image encoding apparatus 100.

The image data decoder 180 may divide the maximum coding unit into coding units having the most efficient tree structure based on the division information. Then, the image data decoder 180 may divide the encoding unit into prediction units according to the partition mode information. The image data decoder 180 may divide the encoding unit into transform units according to the transform division information.

The image data decoder 180 may predict the prediction unit according to information about the prediction method. The image data decoder 180 may perform inverse quantization and inverse transformation on residual data, which is a difference between an original value and a predicted value of a pixel, according to information about a method of transforming a transformation unit. The image data decoder 180 may reconstruct pixels of the encoding unit according to the result of prediction of the prediction unit and the result of transformation of the transformation unit.

Fig. 2 illustrates a process of determining at least one coding unit when the image decoding apparatus 150 partitions a current coding unit according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may determine the shape of the encoding unit by using the block shape information, and may determine the shape according to which the encoding unit is to be divided by using the division shape information. That is, the method of dividing the encoding unit indicated by the division shape information may be determined based on which block shape is indicated by the block shape information used by the image decoding apparatus 150.

According to the present embodiment, the image decoding apparatus 150 may use block shape information indicating that the current encoding unit has a square shape. For example, the image decoding apparatus 150 may determine whether to divide the square coding unit, whether to divide the square coding unit vertically, whether to divide the square coding unit horizontally, or whether to divide the square coding unit into four coding units according to the division shape information. Referring to fig. 2, when the block shape information of the current coding unit 200 indicates a square shape, the image data decoder 180 may not divide the coding unit 210a having the same size as the current coding unit 200 according to the division shape information indicating not dividing, or may determine the coding units 210b, 210c, and 210d divided based on the division shape information indicating a predetermined division method.

Referring to fig. 2, according to an embodiment, the image decoding apparatus 150 may determine two encoding units 210b obtained by dividing the current encoding unit 200 in the vertical direction based on division shape information indicating division in the vertical direction. The image decoding apparatus 150 may determine two encoding units 210c obtained by dividing the current encoding unit 200 in the horizontal direction based on division shape information indicating division in the horizontal direction. The image decoding apparatus 150 may determine four encoding units 210d obtained by dividing the current encoding unit 200 in the vertical and horizontal directions based on division shape information indicating division in the vertical and horizontal directions. However, the division shape for dividing the square coding unit may not be limitedly interpreted as the above-described shape, and may include various shapes that are indicated by division shape information. The predetermined division shape for dividing the square coding unit will be described in detail below by various embodiments.

Fig. 3 illustrates a process of determining at least one coding unit when the image decoding apparatus 150 partitions coding units having a non-square shape according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may use block shape information indicating that the current encoding unit has a non-square shape. The image decoding apparatus 150 may determine whether to divide the current coding unit having the non-square shape or whether to divide the current coding unit having the non-square shape by using a predetermined method. Referring to fig. 3, when the block shape information of the current coding unit 300 or 350 indicates a non-square shape, the image decoding apparatus 150 may not divide the coding unit 310 or 360 having the same size as the current coding unit 300 or 350 according to division shape information indicating non-division, or may determine the coding units 320a, 320b, 330a, 330b, 330c, 370a, 370b, 380a, 380b, and 380c divided according to division shape information indicating a predetermined division method. The predetermined partitioning method of partitioning non-square coding units will be described in detail below by various embodiments.

According to the present embodiment, the image decoding apparatus 150 may determine that the shape of the encoding unit is divided by using the division shape information, and in this case, the division shape information may indicate the number of at least one encoding unit generated when the encoding unit is divided. Referring to fig. 3, when the division shape information indicates that the current encoding unit 300 or 350 is divided into two encoding units, the image decoding apparatus 150 may determine that the two encoding units 320a and 320b or 370a and 370b included in the current encoding unit 300 or 350, respectively, are divided by dividing the current encoding unit 300 or 350 based on the division shape information.

According to the present embodiment, when the image decoding apparatus 150 partitions the current encoding unit 300 or 350 having a non-square shape based on the partition shape information, the image decoding apparatus 150 may partition the current encoding unit 300 or 350 having a non-square shape in consideration of the position of the longer side. For example, the image decoding apparatus 150 may determine a plurality of encoding units by dividing the current encoding unit 300 or 350 in a direction of dividing the longer side of the current encoding unit 300 or 350 in consideration of the shape of the current encoding unit 300 or 350.

According to the present embodiment, when the division shape information indicates that the encoding unit is divided into an odd number of blocks, the image decoding apparatus 150 may determine an odd number of encoding units included in the current encoding unit 300 or 350. For example, when the division shape information indicates that the current encoding unit 300 or 350 is divided into three encoding units, the image decoding apparatus 150 may divide the current encoding unit 300 or 350 into three encoding units 330a, 330b and 330c or 380a, 380b and 380c. According to the present embodiment, the image decoding apparatus 150 may determine an odd number of encoding units included in the current encoding unit 300 or 350, wherein the determined encoding units are not identical in size. For example, the size of the encoding unit 330b or 380b from among the odd number of encoding units 330a, 330b and 330c or 380a, 380b and 380c may be different from the size of the encoding unit 330a and 330c or 380a or 380c. That is, the coding units that may be determined when the current coding unit 300 or 350 is divided may have different types of sizes.

According to the present embodiment, when the division shape information indicates that the encoding unit is divided into an odd number of blocks, the image decoding apparatus 150 may determine an odd number of encoding units included in the current encoding unit 300 or 350 and, in addition, set a predetermined limit on at least one encoding unit from among the odd number of encoding units generated by dividing the current encoding unit 300 or 350. Referring to fig. 3, the image decoding apparatus 150 may decode the encoding unit 330b or 380b at the center of the encoding unit 330a, 330b, and 330c or 380a, 380b, and 380c generated when the encoding unit 330b or 380b divides the current encoding unit 300 or 350 in a different manner from the encoding units 330a and 330c or 380a and 380 c. For example, the image decoding apparatus 150 may limit the encoding unit 330b or 380b at the center to be further divided unlike the encoding units 330a and 330c or 380a and 380c or to be divided only a certain number of times.

Fig. 4 illustrates a process of dividing an encoding unit based on at least one of block shape information and division shape information by the image decoding apparatus 150 according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may determine whether to divide the first encoding unit 400 having a square shape into encoding units based on at least one of the block shape information and the division shape information. According to the present embodiment, when the division shape information indicates that the first encoding unit 400 is divided in the horizontal direction, the image decoding apparatus 150 may determine the second encoding unit 410 by dividing the first encoding unit 400 in the horizontal direction. The terms "first coding unit", "second coding unit" and "third coding unit" according to the embodiments are used in the context of the split coding units. For example, the second coding unit may be determined when the first coding unit is partitioned and the third coding unit may be determined when the second coding unit is partitioned. The relationship between the first to third coding units used hereinafter may be understood to follow the above-described sequential characteristics.

According to the present embodiment, the image decoding apparatus 150 may determine whether to divide the determined second encoding unit 410 into a plurality of encoding units based on at least one of the block shape information and the division shape information. Referring to fig. 4, the image decoding apparatus 150 may divide the second encoding unit 410 having the non-square shape determined by dividing the first encoding unit 400 into at least one third encoding unit, for example, third encoding units 420a, 420b, 420c and 420d, or may not divide the second encoding unit 410, based on at least one of the block shape information and the division shape information. The image decoding apparatus 150 may obtain at least one of block shape information and division shape information, the image decoding apparatus 150 may divide the first encoding unit 400 based on the at least one of block shape information and division shape information to obtain a plurality of second encoding units (e.g., second encoding units 410) having various shapes, and may divide the second encoding units 410 in such a manner that the first encoding units 400 are divided based on the at least one of block shape information and division shape information. According to the present embodiment, when the first encoding unit 400 is divided into the second encoding units 410 based on at least one of the block shape information and the division shape information about the first encoding unit 400, the second encoding unit 410 may also be divided into the third encoding units, for example, the third encoding units 420a, 420b and 420c, 420d, based on at least one of the block shape information and the division shape information about the second encoding unit 410. That is, the encoding unit may be recursively divided based on at least one of division shape information and block shape information related to the encoding unit. A method for recursively partitioning a coding unit will be described below by various embodiments.

According to the present embodiment, the image decoding apparatus 150 may determine that each of the third encoding units (e.g., the third encoding units 420a, 420b, 420c, and 420 d) is to be divided into encoding units or not to divide the second encoding unit 410 based on at least one of the block shape information and the division shape information. The image decoding apparatus 150 may divide the second encoding unit 410 having a non-square shape into an odd number of third encoding units 420b, 420c and 420d. The image decoding apparatus 150 may set a predetermined limit on a predetermined third encoding unit from among the odd number of third encoding units 420b, 420c, and 420d. For example, the image decoding apparatus 150 may limit the encoding unit 420c located at the center from among the odd number of third encoding units 420b, 420c, and 420d to be no longer divided or to be divided a settable number of times. Referring to fig. 4, the image decoding apparatus 150 may limit the encoding unit 420c located at the center from among the odd-numbered third encoding units 420b, 420c and 420d to be no longer divided, to be divided into a predetermined division manner (for example, to be divided into only four encoding units or to be divided into shapes corresponding to the shapes into which the second encoding unit 410 is divided) or to be divided only a predetermined number of times (for example, to be divided only n times, where n > 0), wherein the odd-numbered third encoding units 420b, 420c and 420d are included in the second encoding unit 410 having a non-square shape. However, the limitation of the encoding unit 420c located at the center is only an embodiment, and thus the present disclosure should not be limitedly construed as the above-described embodiment, and it should be construed that the limitation includes various limitations of decoding the encoding unit 420c located at the center differently from the encoding units 420b and 420d.

According to the present embodiment, the image decoding apparatus 150 may obtain at least one of block shape information and division shape information for dividing the current coding unit from a predetermined position in the current coding unit.

Fig. 5 illustrates a method of determining an encoding unit at a predetermined position from among an odd number of encoding units by the image decoding apparatus 150 according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may use information indicating the position of each of the odd-numbered encoding units so as to determine the encoding unit located at the center of the odd-numbered encoding units. Referring to fig. 5, the image decoding apparatus 150 may determine an odd number of encoding units 520a, 520b, and 520c by dividing the current encoding unit 500. The image decoding apparatus 150 may determine the encoding unit 520b at the center by using information on the positions of the odd-numbered encoding units 520a, 520b, and 520c. For example, the image decoding apparatus 150 may determine the encoding unit 520 located at the center by determining the positions of the encoding units 520a, 520b, and 520c based on information indicating the positions of predetermined sampling points included in the encoding units 520a, 520b, and 520c. In detail, the image decoding apparatus 150 may determine the encoding unit 520b located at the center by determining the positions of the encoding units 520a, 520b, and 520c based on information indicating the positions of the upper left sampling points 530a, 530b, and 530c of the encoding units 520a, 520b, and 520c.

According to the present embodiment, the information indicating the positions of the upper left sampling points 530a, 530b, and 530c included in the encoding units 520a, 520b, and 520c, respectively, may include information about positions or coordinates in the pictures of the encoding units 520a, 520b, and 520 c. According to the present embodiment, the information indicating the positions of the upper left sampling points 530a, 530b, and 530c respectively included in the encoding units 520a, 520b, and 520c may include information indicating the widths or heights of the encoding units 520a, 520b, and 520c included in the current encoding unit 500, wherein the widths or heights may correspond to information indicating differences between coordinates in the pictures of the encoding units 520a, 520b, and 520 c. That is, the image decoding apparatus 150 may determine the encoding unit 520b located at the center by directly using information about positions or coordinates in the pictures of the encoding units 520a, 520b, and 520c or by using information about the width or height of the encoding units indicating a difference between the coordinates.

According to the present embodiment, the information indicating the position of the upper left sampling point 530a of the top encoding unit 520a may indicate (xa, ya) coordinates, the information indicating the position of the upper left sampling point 530b of the center encoding unit 520b may indicate (xb, yb) coordinates, and the information indicating the position of the upper left sampling point 530c of the bottom encoding unit 520c may indicate (xc, yc) coordinates. The image decoding apparatus 150 may determine the center encoding unit 520b by using coordinates of the upper left sampling points 530a,530b, and 530c respectively included in the encoding units 520a, 520b, and 520 c. For example, when the coordinates of the upper left sampling points 530a,530b, and 530c are aligned in an ascending or descending order, the center coding unit 520b including (xb, yb) as the coordinates of the upper left sampling point 530b may be determined to be a coding unit located at the center from among the coding units 520a, 520b, and 520c determined when the current coding unit 500 is divided. Here, the coordinates indicating the positions of the upper left sampling points 530a,530b, and 530c may represent coordinates indicating absolute positions in a picture, and further, the (dxb, dyb) coordinates, which are information indicating the relative position of the upper left sampling point 530b of the center coding unit 520b, and the (dxc, dyc) coordinates, which are information indicating the relative position of the upper left sampling point 530c of the bottom coding unit 520c, may be used based on the position of the upper left sampling point 530c of the top coding unit 520 a. In addition, a method of determining the encoding unit at the predetermined position by using the coordinates of the sampling point included in the encoding unit as information indicating the position of the sampling point should not be interpreted as the above-described method in a limited manner, and may be interpreted as various arithmetic methods capable of using the coordinates of the sampling point.

According to the present embodiment, the image decoding apparatus 150 may divide the current encoding unit 500 into a plurality of encoding units 520a, 520b, and 520c, and select an encoding unit from among the encoding units 520a, 520b, and 520c according to a predetermined criterion. For example, the image decoding apparatus 150 may select the encoding unit 520b having a different size from among the encoding units 520a, 520b, and 520 c.

According to the present embodiment, the image decoding apparatus 150 may determine the width or height of each of the encoding units 520a, 520b, and 520c by using (xa, ya) coordinates, which are information indicating the position of the upper left sampling point 530a of the top encoding unit 520a, (xb, yb) coordinates, which are information indicating the position of the upper left sampling point 530b of the center encoding unit 520b, and (xc, yc) coordinates, which are information indicating the position of the upper left sampling point 530c of the bottom encoding unit 520 c. The image decoding apparatus 150 may determine the size of each of the encoding units 520a, 520b, and 520c by using coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the encoding units 520a, 520b, and 520 c.

According to an embodiment, the image decoding apparatus 150 may determine the width of the top encoding unit 520a as xb-xa and the height as yb-ya. According to this embodiment, the image decoding apparatus 150 may determine the width of the center encoding unit 520b as xc-xb and the height as yc-yb. According to the present embodiment, the image decoding apparatus 150 may determine the width or height of the bottom coding unit by using the width or height of the current coding unit and the widths and heights of the top coding unit 520a and the center coding unit 520 b. The image decoding apparatus 150 may determine one coding unit having a different size from other coding units based on the determined widths and heights of the coding units 520a, 520b, and 520 c. Referring to fig. 5, the image decoding apparatus 150 may determine a center encoding unit 520b having a size different from that of the top encoding unit 520a and the bottom encoding unit 520c as an encoding unit at a predetermined position. However, since the process of determining the encoding unit having a size different from other encoding units by the image decoding apparatus 150 is only an embodiment of determining the encoding unit at the predetermined position by using the size of the encoding unit determined based on the sampling point coordinates, various processes of determining the encoding unit at the predetermined position by comparing the size of the encoding unit determined according to the predetermined sampling point coordinates may be used.

However, the position of the sampling point considered to determine the position of the encoding unit should not be interpreted limitedly as the upper left, but may be interpreted as information about the position of any sampling point included in the encoding unit is available.

According to the present embodiment, the image decoding apparatus 150 may select a coding unit at a predetermined position from among the odd-numbered coding units determined when the current coding unit is divided, taking into consideration the shape of the current coding unit. For example, when the current coding unit has a non-square shape having a width longer than a height, the image decoding apparatus 150 may determine the coding unit at a predetermined position along the horizontal direction. In other words, the image decoding apparatus 150 may determine an encoding unit from among encoding units having different positions in the horizontal direction, and may set a limit on the encoding unit. When the current coding unit has a non-square shape having a height longer than a width, the image decoding apparatus 150 may determine the coding unit at a predetermined position along the vertical direction. In other words, the image decoding apparatus 150 may determine an encoding unit from among encoding units having different positions in the vertical direction, and set a limit on the encoding unit.

According to the present embodiment, the image decoding apparatus 150 may use information indicating the position of each of the even number of encoding units so as to determine an encoding unit at a predetermined position from among the even number of encoding units. The image decoding apparatus 150 may determine an even number of encoding units by dividing the current encoding unit and determine the encoding units at predetermined positions by using information on positions of the even number of encoding units. The detailed process thereof may correspond to the process of determining a coding unit at a predetermined position (e.g., a center position) from among the odd number of coding units, which has been described above with reference to fig. 5, and thus a description thereof will not be provided.

According to the present embodiment, when a current coding unit having a non-square shape is divided into a plurality of coding units, predetermined information about the coding unit at a predetermined position may be used during the dividing process in order to determine the coding unit at the predetermined position from among the plurality of coding units. For example, the image decoding apparatus 150 may use at least one of block shape information and division shape information stored in a sampling point included in a center encoding unit during a division process so as to determine an encoding unit located at the center from among a plurality of encoding units obtained by dividing a current encoding unit.

Referring to fig. 5, the image decoding apparatus 150 may divide the current encoding unit 500 into a plurality of encoding units 520a, 520b, and 520c based on at least one of block shape information and division shape information, and determine an encoding unit 520b located at the center from among the plurality of encoding units 520a, 520b, and 520 c. In addition, the image decoding apparatus 150 may determine the encoding unit 520b located at the center in consideration of a position at which at least one of the block shape information and the division shape information is obtained. That is, at least one of the block shape information and the division shape information of the current encoding unit 500 may be obtained from the sampling point 540 located at the center of the current encoding unit 500, and when the current encoding unit 500 is divided into the plurality of encoding units 520a, 520b, and 520c based on at least one of the block shape information and the division shape information, the encoding unit 520b including the sampling point 540 may be determined as the encoding unit located at the center. However, the information for determining the coding unit located at the center should not be interpreted limitedly as at least one of the block shape information and the division shape information, and various types of information may be used during the process of determining the coding unit located at the center.

According to the present embodiment, predetermined information for identifying the coding unit at a predetermined position can be obtained from predetermined sampling points included in the coding unit to be determined. Referring to fig. 5, the image decoding apparatus 150 may use at least one of block shape information and partition shape information obtained from a sampling point located at a predetermined position in the current coding unit 500 (e.g., a sampling point located at the center of the current coding unit 500) so as to determine a coding unit located at the predetermined position (e.g., a coding unit located at the center from among the plurality of coding units 520a, 520b, and 520c determined when the current coding unit 500 is partitioned). That is, the image decoding apparatus 150 may determine the sampling point at the predetermined position in consideration of the block shape of the current encoding unit 500, and the image decoding apparatus 150 may determine the encoding unit 520b from among the plurality of encoding units 520a, 520b, and 520c determined when the current encoding unit 500 is divided and set a predetermined limit to the encoding unit 520b, wherein the encoding unit 520b includes the sampling point from which the predetermined position (e.g., at least one of the block shape information and the division shape information) is obtained. Referring to fig. 5, the image decoding apparatus 150 may determine a sampling point 540 located at the center of the current encoding unit 500 as a sampling point to obtain predetermined information, and the image decoding apparatus 150 may set a predetermined position during the decoding process for the encoding unit 520b including the sampling point 540. However, the position of the sampling point at which the predetermined information is obtained should not be interpreted as the above-described position in a limited manner, and the sampling point may be interpreted as a sampling point included at an arbitrary position determined to be limited in the encoding unit 520.

According to the present embodiment, the position of the sampling point from which the predetermined position is obtained can be determined based on the shape of the current encoding unit 500. According to the present embodiment, it is possible to determine whether the shape of the current coding unit is square or non-square using the block shape information, and it is possible to determine the position of the sampling point that obtains the predetermined information based on the shape. For example, the image decoding apparatus 150 may determine a sampling point on a boundary dividing at least one of the width and the height of the current coding unit into two halves by using at least one of the information on the width of the current coding unit and the information on the height of the current coding unit as the sampling point to obtain the predetermined information. As another example, when the block shape information on the current coding unit indicates a non-square shape, the image decoding apparatus 150 may determine one of sampling points adjacent to a boundary dividing a longer side of the current coding unit into two halves as a sampling point from which predetermined information is obtained.

According to the present embodiment, when the current coding unit is divided into a plurality of coding units, the image decoding apparatus 150 may use at least one of block shape information and division shape information in order to determine a coding unit at a predetermined position from among the plurality of coding units. According to an embodiment, the image decoding apparatus 150 may obtain at least one of block shape information and division shape information from sampling points included at predetermined positions in the encoding units, and the image decoding apparatus 150 may divide a plurality of encoding units generated when the current encoding unit is divided by using at least one of the division shape information and the block shape information, wherein the at least one of the division shape information and the block shape information is obtained from sampling points included at predetermined positions in each of the plurality of encoding units. In other words, the encoding units may be recursively divided by using at least one of block shape information and division shape information obtained from sampling points at predetermined positions in each encoding unit. Since the recursive partitioning process of the coding unit has been described above with reference to fig. 4, details thereof are not provided.

According to the present embodiment, the image decoding apparatus 150 may determine at least one coding unit by dividing a current coding unit and determine an order in which the at least one coding unit is decoded according to a predetermined block (e.g., the current coding unit).

Fig. 6 illustrates an order of processing a plurality of coding units when the image decoding apparatus 150 determines the plurality of coding units by dividing the current coding unit according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may determine the second encoding units 610a and 610b by dividing the first encoding unit 600 in the vertical direction, determine the second encoding units 630a and 630b by dividing the first encoding unit 600 in the horizontal direction, or determine the second encoding units 650a, 650b, 650c, and 650d by dividing the first encoding unit 600 in the vertical and horizontal directions according to the block shape information and the division shape information.

Referring to fig. 6, the image decoding apparatus 150 may determine an order such that the second encoding units 610a and 610b determined by dividing the first encoding unit 600 in the vertical direction are to be processed in the horizontal direction 610 c. The image decoding apparatus 150 may determine the processing order of the second encoding units 630a and 630b determined by dividing the first encoding unit 600 in the horizontal direction as in the vertical direction 630 c. The image decoding apparatus 150 may determine the second encoding units 650a, 650b, 650c, and 650d determined by dividing the first encoding unit 600 in the vertical and horizontal directions according to a predetermined order (e.g., a raster scan order or a z scan order 650 e) in which the encoding units in one row are processed and then the encoding units in the next row are processed.

According to the present embodiment, the image decoding apparatus 150 can recursively divide the encoding unit. Referring to fig. 6, the image decoding apparatus 150 may determine a plurality of encoding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d by dividing the first encoding unit 600, and may recursively divide each of the determined plurality of encoding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650 d. The method of dividing the plurality of encoding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d may be similar to the method of dividing the first encoding unit 600. Accordingly, the plurality of encoding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d may each be independently divided into a plurality of encoding units. Referring to fig. 6, the image decoding apparatus 150 may determine the second encoding units 610a and 610b by dividing the first encoding unit 600 in the vertical direction, and in addition, may independently determine each of the second encoding units 610a and 610b to be divided or not to be divided.

According to the present embodiment, the image decoding apparatus 150 may divide the left second encoding unit 610a in the horizontal direction to obtain the third encoding units 620a and 620b, and may not divide the right second encoding unit 610b.

According to the present embodiment, the processing order of the coding units can be determined based on the segmentation process of the coding units. In other words, the processing order of the divided encoding units may be determined based on the processing order of the encoding units just before being divided. The image decoding apparatus 150 may determine the order in which the processing is performed for the third encoding units 620a and 620b determined when the left second encoding unit 610a is divided independently of the right second encoding unit 610b. Since the third encoding units 620a and 620b are determined when the left second encoding unit 610a is divided in the horizontal direction, the third encoding units 620a and 620b may be processed in the vertical direction 620 c. In addition, since the order in which the left and right second encoding units 610a and 610b are processed is in the horizontal direction 610c, the third encoding units 620a and 620b included in the left second encoding unit 610a may be processed in the vertical direction 620c, and then the right second encoding unit 610b may be processed. Since the above description is for describing a process of determining a processing order according to the encoding units before being divided, the process should not be construed as limited to the above-described embodiment, and various methods of independently processing the encoding units divided and determined in various shapes according to a predetermined order may be used.

Fig. 7 illustrates a process of determining, by the image decoding apparatus 150, that a current coding unit is divided into an odd number of coding units when the coding units cannot be processed in a predetermined order according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may determine that the current coding unit is divided into an odd number of coding units based on the obtained block shape information and division shape information. Referring to fig. 7, a first encoding unit 700 having a square shape may be divided into second encoding units 710a and 710b having a non-square shape, and the second encoding units 710a and 710b may be independently divided into third encoding units 720a, 720b, 720c, 720d, and 720e. According to the present embodiment, the image decoding apparatus 150 may determine the plurality of third encoding units 720a and 720b by dividing the left encoding unit 710a from among the second encoding units in the horizontal direction, and may divide the right encoding unit 710b into the odd-numbered third encoding units 720c, 720d and 720e.

According to the present embodiment, the image decoding apparatus 150 can determine whether there are encoding units divided into odd numbers by determining whether the third encoding units 720a, 720b, 720c, 720d, and 720e can be processed in a predetermined order. Referring to fig. 7, the image decoding apparatus 150 may determine the third encoding units 720a, 720b, 720c, 720d, and 720e by recursively dividing the first encoding unit 700. The image decoding apparatus 150 may determine whether or not an encoding unit divided into an odd number exists from among the first encoding unit 700, the second encoding units 710a and 710b, and the third encoding units 720a, 720b, 720c, 720d, and 720e based on at least one of the block shape information and the division shape information. For example, the coding unit located at the right portion from among the second coding units 710a and 710b may be divided into an odd number of third coding units 720c, 720d, and 720e. The order of processing the plurality of encoding units included in the first encoding unit 700 may be a predetermined order 730 (e.g., a z-scan order), and the image decoding apparatus 150 may determine whether the third encoding units 720c, 720d, and 720e determined when the right second encoding unit 710b is divided into the odd numbers satisfy a condition that can be processed according to the predetermined order.

According to the present embodiment, the image decoding apparatus 150 may determine whether the third encoding units 720a, 720b, 720c, 720d, and 720e included in the first encoding unit 700 satisfy a condition that can be processed according to a predetermined order, wherein the condition is related to whether at least one of the width and the height of the second encoding units 710a and 710b is divided into two halves along the boundary of the third encoding units 720a, 720b, 720c, 720d, and 720 e. For example, the third coding units 720a and 720b determined when the left second coding unit 710a having a non-square shape is divided into two halves satisfy the condition, but the third coding units 720c, 720d and 720e do not satisfy the condition because the boundaries of the third coding units 720c, 720d and 720e determined when the right second coding unit 710b is divided into three coding units cannot divide the width or height of the right second coding unit 710b into two halves. In addition, the image decoding apparatus 150 may determine that the scanning order is turned off when the condition is not satisfied, and determine that the right second encoding unit 710b is divided into an odd number of encoding units based on the determination result. According to the present embodiment, when the encoding units are divided into an odd number of encoding units, the image decoding apparatus 150 may set a predetermined restriction on the encoding units at a predetermined position from among the encoding units, and since details on the restriction or the predetermined position have been described above by various embodiments, details thereof are not provided.

Fig. 8 illustrates a process of determining at least one encoding unit by the image decoding apparatus 150 when the first encoding unit 800 is divided according to an embodiment. According to the present embodiment, the image decoding apparatus 150 may divide the first encoding unit 800 based on at least one of the block shape information and the division shape information obtained by the receiver 160. The first coding unit 800 having a square shape may be divided into four coding units having a square shape or a non-square shape. For example, referring to fig. 8, when the block shape information indicates that the first encoding unit 800 is square and the division shape information indicates that the first encoding unit 800 is divided into non-square encoding units, the image decoding apparatus 150 may divide the first encoding unit 800 into a plurality of non-square encoding units. In detail, when the division shape information indicates that the first encoding unit 800 is divided into the horizontal or vertical directions to determine an odd number of encoding units, the image decoding apparatus 150 may divide the first encoding unit 800 having a square shape into the following encoding units as the odd number of encoding units: the second coding units 810a, 810b, and 810c determined when the first coding unit 800 is divided in the vertical direction or the second coding units 820a, 820b, and 820c determined when the first coding unit 800 is divided in the horizontal direction.

According to the present embodiment, the image decoding apparatus 150 may determine whether the second encoding units 810a, 810b and 810c and 820a,820b and 820c included in the first encoding unit 800 satisfy a condition that can be processed according to a predetermined order, wherein the condition is related to whether at least one of the width and the height of the first encoding unit 800 is divided into two halves along the boundaries of the second encoding units 810a, 810b and 810c and 820a,820b and 820 c. Referring to fig. 8, since the boundaries of the second coding units 810a, 810b, and 810c determined when the first coding unit 800 having a square shape is divided in the vertical direction cannot divide the width of the first coding unit 800 in half, it may be determined that the first coding unit 800 does not satisfy the condition that can be processed according to the predetermined order. In addition, since the boundaries of the second coding units 820a,820b, and 820c determined when the first coding unit 800 having a square shape is divided in the horizontal direction cannot divide the width of the first coding unit 800 in half, it can be determined that the first coding unit 800 does not satisfy the condition that can be processed according to the predetermined order. When the condition is not satisfied, the image decoding apparatus 150 determines disconnection of the scan order and may determine that the first encoding unit 800 is divided into an odd number of encoding units based on the determination result. According to the present embodiment, when the encoding units are divided into an odd number of encoding units, the image decoding apparatus 150 may set a predetermined restriction on the encoding units at a predetermined position from among the encoding units, and since details on the restriction or the predetermined position have been described above by various embodiments, details thereof are not provided.

According to the present embodiment, the image decoding apparatus 150 can determine the encoding units having various shapes by dividing the first encoding unit.

Referring to fig. 8, the image decoding apparatus 150 may divide the first encoding unit 800 having a square shape and the first encoding unit 830 or 850 having a non-square shape into encoding units having various shapes.

Fig. 9 illustrates that when a second encoding unit having a non-square shape determined when the first encoding unit 900 is divided satisfies a predetermined condition, the shape of the divisible second encoding unit is limited by the image decoding apparatus 150 according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 may determine that the first encoding unit 900 having a square shape is to be divided into the second encoding units 910a, 910b, 920a, and 920b having non-square shapes based on at least one of the block shape information and the division shape information obtained by the receiver 160. The second encoding units 910a, 910b, 920a, and 920b may be independently divided. Accordingly, the image decoding apparatus 150 may determine whether to divide or not to divide the second encoding units 910a, 910b, 920a, and 920b based on at least one of the block shape information and the division shape information related to each of the second encoding units 910a, 910b, 920a, and 920b. According to the present embodiment, the image decoding apparatus 150 may determine the third encoding units 912a and 912b by dividing the left second encoding unit 910a having a non-square shape and determined when dividing the first encoding unit 900 in the vertical direction. However, when the left second encoding unit 910a is divided in the horizontal direction, the image decoding apparatus 150 may restrict the right second encoding unit 910b to not be divided in the horizontal direction as the direction in which the left second encoding unit 910a is divided. When the right second encoding unit 910b is divided in the same direction and the third encoding units 914a and 914b are determined, the third encoding units 912a, 912b, 914a and 914b may be determined when the left second encoding unit 910a and the right second encoding unit 910b are divided independently in the horizontal direction. However, this is the same result as the image decoding apparatus 150 divides the first encoding unit 900 into four second encoding units 930a, 930b, 930c, and 930d having square shapes based on at least one of the block shape information and the division shape information, and thus may be inefficient in image decoding.

According to the present embodiment, the image decoding apparatus 150 may determine the third encoding units 922a, 922b, 924a and 924b by dividing the second encoding unit 920a or 920b having a non-square shape and determined when dividing the first encoding unit 900 in the horizontal direction. However, when one of the second encoding units (e.g., the top second encoding unit 920 a) is divided in the vertical direction, the image decoding apparatus 150 may restrict the other second encoding unit (e.g., the bottom second encoding unit 920 b) from being divided in the vertical direction as the direction in which the top second encoding unit 920a is divided, based on the above-described reasons.

Fig. 10 illustrates a process of dividing an encoding unit having a square shape when division shape information does not indicate division of the encoding unit into four encoding units having a square shape by the image decoding apparatus 150 according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 can determine the second encoding units 1010a, 1010b, 1020a, 1020b, and the like by dividing the first encoding unit 1000 based on at least one of the block shape information and the division shape information. The division shape information may include information about various shapes into which the coding unit is divisible, but sometimes, the information about various shapes may not include information for dividing the coding unit into four square coding units. According to such division shape information, the image decoding apparatus 150 cannot divide the first encoding unit 1000 having a square shape into four square second encoding units 1030a, 1030b, 1030c, and 1030d. Based on the division shape information, the image decoding apparatus 150 may determine the second encoding units 1010a, 1010b, 1020a, 1020b, etc. having the non-square shape.

According to the present embodiment, the image decoding apparatus 150 can independently divide the second encoding units 1010a, 1010b, 1020a, 1020b, etc. having the non-square shape. Each of the second encoding units 1010a, 1010b, 1020a, 1020b, etc. may be divided in a predetermined order by a recursive method, which may correspond to a method of dividing the first encoding unit 1000 based on at least one of block shape information and division shape information.

For example, the image decoding apparatus 150 may determine the third encoding units 1012a and 1012b having square shapes by dividing the left second encoding unit 1010a in the horizontal direction, and may determine the third encoding units 1014a and 1014b having square shapes by dividing the right second encoding unit 1010b in the horizontal direction. In addition, the image decoding apparatus 150 may determine the third coding units 1016a, 1016b, 1016c, and 1016d having square shapes by dividing both the left second coding unit 1010a and the right second coding unit 1010b in the horizontal direction. In this case, the encoding units may be determined in the same manner that the first encoding unit 1000 is divided into four square second encoding units 1030a, 1030b, 1030c, and 1030 d.

As another example, the image decoding apparatus 150 may determine the third encoding units 1022a and 1022b having square shapes by dividing the top second encoding unit 1020a in the vertical direction, and determine the third encoding units 1024a and 1024b having square shapes by dividing the bottom second encoding unit 1020b in the vertical direction. In addition, the image decoding apparatus 150 may determine the third encoding units 1022a, 1022b, 1024a, and 1024b having square shapes by dividing both the top second encoding unit 1020a and the bottom second encoding unit 1020b in the vertical direction. In this case, the encoding units may be determined in the same manner that the first encoding unit 1000 is divided into four square second encoding units 1030a, 1030b, 1030c, and 1030 d.

According to the present embodiment, the image decoding apparatus 150 may divide the first encoding unit 1100 based on the block shape information and the division shape information. When the block shape information indicates a square shape and the division shape information indicates that the first encoding unit 1100 is divided in at least one of the horizontal direction and the vertical direction, the image decoding apparatus 150 may divide the first encoding unit 1100 to determine a second encoding unit (e.g., second encoding units 1110a, 1110b, 1120a, 1120b, 1130a, 1130b, 1130c, 1130d, etc.). Referring to fig. 11, the second encoding units 1110a, 1110b, 1120a, and 1120b having a non-square shape and determined when the first encoding unit 1100 is divided in only the horizontal or vertical direction may be separately divided based on block shape information and division shape information about each of the second encoding units 1110a, 1110b, 1120a, and 1120 b. For example, the image decoding apparatus 150 may determine the third encoding units 1116a, 1116b, 1116c, and 1116d by dividing the second encoding units 1110a and 1110b in the horizontal direction, wherein the second encoding units 1110a and 1110b are generated when the first encoding unit 1100 is divided in the vertical direction, and may determine the third encoding units 1126a, 1126b, 1126c, and 1126d by dividing the second encoding units 1120a and 1120b in the horizontal direction, wherein the second encoding units 1120a and 1120b are generated when the first encoding unit 1100 is divided in the horizontal direction. Since the dividing process of the second encoding units 1110a, 1110b, 1120a, and 1120b has been described with reference to fig. 9, details thereof are not provided.

According to the present embodiment, the image decoding apparatus 150 may process the encoding units according to a predetermined order. Since the characteristics regarding processing of the encoding units according to the predetermined order have been described above with reference to fig. 6, details thereof will not be provided. Referring to fig. 11, the image decoding apparatus 150 may determine four square third encoding units 1116a, 1116b, 1116c and 1116d or 1126a, 1126b, 1126c and 1126d by dividing the first encoding unit 1100 having a square shape. According to the present embodiment, the image decoding apparatus 150 may determine the processing order of the third encoding units 1116a, 1116b, 1116c and 1116d or 1126a, 1126b, 1126c and 1126d according to the shape of the divided first encoding unit 1100.

According to the present embodiment, the image decoding apparatus 150 may determine the third encoding units 1116a, 1116b, 1116c, and 1116d by dividing each of the second encoding units 1110a and 1110b in the horizontal direction, wherein the second encoding units 1110a and 1110b are generated when the first encoding unit 1100 is divided in the vertical direction, and the image decoding apparatus 150 may process the third encoding units 1116a, 1116b, 1116c, and 1116d according to the order 1117 in which the third encoding units 1116a and 1116b included in the left second encoding unit 1110a are first processed in the vertical direction and then the third encoding units 1116c and 1116d included in the right second encoding unit 1110b are processed in the vertical direction.

According to the present embodiment, the image decoding apparatus 150 may determine the second encoding units 1126a, 1126b, 1126c, and 1126d by dividing each of the second encoding units 1120a and 1120b in the vertical direction, wherein the second encoding units 1120a and 1120b are generated when the first encoding unit 1100 is divided in the horizontal direction, and the image decoding apparatus 150 may process the third encoding units 1126a, 1126b included in the top second encoding unit 1120a according to an order in which the third encoding units 1126a and 1126b included in the bottom second encoding unit 1120b are first processed in the horizontal direction and then the third encoding units 1126c and 1126d included in the bottom second encoding unit 1120b are processed in the horizontal direction.

Referring to fig. 11, the third encoding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d having square shapes may be determined while each of the second encoding units 1110a, 1110b, 1120a, and 1120b is divided. The second coding units 1110a and 1110b determined when the first coding unit 1100 is divided in the vertical direction and the second coding units 1120a and 1120b determined when the first coding unit 1100 is divided in the horizontal direction have different shapes, but the first coding unit 1100 is divided into coding units having the same shape according to the third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d determined thereafter. Therefore, even when the encoding units having the same shape are determined as a result of recursively dividing the encoding units via different processes based on at least one of the block shape information and the division shape information, the image decoding apparatus 150 can process the encoding units having the same shape in different orders.

According to the present embodiment, the image decoding apparatus 150 may determine the depth of the encoding unit according to a predetermined criterion. For example, the predetermined criterion may be the length of the longer side of the coding unit. When the length of the longer side of the encoding unit before being divided is 2n times (where n > 0) the length of the longer side of the current encoding unit, the image decoding apparatus 150 may determine that the depth of the current encoding unit is higher than the depth of the encoding unit before being divided by n. Hereinafter, the coding unit having a higher depth will be referred to as a coding unit having a lower depth.

Referring to fig. 12, according to the present embodiment, the image decoding apparatus 150 may determine the second encoding unit 1202 and the third encoding unit 1204 of the lower depth by dividing the first encoding unit 1200 having a square shape based on block shape information indicating a square shape (for example, the block shape information may indicate "0: square"). When the size of the first encoding unit 1200 having the square shape is 2n×2n, the second encoding unit 1202 determined by dividing the width and height of the first encoding unit 1200 by 1/2 may have the size of n×n. In addition, the third encoding unit 1204 determined by dividing the width and height of the second encoding unit 1202 by 1/2 may have a size of N/2×n/2. In this case, the width and height of the third encoding unit 1204 correspond to 1/2 times the width and height of the first encoding unit 1200. When the depth of the first encoding unit 1200 is D, the depth of the second encoding unit 1202, which is 1/2 times the width and height of the first encoding unit 1200, may be d+1, and the depth of the third encoding unit 1204, which is 1/2 times the width and height of the first encoding unit 1200, may be d+2.

According to the present embodiment, the image decoding apparatus 150 may determine the second encoding unit 1212 or 1222 and the third encoding unit 1214 or 1224 of the lower depth by dividing the first encoding unit 1210 or 1220 having the non-square shape based on block shape information indicating the non-square shape (for example, the block shape information may indicate "1:ns_ver" indicating that the height is longer than the width or indicate "2:ns_hor" indicating that the width is longer than the height).

The image decoding apparatus 150 may determine the second encoding unit (e.g., the second encoding units 1202, 1212, 1222, etc.) by dividing at least one of the width and the height of the first encoding unit 1210 having a size of n×2n. In other words, the image decoding apparatus 150 may determine the second encoding unit 1202 having the size n×n or the second encoding unit 1222 having the size n×n/2 by dividing the first encoding unit 1210 in the horizontal direction, or may determine the second encoding unit 1212 having the size N/2×n by dividing the first encoding unit 1210 in the horizontal and vertical directions.

According to the present embodiment, the image decoding apparatus 150 can determine the second encoding unit (e.g., the second encoding units 1202, 1212, 1222, etc.) by dividing at least one of the width and the height of the first encoding unit 1220 having a size of 2n×n. That is, the image decoding apparatus 150 may determine the second encoding unit 1202 having the size n×n or the second encoding unit 1212 having the size N/2×n by dividing the first encoding unit 1220 in the vertical direction, or may determine the second encoding unit 1222 having the size n×n/2 by dividing the first encoding unit 1220 in the horizontal and vertical directions.

According to the present embodiment, the image decoding apparatus 150 can determine the third encoding unit (e.g., the third encoding units 1204, 1214, 1224, etc.) by dividing at least one of the width and the height of the second encoding unit 1202 having the size n×n. That is, the image decoding apparatus 150 can determine the third encoding unit 1204 having the size N/2×n/2, the third encoding unit 1214 having the size N/2×n/2, or the third encoding unit 1224 having the size N/2×n/2 by dividing the second encoding unit 1202 in the vertical and horizontal directions.

According to the present embodiment, the image decoding apparatus 150 may determine the third encoding unit (e.g., the third encoding units 1204, 1214, 1224, etc.) by dividing at least one of the width and the height of the second encoding unit 1212 having a size of N/2×n. That is, the image decoding apparatus 150 may determine the third encoding unit 1204 having a size of N/2×n/2 or the third encoding unit 1224 having a size of N/2×n/2 by dividing the second encoding unit 1212 in the horizontal direction, or may determine the third encoding unit 1214 having a size of N/2×n/2 by dividing the second encoding unit 1212 in the vertical and horizontal directions.

According to the present embodiment, the image decoding apparatus 150 can determine the third encoding unit (e.g., the third encoding units 1204, 1214, 1224, etc.) by dividing at least one of the width and the height of the second encoding unit 1214 having the size n×n/2. That is, the image decoding apparatus 150 may determine the third encoding unit 1204 having a size of N/2×n/2 or the third encoding unit 1214 having a size of N/2×n/2 by dividing the second encoding unit 1212 in the vertical direction, or the third encoding unit 1224 having a size of N/2×n/2 by dividing the second encoding unit 1212 in the vertical and horizontal directions.

According to the present embodiment, the image decoding apparatus 150 may divide the encoding units (e.g., the first encoding units 1200, 1202, and 1204) having a square shape in the horizontal or vertical direction. For example, the first encoding unit 1200 having the size of 2n×2n may be divided in the vertical direction to determine the first encoding unit 1210 having the size of nx2n, or the first encoding unit 1200 having the size of 2n×2n may be divided in the horizontal direction to determine the first encoding unit 1220 having the size of 2n×n. According to the present embodiment, when determining the depth based on the length of the longest side of the coding unit, the depth of the coding unit determined when dividing the first coding unit 1200, 1202, or 1204 in the horizontal or vertical direction may be the same as the depth of the first coding unit 1200, 1202, or 1204.

According to the present embodiment, the width and height of the third encoding unit 1214 or 1224 may be 1/2 times that of the first encoding unit 1210 or 1220. When the depth of the first encoding unit 1210 or 1220 is D, the depth of the second encoding unit 1212 or 1214, which is 1/2 times the width and height of the first encoding unit 1210 or 1220, may be d+1, and the depth of the third encoding unit 1214 or 1224, which is 1/2 times the width and height of the first encoding unit 1210 or 1220, may be d+2.

Fig. 13 illustrates a depth determinable according to the shape and size of coding units and a Partial Index (PID) for distinguishing between coding units according to an embodiment.

According to the present embodiment, the image decoding apparatus 150 can determine the second encoding unit having various shapes by dividing the first encoding unit 1300 having a square shape. Referring to fig. 13, the image decoding apparatus 150 may determine the second encoding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d by dividing the first encoding unit 1300 in at least one of the vertical direction and the horizontal direction according to the division shape information. That is, the image decoding apparatus 150 may determine the second encoding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d based on the division shape information about the first encoding unit 1300.

According to the present embodiment, the depths of the second encoding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d determined according to the divided shape information on the first encoding unit 1300 having a square shape can be determined based on the lengths of the longer sides. For example, since the length of the longer sides of the second encoding units 1302a, 1302b, 1304a and 1304b having the non-square shape is the same as the length of one side of the first encoding unit 1300 having the square shape, the depth of the first encoding unit 1300 and the second encoding units 1302a, 1302b, 1304a and 1304b having the non-square shape may be D, that is, the same. On the other hand, when the image decoding apparatus 150 divides the first encoding unit 1300 into four second encoding units 1306a, 1306b, 1306c and 1306D having square shapes based on the division shape information, since the length of one side of each of the second encoding units 1306a, 1306b, 1306c and 1306D having square shapes is 1/2 of the length of one side of the first encoding unit 1300, the depth of the second encoding units 1306a, 1306b, 1306c and 1306D may be d+1, that is, one depth lower than the depth D of the first encoding unit 1300.

According to the present embodiment, the image decoding apparatus 150 may divide the first encoding unit 1310 having a height longer than a width into the plurality of second encoding units 1312a, 1312b, 1314a, 1314b and 1314c by dividing the first encoding unit 1310 in the horizontal direction according to the division shape information. According to the present embodiment, the image decoding apparatus 150 may divide the first encoding unit 1320 having a width longer than a height into a plurality of second encoding units 1322a and 1322b or 1324a, 1324b and 1324c by dividing the first encoding unit 1320 in the vertical direction according to the division shape information.

According to the present embodiment, depths of the second coding units 1312a, 1312b, 1314a, 1314b, 1316a, 1316b, 1316c, and 1316d determined according to the division shape information on the first coding unit 1310 or 1320 having a non-square shape may be determined based on the length of the longer side. For example, since the length of one side of each of the second coding units 1312a and 1312b having a square shape is 1/2 of the length of one side of the first coding unit 1310 having a non-square shape with a height longer than a width, the depth of the second coding units 1302a, 1302b, 1304a and 1304b having a square shape is d+1, i.e., one depth lower than the depth D of the first coding unit 1310 having a non-square shape.

In addition, the image decoding apparatus 150 may divide the first encoding unit 1310 having a non-square shape into an odd number of second encoding units 1314a, 1314b and 1314c based on the division shape information. The odd number of second encoding units 1314a, 1314b, and 1314c may include second encoding units 1314a and 1314c having a non-square shape and second encoding units 1314b having a square shape. Here, since the length of the longer sides of the second encoding units 1314a and 1314c having the non-square shape and the length of one side of the second encoding unit 1314b having the square shape are 1/2 of the length of one side of the first encoding unit 1310, the depths of the second encoding units 1314a, 1314b and 1314c may be d+1, i.e., one depth lower than the depth D of the first encoding unit 1310. The image decoding apparatus 150 may determine the depth of the coding unit related to the first coding unit 1310 having a non-square shape having a width longer than a height in a similar manner to determining the depth of the coding unit related to the first coding unit 1310.

According to the present embodiment, while determining PIDs for distinguishing between encoding units, when an odd number of encoding units do not have the same size, the image decoding apparatus 150 may determine PIDs based on a size ratio between encoding units. Referring to fig. 13, the encoding units 1314b located at the centers of the odd-numbered encoding units 1314a, 1314b and 1314c have the same width as the encoding units 1314a and 1314c, but have a height twice that of the encoding units 1314a and 1314 c. In this case, the encoding unit 1314b located at the center may include two in each of the encoding units 1314a and 1314 c. Therefore, when the PID of the encoding unit 1314b located at the center according to the scan order is 1, the PID of the encoding unit 1314c located in the next order may be increased by 2, i.e., 3. That is, the value of the PID may be discontinuous. According to the present embodiment, the image decoding apparatus 150 may determine whether or not the odd-numbered encoding units have the same size based on the discontinuity of the PID for distinguishing between the encoding units.

According to the present embodiment, the image decoding apparatus 150 may determine whether a plurality of encoding units determined when a current encoding unit is divided have certain division shapes based on the value of PID for distinguishing between encoding units. Referring to fig. 13, the image decoding apparatus 150 may determine even number of encoding units 1312a and 1312b or odd number of encoding units 1314a, 1314b and 1314c by dividing the first encoding unit 1310 having a rectangular shape with a height longer than a width. The image decoding apparatus 150 may use an ID indicating each coding unit in order to distinguish between a plurality of coding units. According to the present embodiment, the PID can be obtained from a sampling point (e.g., an upper left sampling point) at a predetermined position of each encoding unit.

According to the present embodiment, the image decoding apparatus 150 can determine the encoding unit at the predetermined position from among the encoding units determined via the division by using the PID for distinguishing between the encoding units. According to an embodiment, when the division shape information regarding the first encoding unit 1310 having a rectangular shape having a height longer than a width indicates division into three encoding units, the image decoding apparatus 150 may divide the first encoding unit 1310 into three encoding units 1314a, 1314b and 1314c. The image decoding apparatus 150 may assign PIDs to each of the three encoding units 1314a, 1314b, and 1314c. The image decoding apparatus 150 may compare PIDs of the encoding units to determine a center encoding unit from among the odd number of encoding units. The image decoding apparatus 150 may determine the encoding unit 1314b having the PID corresponding to the center value from among the PIDs as the encoding unit located at the center from among the encoding units determined when the first encoding unit 1310 is divided, based on the PIDs of the encoding units. According to the present embodiment, the image decoding apparatus 150 may determine the PID based on the size ratio between the encoding units while determining the PID for distinguishing between the encoding units, the encoding units not having the same size. Referring to fig. 13, the encoding unit 1314b generated when the first encoding unit 1310 is divided may have the same width as the encoding units 1314a and 1314c, but may have a height twice that of the encoding units 1314a and 1314c. In this case, when the PID of the encoding unit 1314b located at the center is 1, the PID of the encoding unit 1314c located in the next order may be increased by 2, i.e., 3. Accordingly, when the increase range is changed while the PID is uniformly increased, the image decoding apparatus 150 may determine that the encoding unit is divided into a plurality of encoding units including encoding units having a different size from other encoding units. According to the present embodiment, when the division shape information indicates division into an odd number of coding units, the image decoding apparatus 150 may divide the current coding unit into an odd number of coding units in which a coding unit (e.g., a center coding unit) at a predetermined position has a different size from other coding units. In this case, the image decoding apparatus 150 may determine center encoding units having different sizes by using PIDs of the encoding units. However, since the PID and the size or the position of the encoding unit at the predetermined position are designated to describe the present embodiment, and thus the present disclosure is not limited thereto, and various PIDs and various positions and sizes of the encoding unit may be used.

According to the present embodiment, the image decoding apparatus 150 can start recursively divided predetermined data units using the encoding unit.

Fig. 14 illustrates determining a plurality of coding units according to a plurality of predetermined data units included in a picture according to an embodiment.

According to the present embodiment, a predetermined data unit may be defined as a data unit that starts to be recursively divided by using at least one coding unit of block shape information and division shape information. That is, the predetermined data unit may correspond to a coding unit of the highest depth used while determining a plurality of coding units dividing the current picture. Hereinafter, for convenience of description, such a predetermined data unit is referred to as a reference data unit.

According to this embodiment, the reference data unit may indicate a predetermined size and shape. According to an embodiment, the reference encoding unit may include m×n sampling points. Here, M and N may be equal to each other, and may be integers expressed as multiples of 2. That is, the reference data unit may indicate a square shape or a non-square shape, and may be later divided into an integer number of coding units.

According to the present embodiment, the image decoding apparatus 150 may divide the current picture into a plurality of reference data units. According to the present embodiment, the image decoding apparatus 150 may divide a plurality of reference data units obtained by dividing the current picture by using the division information on each of the reference data units. Such a segmentation process of the reference data unit may correspond to a segmentation process using a quadtree structure.

According to the present embodiment, the image decoding apparatus 150 may previously determine the minimum size of the reference data unit available for inclusion in the current picture. Accordingly, the image decoding apparatus 150 may determine reference data units having various sizes equal to or larger than the minimum size, and determine at least one encoding unit based on the determined reference data units by using the block information and the division shape information.

Referring to fig. 14, the image decoding apparatus 150 may use a reference encoding unit 1400 having a square shape, or may use a reference encoding unit 1402 having a non-square shape. According to the present embodiment, the shape and size of the reference coding unit may be determined according to various data units (e.g., sequence, picture, slice segment, and maximum coding unit) that may include at least one reference coding unit.

According to the present embodiment, the receiver 160 of the image decoding apparatus 150 may obtain at least one of information on the shape of the reference coding unit and information on the size of the reference coding unit from the bitstream according to various data units. The process of determining at least one coding unit included in the reference coding unit 1400 having a square shape has been described above through the process of dividing the current coding unit 1000 of fig. 10, and the process of determining at least one coding unit included in the reference coding unit 1402 having a non-square shape has been described above through the process of dividing the current coding unit 1100 or 1150 of fig. 11, and thus a description thereof is not provided herein.

According to the present embodiment, in order to determine the size and shape of the reference coding unit according to some data units predetermined based on predetermined conditions, the image decoding apparatus 150 may use a PID for checking the size and shape of the reference coding unit. That is, the receiver 160 may obtain only PIDs for checking the size and shape of the reference coding unit from the bitstream according to the slice, slice segment, and maximum coding unit as data units (e.g., data units having equal to or less than the slice) satisfying a predetermined condition from among various data units (e.g., sequence, picture, slice segment, and maximum coding unit). The image decoding apparatus 150 may determine the size and shape of the reference data unit from the data units satisfying the predetermined condition by using the PID. When information on the shape of the reference coding unit and information on the size of the reference coding unit are obtained from the bitstream and used according to the data unit having a relatively small size, the use efficiency of the bitstream may be insufficient, and thus instead of directly obtaining the information on the shape of the reference coding unit and the information on the size of the reference coding unit, only the PID may be obtained and used. In this case, at least one of the size and the shape of the reference coding unit corresponding to the PID indicating the size and the shape of the reference coding unit may be predetermined. That is, the image decoding apparatus 150 may select at least one of the size and the shape of the reference coding unit determined in advance according to the PID so as to determine at least one of the size and the shape of the reference coding unit included in the data unit as a criterion for obtaining the PID.

According to the present embodiment, the image decoding apparatus 150 may use at least one reference encoding unit included in one maximum encoding unit. That is, the maximum coding unit of the divided image may include at least one reference coding unit, and the coding unit may be determined when each reference coding unit is recursively divided. According to the present embodiment, at least one of the width and the height of the maximum coding unit may be an integer multiple of at least one of the width and the height of the reference coding unit. According to the present embodiment, the size of the reference coding unit may be equal to the size of the largest coding unit divided n times according to the quadtree structure. That is, the image decoding apparatus 150 may determine the reference encoding unit by dividing the maximum encoding unit n times according to the quadtree structure, and divide the reference encoding unit based on at least one of the block shape information and the division shape information according to various embodiments.

Fig. 15 illustrates a processing block as a criterion when determining the determination order of the reference coding units included in the picture 1500 according to the embodiment.

According to the present embodiment, the image decoding apparatus 150 may determine at least one processing block of the divided picture. The processing block is a data unit including at least one reference coding unit of the divided image, and the at least one reference coding unit included in the processing block may be determined in a certain order. That is, the determination order of the at least one reference coding unit determined in each processing block may correspond to one of various orders for determining the reference coding units, and may vary according to the processing blocks. The determination order of the reference encoding units determined in accordance with the processing blocks may be one of various orders, such as a raster scan order, a Z-scan order, an N-scan order, an upper right diagonal scan order, a horizontal scan order, and a vertical scan order, but should not be construed restrictively by the scan order.

According to the present embodiment, the image decoding apparatus 150 can determine the size of at least one processing block included in an image by obtaining information on the size of the processing block. The image decoding apparatus 150 may obtain information on the size of the processing block from the bitstream to determine the size of at least one processing block included in the image. The size of the processing block may be a predetermined size of the data unit indicated by the information about the size of the processing block.

According to the present embodiment, the receiver 160 of the image decoding apparatus 150 may obtain information about the size of the processing block from the bitstream according to certain data units. For example, information about the size of the processing block may be obtained from the bit stream in the data units of images, sequences, pictures, slices, and slice segments. That is, the receiver 160 may obtain information on the size of a processing block from a bitstream according to such several data units, and the image decoding apparatus 150 may determine the size of at least one processing block of the divided picture by using the obtained information on the size of the processing block, wherein the size of the processing block may be an integer multiple of the size of the reference encoding unit.

According to the present embodiment, the image decoding apparatus 150 can determine the sizes of the processing blocks 1502 and 1512 included in the picture 1500. For example, the image decoding apparatus 150 may determine the size of the processing block based on information about the size of the processing block, which is obtained from the bitstream. Referring to fig. 15, according to an embodiment, the image decoding apparatus 150 may determine the horizontal sizes of the processing blocks 1502 and 1512 to be four times the horizontal size of the reference encoding unit, and the vertical size thereof to be four times the vertical size of the reference encoding unit. The image decoding apparatus 150 may determine a determination order of at least one reference coding unit in at least one processing block.

According to the present embodiment, the image decoding apparatus 150 may determine each of the processing blocks 1502 and 1512 included in the picture 1500 based on the size of the processing block, and may determine the determination order of at least one reference encoding unit included in each of the processing blocks 1502 and 1512. According to this embodiment, determining the reference coding unit may include determining a size of the reference coding unit.

According to the present embodiment, the image decoding apparatus 150 may obtain information on the determination order of at least one reference coding unit included in at least one processing block from the bitstream, and may determine the determination order of at least one reference coding unit based on the obtained information. The information on the determined order may be defined as an order or direction of the reference coding units in the determination processing block. That is, the order in which the reference coding units are determined may be independently determined in accordance with the processing block.

According to the present embodiment, the image decoding apparatus 150 may obtain information on the determined order of the reference encoding units from the bitstream according to certain data units. For example, the receiver 160 may obtain information about the determined order of the reference coding units from the bitstream according to data units such as images, sequences, pictures, slices, slice segments, and processing blocks. Since the information on the determination order of the reference coding units indicates the determination order of the reference coding units in the processing block, the information on the determination order can be obtained in a certain data unit including an integer number of processing blocks.

According to the present embodiment, the image decoding apparatus 150 may determine at least one reference encoding unit based on the determined order.

According to the present embodiment, the receiver 160 may obtain information on the determined order of the reference coding units from the bitstream as information on the processing blocks 1502 and 1512, and the image decoding apparatus 150 may determine the order of determining at least one reference coding unit included in the processing blocks 1502 and 1512 and determine at least one reference coding unit included in the picture 1500 according to the determined order of the coding units. Referring to fig. 15, the image decoding apparatus 150 may determine the determination orders 1504 and 1514 of at least one reference coding unit associated with the processing blocks 1502 and 1512, respectively. For example, when information on the determination order of the reference coding units is obtained in accordance with the processing blocks, the determination orders of the reference coding units related to the processing blocks 1502 and 1512 may be different from each other. When the determined order 1504 associated with the processing block 1502 is a raster scan order, reference encoding units included in the processing block 1502 may be determined based on the raster scan order. On the other hand, when the determination order 1514 related to the processing block 1512 is the reverse order of the changed raster scan order, the reference encoding units included in the processing block 1512 may be determined in the reverse order of the changed raster scan order. With reference to fig. 1 to 15, a method of dividing an image into maximum coding units and dividing each maximum coding unit into coding units having a hierarchical tree structure is described above. With reference to fig. 16 to 25, how to encode or decode the coding units of the same depth according to which coding order will now be described.

Fig. 16 illustrates a video decoding apparatus 1600 that involves determining the coding order of blocks according to an embodiment.

The video decoding apparatus 1600 includes an encoding order information acquirer 1610, an encoding order determiner 1620, and a decoder 1630. Referring to fig. 16, the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 are formed as separate elements, but in another embodiment, the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 may be integrated to be implemented as one element.

Referring to fig. 16, the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 are regarded as elements located within one device, but the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 are not required to be physically adjacent to each other. Thus, in another embodiment, the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 may be distributed.

The coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 may be implemented by one processor. In another embodiment, the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 may be implemented by a plurality of processors.

The functions performed by the coding order information acquirer 1610, the coding order determiner 1620, and the decoder 1630 may be performed by the image data decoder 180 of fig. 1 b.

The coding order information acquirer 1610 acquires coding order information regarding coding orders of neighboring blocks.

The coding order information indicates information about the coding order of the blocks. The coding order information may indicate whether to code the at least two blocks according to a default coding order. When the coding order of the blocks does not follow the default coding order, the coding order of the blocks is changed based on the coding order information according to the coding order information. On the other hand, when the coding order of the blocks follows the default coding order, the coding order of the blocks is changed based on the default coding order according to the coding order information.

The default coding order indicates a coding order applied to all blocks when no coding order information exists. The default coding order may be the coding order described with reference to fig. 15 or may be a changed coding order.

The coding sequence information may include a coding sequence flag having a 1-bit size indicating the coding sequence of the two blocks. For example, when the coding order flag indicates 0, the coding order of the two blocks is determined according to the default coding order. On the other hand, when the coding order flag indicates 1, the coding order of the two blocks is determined as the reverse order of the default coding order.

It is determined whether the coding order of two blocks from among the at least three blocks complies with a default coding order based on one coding order flag. Thus, the coding order of at least three blocks may be determined from a plurality of coding order flags.

However, in order to reduce the bits of the coding order information, the coding order information may include a 1-bit coding order flag indicating the coding order of at least three spatially neighboring blocks. For example, when the coding order information indicates 0, the coding order of at least three blocks is determined according to a default coding order. On the other hand, when the coding order information indicates 1, the coding order of at least three blocks is determined as the reverse order of the default coding order.

The coding order information acquirer 1610 may acquire coding order information from the bitstream. When the bitstream does not include coding order information, the coding order of the blocks may be determined according to a default coding order. When the coding order information acquirer 1610 determines the coding order internally according to the environment surrounding the current block, the coding order information acquirer 1610 does not acquire the coding order information from the bitstream.

The coding order information acquirer 1610 may check coding order change permission information for an upper data unit of the current block. The coding order change permission information indicates whether the coding order change is permitted for a block included in an upper data unit of the current block. When the coding order change permission information indicates that the coding order change is not permitted, all blocks of the upper data unit are decoded according to the default coding order. The coding order information acquirer 1610 may acquire coding order information when the coding order change permission information indicates that coding order information for the current block has been encoded.

In addition, the meaning of the value indicated by the obtained coding order flag can be determined based on the coding order change permission information. According to the coding order change permission information, it can be determined that the coding order flag is to be applied to two blocks, at least three blocks, or a plurality of blocks based on another predetermined scheme.

The coding order change permission information may be included in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a header of a maximum coding unit, and the like. When there are at least two types of coding order information, two pieces of coding order change permission information regarding the at least two types of coding order information may be stored separately in different headers.

The coding order determiner 1620 determines the coding order of the blocks based on the coding order information.

When the coding order flag is applied to two blocks, the coding order determiner 1620 may determine the coding order of the two blocks. For example, when the coding order flag indicates 0 for a first block and a second block having a coding order following the first block according to a default coding order, the coding order of the first block and the second block is determined according to the default coding order. However, when the coding order flag indicates 1, the coding order of the first block is exchanged with the coding order of the second block, and thus, the second block is determined to have a coding order before the coding order of the first block.

When the coding order flag is applied to at least three blocks, the coding order determiner 1620 may determine a direction of a coding order of the at least three blocks. For example, when the coding order of at least three blocks according to the default coding order indicates a first block, a second block, and then a third block, when the coding order flag indicates 0, the coding order of at least three blocks is determined as the first block, the second block, and then the third block with respect to the default coding order. On the other hand, when the coding order flag indicates 1, the coding order of at least three blocks is determined as a third block, a second block, and then a first block according to the opposite direction of the default coding order.

The encoding order determiner 1620 may internally determine encoding order information according to the environment surrounding the current block. For example, the coding order of the current block may be determined by referring to the coding order of the neighboring blocks applied to the reconstruction. In addition, the encoding order may be determined according to characteristics of the current block or the reconstructed neighboring block.

The decoder 1630 decodes the blocks according to the determined coding order. The methods described with reference to fig. 2 to 15 may be applied as a decoding method by the decoder 1630.

Fig. 17 illustrates a video encoding apparatus 1700 that involves determining the encoding order of blocks according to an embodiment.

The video encoding apparatus 1700 includes an encoding order determiner 1710, an encoder 1720, and an output unit 1730. Referring to fig. 17, the encoding order determiner 1710, the encoder 1720, and the output unit 1730 are formed as separate elements, but in another embodiment, the encoding order determiner 1710, the encoder 1720, and the output unit 1730 may be integrated to be implemented as one element.

Referring to fig. 17, the encoding order determiner 1710, the encoder 1720, and the output unit 1730 are regarded as elements located within one device, but the encoding order determiner 1710, the encoder 1720, and the output unit 1730 are not required to be physically adjacent to one another. Accordingly, in another embodiment, the encoding order determiner 1710, the encoder 1720, and the output unit 1730 may be scattered.

The encoding order determiner 1710, the encoder 1720, and the output unit 1730 may be implemented by one processor. In another embodiment, the encoding order determiner 1710, the encoder 1720, and the output unit 1730 may be implemented by a plurality of processors.

The functions performed by the coding order determiner 1710 and the encoder 1720 may be performed by the coding unit determiner 120 of fig. 1 a. In addition, the functions performed by the output unit 1730 of fig. 17 may be performed by the output unit 130 of fig. 1 a.

The coding order determiner 1710 determines whether to change the coding order of neighboring blocks. The coding efficiency of the block based on the default coding order is compared with the coding efficiency of the block based on the order changed from the default coding order, and then it may be determined whether to change the coding order. When the coding efficiency based on the default coding order is better, the coding order based on the default coding order is not changed. On the other hand, when the coding efficiency based on the order changed from the default coding order is better, the coding order based on the default coding order is changed.

The coding order determiner 1710 may compare coding efficiencies of predetermined coding orders. For example, it may be determined whether the coding order of the dual block unit is determined according to a default coding order. As another example, it may be determined whether the coding order of the three block units is determined according to a default coding order.

The encoding order determiner 1710 may internally determine encoding order information based on the environment surrounding the current block according to a predetermined criterion. When the coding order is determined internally based on the environment around the current block, the process of determining coding order candidates may be omitted. In another embodiment, the encoding order determiner 1710 may determine the encoding order by analyzing neighboring blocks of the current block and textures of the current block. Since the coding order is determined based on the similarity of textures, the coding efficiency of coding order candidates is not always calculated. Therefore, the calculation efficiency in the encoding process can be improved.

When the coding order change permission information of the current block indicates that the coding order change for the current block is permitted, the coding order determiner 1710 may determine whether to change the coding order. On the other hand, the coding order determiner 1710 may determine the coding order of all blocks according to the default coding order based on the coding order change permission information not permitting the coding order change.

The encoder 1720 encodes the block according to the encoding order determined by the encoding order determiner 1710.

The output unit 1730 outputs a bitstream including coding order information indicating whether the coding order of the block is changed from a default coding order. In addition, the output unit 1730 may output an encoding method for a block and encoding information about an encoding result.

The blocks referring to fig. 16 and 17 may indicate a maximum coding unit or a coding unit included in the maximum coding unit. Referring to fig. 16, blocks may be sequentially adjacent to each other according to a default coding order. Referring to fig. 16, the blocks may be horizontally or vertically adjacent to each other.

Fig. 18 is a diagram for describing the necessity of changing the coding order of blocks. The connection line 1850 and the connection line 1860 connect sampling points having similar characteristics. Accordingly, in decoding the block 1800, high encoding efficiency can be achieved by using encoding information corresponding to sampling points located at the connection lines 1850 and 1860. The connection line 1850 in the left embodiment of FIG. 18 passes through the left and upper boundaries of the block 1810. The connection line 1850 in the right embodiment of FIG. 18 passes through the left and right boundaries of the block 1810.

When reconstructing block 1800 according to the raster scan order, only the left, upper left, and upper right blocks of block 1800 have been reconstructed. Accordingly, only the coding information of the left, upper left, and upper right blocks of the block 1800 may be used in decoding the block 1800.

In the left embodiment of fig. 18, the connection line 1850 passes through the upper left block 1820 of the block 1800. Because the upper left block 1820 has been reconstructed, the block 1800 may be reconstructed based on the upper left block 1820. Thus, the block 1810 does not have to be reconstructed prior to the block 1800.

However, in the embodiment to the right of fig. 18, the connecting line 1860 does not pass through a block that has been reconstructed from among the neighboring blocks of block 1800. Accordingly, block 1800 cannot be efficiently reconstructed unless block 1810 was previously reconstructed. Therefore, it is necessary to change the coding order of the block 1800 and the block 1810.

Because the connecting line 1860 passes through the upper right block 1840 of the block 1810, the block 1810 may be efficiently reconstructed by using the encoding information of the upper right block 1840. After the block 1810 is reconstructed, the block 1800 may be reconstructed by using the encoding information of the block 1810. Thus, in the right-hand embodiment of fig. 18, it may be more efficient to reconstruct block 1810 before block 1800.

Fig. 19a and 19b illustrate an embodiment of a method of determining the coding order of a block. Fig. 19a illustrates an embodiment using a coding order flag indicating whether to change the coding order of two neighboring blocks. Fig. 19b illustrates an embodiment using a coding order flag indicating whether the direction of the coding order of at least three neighboring blocks is changed.

The coding order flag 1912 of fig. 19a indicates whether the coding order of the block 1902 and the block 1904 is changed. When the coding order flag 1912 indicates 0, the block 1902 has a coding order preceding the block 1904. On the other hand, when the encoding order flag 1912 indicates 1, the block 1904 has the encoding order before the block 1902. Likewise, the coding order flag 1914 determines the coding order of the blocks 1904 and 1906, and the coding order flag 1916 determines the coding order of the blocks 1906 and 1908.

According to another embodiment, when the coding order flag 1912 indicates 0, the coding order of blocks 1902 and 1904 is not changed and only block 1902 is reconstructed. Then, the coding order of the blocks 1904 and 1906 adjacent to the reconstructed block 1902 is determined from the coding order flag 1914.

When the encoding order flag 1912 indicates 1, the encoding order of the blocks 1902 and 1904 is changed, and the blocks 1904 and 1902 are sequentially reconstructed. Because the block 1904 has been reconstructed, the encoding order flag 1914 is not obtained, but the encoding order flag 1916 is obtained. Based on the coding order flag 1916, the coding order of the blocks 1906 and 1908 adjacent to the block 1904 is determined.

The foregoing embodiments can be applied to all blocks of one line regardless of the location and type of the blocks. Thus, until block 1910 is reconstructed, a determination of the coding order and reconstruction is performed.

The coding order flag 1940 of fig. 19b indicates the direction of the coding order to be applied to all blocks. When the encoding order flag 1940 indicates 0, the block 1922 located at the left portion is encoded according to the raster scan order. For example, after block 1922 is reconstructed, block 1924 may be reconstructed and after block 1924 is reconstructed, block 1926 may be reconstructed.

On the other hand, when the encoding order flag 1940 indicates 1, the block 1930 located at the right portion is encoded according to the reverse raster scan order.

Fig. 20 illustrates a method of comparing coding efficiency to determine whether to change the coding order of blocks. Fig. 20 illustrates two blocks 2000 and 2020 to be encoded according to a raster scan order, a sub-block 2010 included in the block 2000 and adjacent to a right boundary of the block 2000, and a sub-block 2030 included in the block 2020 and adjacent to a left boundary of the block 2020.

According to the raster scan order, block 2000 is encoded prior to block 2020. Accordingly, the sub-block 2010 included in the block 2000 is encoded before the sub-block 2030 included in the block 2020.

When only the coding order of the blocks 2000 and 2020 is changed without changing the coding order of the sub-blocks, the sub-block 2030 is coded without coding information on the left and right sub-blocks, thereby reducing coding efficiency of the sub-block 2030. On the other hand, by encoding the sub-block 2010 using all the encoding information on the left and right sub-blocks, the encoding efficiency of the sub-block 2010 can be improved.

Accordingly, it may be determined whether the coding order change of the blocks 2000 and 2020 is efficient by summing the improvement of the coding efficiency of the sub-block 2010 and the decrease of the coding efficiency of the sub-block 2030. When the improvement of the coding efficiency of the sub-block 2010 is greater than the decrease of the coding efficiency of the sub-block 2030, it is preferable to change the coding order of the block 2000 and the block 2020. However, on the other hand, when the decrease in coding efficiency of the sub-block 2030 is greater than the increase in coding efficiency of the sub-block 2010, it is preferable not to change the coding order of the blocks 2000 and 2020.

Fig. 21 illustrates reference sampling points to be used when predicting the block 2100 according to intra mode.

Referring to fig. 21, a first embodiment 2120 illustrates reference pixels 2102, 2106, 2108, and 2110 used in intra prediction when a block in an upper line and a left block are reconstructed. In the first embodiment 2120, the reference pixels 2102 and 2106 of the reconstructed upper block and the reference pixel 2108 of the reconstructed left block may be used in intra prediction. The reference pixel 2110 of the lower left block may be used only when the lower left block is reconstructed. To use the reference pixels 2102, 2106, 2108, and 2110, a prediction direction included in a first intra-prediction direction group 2125 may be used when intra-predicting the block 2100.

The second embodiment 2130 illustrates reference pixels 2102, 2104, 2112, and 2114 used in intra prediction when a block in an upper line and a right block are reconstructed. In the second embodiment 2130, the reference pixels 2102 and 2104 of the reconstructed upper block and the reference pixel 2112 of the reconstructed right block may be used in intra prediction. The reference pixel 2114 of the lower right block may be used only when the lower right block is reconstructed. In order to use the reference pixels 2102, 2104, 2112, and 2114, a prediction direction included in the second intra prediction direction group 2135 may be used when intra-predicting the current block 2100.

The third embodiment 2140 illustrates reference pixels 2102, 2108, and 2112 used in intra prediction when an upper block, a right block, and a left block are reconstructed. In the third embodiment 2140, the reference pixel 2102 of the upper block, the reference pixel 2108 of the left block, and the reference pixel 2112 of the right block may be used in intra prediction. The prediction directions included in the third intra prediction direction group 2145 may be used in intra prediction of the block 2100.

According to the first and second embodiments 2120 and 2130, when the reference pixel 2110 of the lower left block and the reference pixel 2114 of the lower right block cannot be used, the accuracy of prediction may be degraded. For example, in the upper right direction mode, the prediction according to the first embodiment 2120 may be inaccurate. However, in the third embodiment 2140, all of the reference pixels 2102, 2108, and 2112 used are adjacent to the block 2100, and thus, the accuracy of prediction may be relatively high compared to other embodiments.

The fourth embodiment 2150 illustrates reference pixels 2102, 2104, and 2106 used in intra prediction when only blocks in the upper line are reconstructed. In the fourth embodiment 2150, only the reference pixels 2102, 2104, and 2106 of the reconstructed upper block may be used in intra prediction. The prediction directions included in the fourth intra prediction direction group 2155 may be used in the intra prediction block 2100.

Unlike the third embodiment 2140, in the fourth embodiment 2150, the reference pixel 2102 of the upper block is the only pixel adjacent to the block 2100. Because the reference pixels 2104 and 2106 are spatially distant from the block 2100, the accuracy of the prediction may be degraded compared to the first, second, and third embodiments 2120, 2130, 2140. Accordingly, the intra prediction used in the fourth embodiment 2150 may be a vertical mode using the reference pixel 2102 of the upper block adjacent to the block 2100 or a direction prediction mode in a direction adjacent to the vertical mode.

Fig. 22a illustrates an intra prediction method in a planar mode. The plane mode indicates an intra prediction mode of a current pixel by performing double interpolation on four pixels from among pixels adjacent to the current block and at the same row or column of the current pixel based on the position of the current pixel. When the encoding order is not changed, the right block and the lower block of the current block are not reconstructed, and thus, instead of the lower pixel and the right pixel, the lower left pixel 2201 and the upper right pixel 2202 are used in order to predict the current pixel. However, when the encoding order is changed such that the right block of the current block is reconstructed, instead of the right upper pixel 2202, a pixel which is from among the right pixels 2203 and is located at the same line as the current pixel may be used as a reference pixel in the plane mode.

Fig. 22b illustrates an intra prediction method in the DC mode. In the DC mode, an average value of pixels adjacent to the current block becomes a predicted value of pixels included in the current block. When the encoding order is not changed, the right and lower blocks of the current block are not reconstructed, and thus, the left and upper pixels 2211 and 2212 are used in prediction according to the DC mode. However, when the encoding order is changed such that the right block of the current block is reconstructed, the right pixel 2213 may be additionally used in the DC mode. Accordingly, the average value of the pixel values of the left pixel 2211, the upper pixel 2212, and the right pixel 2213 may be used as a prediction value of the pixel included in the current block.

Fig. 22c illustrates an intra prediction method in the horizontal mode. In the horizontal mode, pixels located in the horizontal direction of the current pixel are used in prediction of the current block. When the coding order is not changed, the right block of the current block is not reconstructed, and thus, only the left pixel 2221 is used in prediction according to the horizontal mode. However, when the encoding order is changed such that the right block of the current block is reconstructed, the right pixel 2222 may be additionally used in the horizontal mode. For example, the predicted value of the current pixel may be determined as an average value of pixels from among the left pixel 2221 and the right pixel 2222 and located in the horizontal direction of the current pixel. Alternatively, the predicted value of the current pixel may be determined by interpolating a pixel from among the left pixel 2221 and the right pixel 2222 and located in the horizontal direction of the current pixel according to the position of the current pixel.

Fig. 22d illustrates an intra prediction method in the upper right direction mode. In the upper right direction mode, a pixel located in the upper right direction of the current pixel is used in prediction of the current pixel. When the coding order is not changed, the right block of the current block is not reconstructed, and thus, the upper right pixel 2232 is used in the prediction according to the upper right direction mode instead of the upper right pixel 2233. However, when the encoding order is changed such that the right block of the current block is reconstructed, the right pixel 2233 may be used in the upper right direction mode. Because the right pixel 2233 is closer to the current block than the right upper pixel 2232, the current block can be predicted further accurately.

Fig. 22e illustrates an intra prediction method in the lower left direction mode. In the lower left direction mode, a pixel located in the lower left direction of the current pixel is used in prediction of the current pixel. When the encoding order is not changed, the right block of the current block is not reconstructed, and thus, only the left pixel 2241 and the left lower pixel 2242 are used in prediction according to the lower left direction mode. However, when the encoding order is changed such that the right block of the current block is reconstructed, the right pixel 2243 may be used in the lower left direction mode. For example, an average value of a pixel and another pixel, which is located in the lower left direction of the current pixel and is from among the left pixel 2241 and the lower left pixel 2242, and which is located in the opposite direction of the lower left direction and is from among the right pixel 2243, may be determined as the predicted value of the current pixel. Alternatively, the predicted value of the current pixel may be determined by interpolating a pixel and another pixel based on the position of the current pixel, wherein the pixel is located in the lower left direction of the current pixel and is from among the lower left pixel 2241 and the lower left pixel 2242, and the other pixel is located in the opposite direction of the lower left direction and is from among the right pixel 2243.

In the intra mode not described with reference to fig. 22a to 22e, a method of using the right pixel of the current block may be used to improve the accuracy of intra prediction.

Fig. 23 illustrates reference blocks to be used when predicting the block 2300 according to the inter mode.

Only when the left block of block 2300 of fig. 23 is reconstructed, some of the motion vectors for the blocks including reference pixels 2302, 2304, 2306, 2308 and 2310 are determined as motion vector candidates of the first candidate list. One motion vector may be selected from among the motion vector candidates of the first candidate list, and pieces of coding information such as reference picture indexes and the like required for inter prediction may be obtained from a block including the selected motion vector.

When only the right block of block 2300 is reconstructed, some of the blocks including reference pixels 2302, 2310, 2312, 2316 and 2318 may be determined as motion vector candidates of the second candidate list. Alternatively, some of the motion vectors for the blocks including the reference pixels 2302, 2308, 2310, 2314 and 2318 may be determined as motion vector candidates of the second candidate list. One motion vector may be selected from among the motion vector candidates of the second candidate list, and pieces of coding information such as reference picture indexes and the like required for inter prediction may be obtained from a block including the selected motion vector.

When the left and right blocks of block 2300 are all reconstructed, an efficient candidate list may be selected from the second candidate list and the first candidate list. Thereafter, a motion vector may be determined from the selected candidate list. In another embodiment, when the left and right blocks of the current block 2300 are all reconstructed, a third candidate list different from the first and second candidate lists may be generated. For example, motion vectors for blocks including reference pixels 2302, 2304, 2310, 2312 and 2316 may be included as motion vector candidates in a third candidate list.

When the left and right blocks of the block 2300 are not reconstructed, the encoded information cannot be obtained from the left and right blocks. Thus, a fourth candidate list including motion vectors for blocks located at the upper line of the block 2300 as motion vector candidates may be used. For example, motion vectors for blocks including reference pixels 2302, 2308, 2310 and 2312 may be included as motion vector candidates in a fourth candidate list.

In the Z coding order, inter prediction according to the first candidate list may be used. However, when the coding order of two horizontally adjacent blocks is changed, the right block may be inter-predicted first according to the second candidate list or the fourth candidate list. After the right block is reconstructed, the left block may be reconstructed by inter prediction from one of the second candidate list and the third candidate list.

Fig. 24 illustrates a video decoding method performed by the video decoding apparatus 1600 according to an embodiment.

In operation 2410, coding order information indicating whether the coding order of the neighboring blocks is changed is obtained.

In operation 2420, the coding order of the block is determined based on the coding order information.

In operation 2430, the block is decoded according to the determined coding order.

According to the present embodiment, the encoding order information may indicate the encoding order of the first block and the second block adjacent to each other. When the encoding order information indicates that there is no change in the encoding order of the first block and the second block, the first block is first decoded. Then, coding order information about the second block and a third block adjacent to the second block may be obtained.

On the other hand, when the encoding order information indicates a change in the encoding order of the first block and the second block, the second block is first decoded. After the second block is decoded, the first block is decoded. After the first block is decoded, encoding order information on the third block and the fourth block may be obtained.

According to the present embodiment, the coding order information may indicate a direction of the coding order. When the encoding order information indicates that the direction of the encoding order is equal to the default encoding order, the block corresponding to the encoding order information is decoded in the same direction as the default encoding order. On the other hand, when the encoding order information indicates that the direction of the encoding order is opposite to the default encoding order, the block corresponding to the encoding order information is decoded in the direction opposite to the default encoding order.

The coding order information may be implemented as a 1-bit-size coding order flag. Thus, the coding order information may indicate whether the coding order is equal to a default coding order.

According to the present embodiment, the video decoding method may further include obtaining coding order change permission information indicating whether coding order change is permitted for a block included in an upper data unit including the current block. Therefore, only when the coding order change permission information indicates that the coding order change is permitted for the current block, the coding order information on the current block can be obtained.

Fig. 25 illustrates a video encoding method performed by the video encoding apparatus 1700 according to an embodiment.

In operation 2510, it is determined whether the coding order of the neighboring blocks is changed.

In operation 2520, the blocks are encoded according to whether the encoding order is changed.

In operation 2530, a bitstream including encoding order information indicating whether the encoding order is changed and encoding information of a block is output.

The blocks of fig. 24 and 25 may be adjacent to each other in the horizontal direction or the vertical direction. In addition, the blocks of fig. 24 and 25 may each be a maximum coding unit or a data unit included in the maximum coding unit.

According to the video encoding technique based on the encoding unit having the tree structure described with reference to fig. 1 to 25, image data of a spatial domain is encoded in each encoding unit having the tree structure, and decoding is performed on each maximum encoding unit according to the video decoding technique based on the encoding unit having the tree structure, so that the image data of the spatial domain is reconstructed, and by doing so, a picture and a video which is a picture sequence can be reconstructed. The reconstructed video may be reproduced by a reproducing apparatus, may be stored in a storage medium, or may be transmitted through a network.

Embodiments according to the present disclosure may be written as computer programs and may be implemented in general-use digital computers that execute the programs using a computer readable recording medium.

Although the preferred embodiment of the present disclosure has been described, those of ordinary skill in the art will appreciate that various substitutions, modifications, or changes may be made therein without departing from the spirit and scope as defined by the following claims. That is, the claims are to be interpreted to include various alternatives, modifications, or variations for the present disclosure. The descriptions provided in the present specification and drawings, therefore, should be considered in an illustrative sense only and not for the purpose of limitation.

Claims

1. A video decoding method, the video decoding method comprising:

Obtaining coding order change permission information for an upper data unit including a first block and a second block, wherein the first block is located at the left side of the second block, the coding order change permission information indicating whether to permit a change in coding order of blocks included in the upper data unit;

Determining an encoding order of the first block and the second block based on a first encoding order when the encoding order change permission information indicates that the encoding order of the blocks is not permitted to change; and decoding the first block and the second block according to the first coding order,

When the coding order change permission information indicates that a change in coding order of the blocks is permitted, coding order information indicating whether the first block is decoded before the second block is obtained, and coding orders of the first block and the second block are determined based on the coding order information, the first block and the second block being adjacent to each other,

Determining a candidate list for a sub-block of the first block when the second block is decoded before the first block according to the determined coding order, using a motion vector of a neighboring block of the sub-block, the sub-block being one of the sub-blocks adjacent to a boundary between the first block and the second block, and the neighboring block of the sub-block including one or more sub-blocks in the second block, and performing inter prediction on the sub-block using the motion vector of the neighboring block determined from the candidate list,

Wherein the coding order change permission information is obtained from a sequence parameter set, and

The first coding order is the same as the coding order of the blocks included in the upper data unit, and the first block and the second block are determined by dividing the blocks.

2. A video encoding method, the video encoding method comprising:

Determining whether a change in the coding order of the blocks included in the upper data unit is allowed;

When the coding order of the blocks is not allowed to be changed, coding a first block and a second block included in the upper data unit according to a first coding order, the first block being located at the left side of the second block;

Determining whether the first block is decoded before the second block when a change in the coding order of the blocks is allowed, the first block and the second block being adjacent to each other;

Determining, when the second block is decoded before the first block, a candidate list for a sub-block of the first block using a motion vector of a neighboring block of the sub-block, the sub-block being one of the sub-blocks adjacent to a boundary between the first block and the second block, and the neighboring block of the sub-block including one or more sub-blocks in the second block, and performing inter prediction on the sub-block using the motion vector of the neighboring block determined from the candidate list; and

Outputting a bitstream including coding order change permission information indicating whether a change in coding order of blocks included in the upper data unit is permitted, coding order information indicating whether the first block is coded before the second block, and coding information of the first block and the second block,

Wherein,

The coding order change permission information is included in a sequence parameter set, and

The first coding order is the same as the coding order of the blocks included in the upper data unit, and the first block and the second block are generated by dividing the blocks.