KR20130036753A - Method and apparatus for video encoding, and method and apparatus for video decoding - Google Patents

Abstract

PURPOSE: A video encoding method, a device thereof, a video decoding method, and a device thereof are provided to increase the maximal size of an encoding unit by considering an image size and to control the encoding unit by considering an image property, thereby increasing image compression efficiency. CONSTITUTION: A maximal encoding unit division unit(110) divides a current picture based on a maximal encoding unit. An encoding depth determination unit(120) encodes one or more division areas per depth and determines the depth which the final encoding result is outputted corresponding to one or more division areas. An output unit(130) outputs information for image data of the maximal encoding unit and an encoding mode corresponding to the depth as a bitstream type. [Reference numerals] (110) Maximal encoding unit division unit; (120) Encoding depth determination unit; (130) Output unit;

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video encoding method and apparatus,

The present invention relates to encoding and decoding of video.

Background of the Invention [0002] As the development and dissemination of hardware capable of playing back and storing high-resolution or high-definition video content increases the need for video codecs to effectively encode or decode high-definition or high-definition video content. According to the conventional video codec, video is encoded according to a limited encoding method based on a macroblock of a predetermined size.

The present invention relates to image encoding and image decoding based on hierarchical encoding units of various sizes.

According to an aspect of the present invention, there is provided a video encoding method including: dividing a current picture into at least one maximum encoding unit that is a maximum-size encoding unit; Encoding the at least one divided area in which the area of the maximum encoding unit is divided for each depth, based on a depth that increases as the number of times of dividing the area of the maximum encoding unit increases, Determining a depth at which the result is to be output; And outputting the image data as the final encoding result for each of the at least one divided area and the encoding information for the depth and the prediction mode for each maximum encoding unit. .

The coding unit according to an embodiment may be characterized by a maximum size and depth.

Depth indicates a stage in which coding units are hierarchically divided. As the depth increases, the depth coding units can be divided from the maximum coding unit to the minimum coding unit. In the present specification, it is defined that the depth is deepened from a high depth or a high depth toward a low depth or a bottom depth. As the depth increases, the number of division of the maximum encoding unit increases and the total number of divisions of the maximum encoding unit corresponds to 'maximum depth'. The maximum size and the maximum depth of the encoding unit may be preset.

The video encoding method according to an exemplary embodiment may further include setting a maximum size and a maximum depth of the encoding size in advance.

According to an exemplary embodiment of the present invention, the image encoding method may include encoding the image data on the basis of at least one depth-dependent encoding unit for each of the at least one maximum encoding unit, The encoding unit can be hierarchically divided.

The coding depth determination step of the video coding method according to an exemplary embodiment of the present invention may further include determining a coding depth for each of the maximum coding units for each of the depth coding units, It is possible to encode the corresponding image data by performing predictive encoding for each partial data unit obtained by dividing at least one of the height and the width of the depth encoding unit.

The coding depth determination step of the video coding method according to an exemplary embodiment of the present invention may further include determining a coding depth for each of the maximum coding units for each of the coding units, It is possible to encode the corresponding image data by performing frequency conversion in a unit of data smaller than or equal to the frequency.

The step of determining a coding depth of the video coding method according to an exemplary embodiment of the present invention includes a step of comparing a coding result of each depth coding unit for each of the coding units to generate a minimum coding error, A partition type and a prediction mode, and a size of a data unit in which frequency conversion is performed.

The prediction mode for each maximum encoding unit is independently determined for each depth encoding unit, and may include at least one of an intra mode, an inter mode, and a skip mode.

Wherein the information on the encoding mode determined for each maximum encoding unit includes at least one of information on a partition type of a data unit for which prediction encoding has been performed for each encoding unit of the determined encoding depth and information on the partial data unit prediction mode, And information on the size of the data unit in which the frequency conversion of the depth encoding unit is performed.

The video coding method according to an exemplary embodiment may further include coding division information indicating whether or not coding is performed in units of sub-depth coding, instead of coding the current coding depth at the maximum coding unit for each maximum coding unit. If encoding is performed for the lower depth based on the division information of the current depth, the encoding may be repeatedly performed for each partial data unit obtained by dividing the encoding unit of the current depth.

The depth of the maximum encoding unit is the highest depth, and the depth encoding unit may be an encoding unit having a height and a width half the height of one higher depth encoding unit. The partial data unit of the current depth may be a data unit of at least one of a height and a width of the data unit of the current depth.

The coding error may be measured using a rate-distortion optimization technique based on a Lagrangian multiplier.

According to one embodiment, the intra-mode predictive encoding and the inter-mode predictive encoding may be performed for each data unit that is obtained by halving at least one of the current-depth encoding unit and the current-depth encoding unit height and width have.

According to an embodiment of the present invention, there is provided a video decoding method including: receiving and parsing a bitstream of encoded video; Extracting, from the bitstream, image data of the current picture allocated to a maximum coding unit, which is a coding unit of a maximum size, and information on coding depth and coding mode of each coding unit; And decoding the image data of each maximum encoding unit based on the information on the encoding depth and the encoding mode of the maximum encoding unit, wherein the depth of the image is determined as the number of times of dividing the area of the maximum encoding unit increases Wherein the coding depth for the maximum coding unit is obtained by coding at least one divided area in which the area of the maximum coding unit is divided by the maximum coding unit in the coding process of the current picture, The final encoding result for each region is determined by the output depth.

The video decoding method may further include extracting information on a maximum depth indicating a total number of times the height and width of the maximum encoding unit can be hierarchically divided from the bit stream.

The decoding step of the video decoding method according to an exemplary embodiment of the present invention may further include a decoding step of decoding, based on a coding depth of information on the extracted coding mode per coding unit, a partition type of a data unit to be predictively coded, And performing predictive encoding on the encoding unit of the encoding depth for each unit.

The decoding step of the video decoding method according to an exemplary embodiment of the present invention may further include decoding the encoded data in units of encoding units of the encoding depth for each of the maximum encoding units based on the extracted encoding depth for the maximum encoding unit and the size of the data unit for performing the inverse frequency conversion And performing frequency inverse transform on the frequency domain.

The video decoding method according to an embodiment may further include extracting division information indicating whether to decode from the bitstream into low-depth coding units instead of decoding the current coding units by depth at the maximum coding unit .

A video encoding apparatus according to an embodiment of the present invention includes: a maximum encoding unit division unit that divides a current picture into at least one maximum encoding unit, which is a maximum-size encoding unit; Encoding the at least one divided area in which the area of the maximum encoding unit is divided for each depth, based on a depth that increases as the number of times of dividing the area of the maximum encoding unit increases, An encoding depth determination unit for determining a depth at which a result is output; And an output unit for outputting image data as a final encoding result for each of the at least one divided area and encoding information about a depth and a prediction mode for each of the maximum encoding units.

In accordance with another aspect of the present invention, a video decoding apparatus includes: a receiver configured to receive and parse a bitstream of an encoded video; An image data and an encoding information extracting unit for extracting image data of the current picture allocated to a maximum encoding unit, which is the encoding unit of the maximum size, from the bitstream, and information on the encoding depth and encoding mode of the maximum encoding unit; And a decoding unit that decodes image data of each maximum encoding unit based on information on the encoding depth and encoding mode for each of the maximum encoding units, wherein the encoding depth for each of the maximum encoding units includes: And at least one divided region in which the region of the maximum encoding unit is divided is encoded for each depth, by the maximum encoding unit, and is determined as a depth at which the final encoding result is output for each of the at least one divided region.

The present invention includes a computer-readable recording medium on which a program for implementing a video encoding method according to an embodiment is recorded. In addition, the present invention includes a computer-readable recording medium on which a program for implementing a video decoding method according to an embodiment is recorded.

1 shows a block diagram of a video encoding apparatus according to an embodiment of the present invention.
2 shows a block diagram of a video decoding apparatus according to an embodiment of the present invention.
3 illustrates a concept of coding units, according to an embodiment of the present invention.
4 is a block diagram of an image encoding unit based on an encoding unit according to an embodiment of the present invention.
5 is a block diagram of an image decoder based on coding units, according to an embodiment of the present invention.
FIG. 6 illustrates a depth-based coding unit and a prediction unit according to an embodiment of the present invention.
FIG. 7 shows a relationship between an encoding unit and a conversion unit according to an embodiment of the present invention.
FIG. 8 illustrates depth-specific encoding information, in accordance with an embodiment of the present invention.
FIG. 9 shows a depth encoding unit according to an embodiment of the present invention.
FIGS. 10A, 10B, and 10C illustrate the relationship between an encoding unit, a prediction unit, and a frequency conversion unit according to an embodiment of the present invention.
11 is a diagram of encoding information according to coding units, according to an embodiment of the present invention.
12 shows a flowchart of a video coding method according to an embodiment of the present invention.
13 shows a flowchart of a video decoding method according to an embodiment of the present invention.

Hereinafter, a video encoding apparatus, a video encoding apparatus, a video encoding method, and a video decoding method according to an embodiment of the present invention will be described with reference to FIGS. 1 to 13.

1 shows a block diagram of a video encoding apparatus according to an embodiment of the present invention.

The video coding apparatus 100 according to an embodiment includes a maximum coding unit division unit 110, a coding depth determination unit 120, and an output unit 130.

The maximum coding unit division unit 110 may divide a current picture based on a maximum coding unit which is a coding unit of a maximum size for a current picture of an image. If the current picture is larger than the maximum coding unit, image data of the current picture may be split into at least one maximum coding unit. The image data may be output to the coding depth determination unit 120 for each of at least one maximum coding unit.

The coding unit according to an embodiment may be characterized by a maximum size and depth. Depth indicates a stage in which coding units are hierarchically divided. As the depth increases, the depth coding units can be divided from the maximum coding unit to the minimum coding unit. The depth of the largest coding unit is the highest depth and the minimum coding unit may be defined as the lowest coding unit. As the maximum coding unit decreases as the depth increases, the size of the coding unit for each depth decreases, and thus, the coding unit of the higher depth may include coding units of a plurality of lower depths.

As described above, the image data of the current picture may be divided into maximum coding units according to the maximum size of the coding unit, and each maximum coding unit may include coding units divided by depths. Since the maximum coding unit is divided according to depths, image data of a spatial domain included in the maximum coding unit may be hierarchically classified according to depths.

The maximum depth and the maximum size of the coding unit that limit the total number of times of hierarchically dividing the height and the width of the maximum coding unit may be preset.

The coding depth determiner 120 encodes at least one divided area in which the area of the maximum coding unit is divided for each depth, and determines the depth at which the final coding result is output for each of at least one of the divided areas. That is, the coding depth determination unit 120 selects the depth at which the smallest coding error occurs, and determines the coding depth as the coding depth by coding the image data in units of depth coding for each maximum coding unit of the current picture. The determined coded depth and the image data for each maximum coding unit are output to the outputter 130.

Image data in the largest coding unit is encoded based on coding units according to depths according to at least one depth less than or equal to the maximum depth, and encoding results based on the coding units for each depth are compared. As a result of comparing the encoding error of the coding units according to depths, a depth having the smallest encoding error may be selected. At least one coding depth may be determined for each maximum coding unit.

As the depth of the maximum coding unit increases, the coding unit is divided into hierarchically and the number of coding units increases. In addition, even in the case of coding units having the same depth included in one largest coding unit, a coding error of each data is measured, and whether or not division into a lower depth is determined. Therefore, even in the data included in one largest coding unit, since the encoding error for each depth is different according to the position, the coding depth may be differently determined according to the position. Accordingly, one or more coding depths may be set for one maximum coding unit, and data of the maximum coding unit may be partitioned according to coding units of one or more coding depths.

The predictive encoding and frequency conversion of the maximum encoding unit can be performed. Likewise, predictive coding and frequency conversion are performed on the basis of the depth coding unit for each maximum coding unit and for each depth below the maximum depth.

Since the number of coding units per depth is increased every time the maximum coding unit is divided by the depth, the coding including the predictive coding and the frequency conversion should be performed for every depth coding unit as the depth increases. For convenience of explanation, predictive coding and frequency conversion will be described based on a coding unit of a current depth among at least one maximum coding unit.

The video encoding apparatus 100 according to an exemplary embodiment may select various sizes or types of data units for encoding image data. The encoding of the image data is performed through prediction encoding, frequency conversion, entropy encoding, and the like. The same data unit may be used in every step, or the data unit may be changed in steps.

For example, the video coding apparatus 100 can select a coding unit and a data unit different from the coding unit in order to perform predictive coding of the video data of the coding unit as well as the coding unit for coding the video data.

For predictive coding of the maximum coding unit, predictive coding may be performed based on the partial data unit of the coding unit for each depth of the maximum coding unit. The partial data unit of the encoding unit may include a data unit in which at least one of the height and the width of the encoding unit and the depth encoding unit is divided.

For example, when the size of the encoding unit is 2Nx2N (where N is a positive integer), the size of the partial data unit may be 2Nx2N, 2NxN, Nx2N, NxN, and the like. Prediction coding may be performed based on data units of various types, in addition to data units of a type in which at least one of the height and the width of an encoding unit is divided by half. Hereinafter, a data unit on which prediction encoding is based may be referred to as a 'prediction unit'.

The prediction mode of the encoding unit may be at least one of an intra mode, an inter mode, and a skip mode. For example, the intra mode and the inter mode can be performed for prediction units of 2Nx2N, 2NxN, Nx2N, and NxN sizes. In addition, the skip mode can be performed only for a prediction unit of 2Nx2N size. The encoding may be performed independently for each prediction unit within the coding unit to select a prediction mode having the smallest encoding error.

In addition, the video encoding apparatus 100 according to an exemplary embodiment may perform frequency conversion of image data of an encoding unit based on not only an encoding unit for encoding image data but also a data unit different from the encoding unit.

For frequency conversion of a coding unit, frequency conversion may be performed based on a data unit having a size smaller than or equal to the coding unit. For example, a data unit for frequency conversion may include a data unit for intra mode and a data unit for inter mode. Hereinafter, the data unit on which the frequency conversion is based may be referred to as a 'conversion unit'.

The encoded information for each coded depth requires not only the coded depth but also prediction related information and frequency transform related information. Therefore, the coding depth determiner 120 not only determines the coding depth at which the minimum coding error is generated, but also the partition type in which the coding unit of the coding depth is divided into prediction units, the prediction unit-specific prediction mode, Can be determined.

The coding depth determiner 120 may measure encoding errors of coding units according to depths using a rate-distortion optimization technique based on a Lagrangian multiplier.

The output unit 130 outputs, in the form of a bit stream, video data of the maximum encoding unit encoded based on at least one encoding depth determined by the encoding depth determination unit 120 and information on the depth encoding mode.

The encoded image data may be a result of encoding residual data of the image.

The information about the encoding modes for each depth may include coded depth information, partition type information of a prediction unit of a coding unit of a coding depth, prediction mode information for each prediction unit, size information of a transformation unit, and the like.

The coded depth information may be defined using depth-specific segmentation information indicating whether to encode to a coding unit of a lower depth without encoding to the current depth. If the current depth of the current coding unit is a coding depth, since the current coding unit is encoded in a coding unit of the current depth, split information of the current depth may be defined so that it is no longer divided into lower depths. On the contrary, if the current depth of the current coding unit is not the coding depth, encoding should be attempted using the coding unit of the lower depth, and thus split information of the current depth may be defined to be divided into coding units of the lower depth.

If the current depth is not the coded depth, encoding is performed on the coding unit divided into the coding units of the lower depth. Since at least one coding unit of a lower depth exists in the coding unit of the current depth, encoding may be repeatedly performed for each coding unit of each lower depth, and recursive coding may be performed for each coding unit of the same depth.

Since at least one coding depth is determined in one maximum coding unit and information about at least one coding mode should be determined for each coding depth, information about at least one coding mode may be determined for one maximum coding unit. Since the data of the maximum encoding unit is hierarchically divided according to the depth and the depth of encoding may be different for each position, information on the encoding depth and the encoding mode may be set for the data.

Therefore, the outputter 130 according to an embodiment may set corresponding encoding information for each minimum coding unit included in the maximum coding unit. That is, the coding unit of the coded depth includes at least one minimum coding unit having the same coding information. By using this, if the neighboring minimum coding units have the same depth-specific encoding information, it may be the minimum coding unit included in the same maximum coding unit.

For example, the encoding information output through the output unit 130 may be classified into encoding information per depth unit and encoding information per prediction unit. The under-coding-by-depth coding unit information may include prediction mode information and partition size information. The encoding information to be transmitted for each prediction unit includes information about the estimation direction of the inter mode, information about the reference picture index of the inter mode, information on the motion vector, information on the chroma component of the intra mode, information on the interpolation mode of the intra mode And the like. Information on the maximum size of a coding unit defined for each picture, slice or GOP, and information on the maximum depth can be inserted into the header of the bitstream.

According to an embodiment of the simplest form of the video encoding apparatus 100, a coding unit according to depths is a coding unit having a size in which a height and a width of a coding unit of one layer higher depth are divided by half. That is, if the size of the coding unit of the current depth is 2Nx2N, the size of the coding unit of the lower depth is NxN. In addition, the current coding unit having a size of 2N × 2N may include up to four lower depth coding units having a size of N × N.

Therefore, the video decoding apparatus 100 according to an embodiment determines an encoding unit of an optimal shape and size for each maximum encoding unit based on the size and the maximum depth of the maximum encoding unit determined in consideration of the characteristics of the current picture . In addition, since each encoding unit can be encoded by various prediction modes, frequency conversion methods, and the like, an optimal encoding mode can be determined in consideration of image characteristics of encoding units of various image sizes.

Therefore, if an image having a very high image resolution or a very large data amount is encoded in units of existing macroblocks, the number of macroblocks per picture becomes excessively large. This increases the amount of compression information generated for each macroblock, so that the burden of transmission of compressed information increases and the data compression efficiency tends to decrease. Therefore, the video encoding apparatus according to an embodiment can increase the maximum size of the encoding unit in consideration of the image size, and adjust the encoding unit in consideration of the image characteristic, so that the image compression efficiency can be increased.

2 shows a block diagram of a video decoding apparatus according to an embodiment of the present invention.

The video decoding apparatus 200 includes a receiving unit 210, an image data and encoding information extracting unit 220, and an image data decoding unit 230. The definition of various terms such as coding unit, depth, prediction unit, conversion unit, and information on various coding modes for various processing of the video decoding apparatus 200 according to an embodiment is the same as that of FIG. 1 and the video coding apparatus 100 Are the same as described above.

The receiver 205 receives and parses a bitstream of an encoded video. The image data and encoding information extractor 220 extracts image data for each largest coding unit from the parsed bitstream and outputs the image data to the image data decoder 230. The image data and encoding information extracting unit 220 can extract information on the maximum size of the encoding unit of the current picture from the header of the current picture.

Also, the image data and encoding information extractor 220 extracts information about a coded depth and an encoding mode for each largest coding unit from the parsed bitstream. The extracted information about the coded depth and the coding mode is output to the image data decoder 230. That is, the image data of the bit string may be divided into maximum coding units so that the image data decoder 230 may decode the image data for each maximum coding unit.

The information about the coded depth and the encoding mode for each largest coding unit may be set for one or more coded depth information, and the information about the coding mode for each coded depth may include partition type information of a prediction unit for each coding unit, prediction mode information, and the like. The size information of the transformation unit may be included. In addition, as the encoding depth information, depth-based segmentation information may be extracted.

The information about the coded depth and the encoding mode according to the maximum coding units extracted by the image data and the encoding information extractor 220 may be encoded according to the depth according to the maximum coding unit, as in the video encoding apparatus 100 according to an embodiment. Information about a coded depth and an encoding mode determined to repeatedly perform encoding for each unit to generate a minimum encoding error. Therefore, the video decoding apparatus 200 may reconstruct an image by decoding data according to an encoding method that generates a minimum encoding error.

The image data and encoding information extracting unit 220 can extract information on the encoding depth and the encoding mode for each minimum encoding unit. If information on the coding depth and the coding mode of the corresponding maximum coding unit is recorded for each minimum coding unit, the minimum coding units having the same coding depth and information on the coding mode are estimated as data units included in the same maximum coding unit . That is, when the minimum coding unit of the same information is collected and decoded, decoding based on the coding unit of the coded depth having the smallest coding error is possible.

The image data decoder 230 reconstructs the current picture by decoding image data of each maximum coding unit based on the information about the coded depth and the encoding mode for each maximum coding unit. Based on the coded depth information for each maximum coding unit, the image data decoder 230 may decode the image data for each coding unit of at least one coded depth. The decoding process may include a prediction process including intra prediction and motion compensation, and an inverse frequency conversion process.

The image data decoding unit 230 decodes each of the prediction units and prediction modes for each coding unit on the basis of the partition type information and the prediction mode information of the prediction unit of the coding unit for each coding depth for each coding unit, Or motion compensation.

In addition, the image data decoding unit 230 may perform frequency inverse transform for each encoding unit on the basis of the size information of the conversion unit of each encoding depth-based encoding unit for frequency inverse conversion for each maximum encoding unit .

The image data decoder 230 may determine a coded depth of a current maximum coding unit using split information according to depths. If the split information indicates decoding to the current depth, the current depth is the coded depth. Therefore, the image data decoder 230 may decode the coding unit of the current depth using the partition type, the prediction mode, and the transformation unit size information of the prediction unit with respect to the image data of the current maximum coding unit.

In other words, by observing the encoding information set for the minimum coding unit, the minimum coding unit holding the encoding information including the same split information can be collected and decoded into one data unit.

The video decoding apparatus 200 according to an exemplary embodiment recursively performs encoding for each maximum encoding unit in the encoding process to obtain information on an encoding unit that has generated the minimum encoding error and can use the encoded information for decoding the current picture have. That is, the image data can be decoded in an optimal coding unit for each maximum coding unit.

Therefore, even if a high resolution image or an excessively large amount of data is used, the image data can be efficiently used according to the coding unit size and the encoding mode that are adaptively determined according to the characteristics of the image by using the information about the optimum encoding mode transmitted from the encoding end. Can be decoded and restored.

FIG. 3 shows the concept of a hierarchical coding unit.

An example of an encoding unit may include 32x32, 16x16, 8x8, and 4x4 from an encoding unit with a width x height of 64x64. In addition to the square coding units, there may be coding units having a width x height of 64x32, 32x64, 32x16, 16x32, 16x8, 8x16, 8x4, and 4x8.

With respect to the video data 310, the resolution is set to 1920 x 1080, the maximum size of the encoding unit is set to 64, and the maximum depth is set to 2. As for the video data 320, the resolution is set to 1920x1080, the maximum size of the coding unit is 64, and the maximum depth is 4. With respect to the video data 330, the resolution is set to 352 x 288, the maximum size of the encoding unit is set to 16, and the maximum depth is set to 2.

It is preferable that the maximum size of the coding size is relatively large in order to improve the coding efficiency as well as to accurately characterize the image characteristics when the resolution or the data amount is large. Accordingly, the video data 310 or 320 having a higher resolution than the video data 330 may be selected to have a maximum size of 64.

The maximum depth represents the total number of layers in the hierarchical coding unit. Therefore, since the maximum depth of the video data 310 is 2, the coding unit 315 of the video data 310 is encoded from the largest coding unit having a long axis size of 64 by two layers of depth and having a long axis size of 32 or 16. It can include up to units. On the other hand, since the maximum depth of the video data 330 is 2, the coding unit 335 of the video data 330 has two long depths of encoding from the coding units having the long axis size of 16, so that the long axis sizes are 8 and 4 It can include up to units.

Since the maximum depth of the video data 320 is 4, the coding unit 325 of the video data 320 has a depth of four layers deeper from the maximum coding unit having a long axis size of 64, so that the long axis size is 32, 16, 8, 4 Up to coding units may be included. As the depth increases, the expressive power of the detailed information may be improved.

4 is a block diagram of an image encoder based on coding units, according to an embodiment of the present invention.

The image encoding unit 400 according to an exemplary embodiment includes operations to encode image data in the encoding depth determination unit 120 of the video encoding apparatus 100. That is, the intraprediction unit 410 performs intraprediction on the intra-mode encoding unit of the current frame 405, and the motion estimation unit 420 and the motion compensation unit 425 perform intraprediction on the current frame 405 of the inter- And a reference frame 495. The inter-frame estimation and the motion compensation are performed using the reference frame and the reference frame.

The data output from the intraprediction unit 410, the motion estimation unit 420 and the motion compensation unit 425 is output as a quantized transform coefficient through the frequency transform unit 430 and the quantization unit 440. The quantized transform coefficients are reconstructed into spatial domain data through the inverse quantization unit 460 and the frequency inverse transform unit 470 and the data of the reconstructed spatial domain is passed through the deblocking unit 480 and the loop filtering unit 490 Processed and output to the reference frame 495. [ The quantized transform coefficients may be output to the bitstream 455 via the entropy encoder 450.

The motion estimation unit 420, the motion compensation unit 425, and the frequency transformation unit 420, which are components of the image encoding unit 400, are applied to the video encoding apparatus 100 according to an embodiment of the present invention. The quantization unit 440, the entropy encoding unit 450, the inverse quantization unit 460, the frequency inverse transform unit 470, the deblocking unit 480, and the loop filtering unit 490, The work should be performed based on the depth encoding unit considering the maximum depth.

In particular, the intra predictor 410, the motion estimator 420, and the motion compensator 425 determine the prediction unit and the prediction mode in the coding unit in consideration of the maximum size and the depth of the coding unit, and the frequency converter 430. ) Should consider the size of the transform unit in consideration of the maximum size and depth of the coding unit.

5 is a block diagram of an image decoding unit based on an encoding unit according to an embodiment of the present invention.

The bitstream 505 passes through the parsing unit 510 and the encoded image data to be decoded and the encoding-related information necessary for decoding are parsed. The encoded image data is output as inverse quantized data through the entropy decoding unit 520 and the inverse quantization unit 530, and the image data in the spatial domain is restored through the frequency inverse transform unit 540.

For the image data of the spatial domain, the intra prediction unit 550 performs intra prediction on the coding unit of the intra mode, and the motion compensator 560 uses the reference frame 585 together to apply the coding unit of the inter mode. Perform motion compensation for the

Data in the spatial domain that has passed through the intra predictor 550 and the motion compensator 560 may be post-processed through the deblocking unit 570 and the loop filtering unit 580 to be output to the reconstructed frame 595. Further, the post-processed data via deblocking unit 570 and loop filtering unit 580 may be output as reference frame 585.

In order to decode the image data in the image data decoder 230 of the video decoding apparatus 200, step-by-step operations after the parser 510 of the image decoder 500 according to an embodiment may be performed.

The entropy decoding unit 520, the inverse quantization unit 530, and the frequency inverse transforming unit 520, which are components of the image decoding unit 400, in order to be applied to the video decoding apparatus 200 according to one embodiment. The intraprediction unit 550, the motion compensation unit 560, the deblocking unit 570 and the loop filtering unit 580 all have to perform an operation based on the encoding unit of the encoding depth for each maximum encoding unit .

In particular, the intra predictor 550 and the motion compensator 560 determine a coding unit and a prediction mode in consideration of the maximum size and depth of the coding unit, and the frequency inverse transformer 540 determines the maximum size and depth of the coding unit. Consideration should be given to the size of the transformation unit.

FIG. 6 illustrates a depth-based coding unit and a prediction unit according to an embodiment of the present invention.

The video encoding apparatus 100 according to an embodiment and the video decoding apparatus 200 according to an embodiment use hierarchical coding units to consider image characteristics. The maximum height, width, and maximum depth of the coding unit may be adaptively determined according to the characteristics of the image, and may be variously set according to a user's request. According to the maximum size of the preset coding unit, the size of the coding unit for each depth may be determined.

The hierarchical structure 600 of the encoding unit according to an embodiment shows a case where the maximum height and width of the encoding unit is 64 and the maximum depth is 4. Since the depth deepens along the vertical axis of the hierarchical structure 600 of the coding unit according to an embodiment, the height and the width of the coding unit for each depth are divided. In addition, along the horizontal axis of the hierarchical structure 600 of the coding unit, a prediction unit which is a partial data unit on which prediction coding of each depth coding unit is based is shown.

That is, the coding unit 610 has a depth of 0 as the largest coding unit of the hierarchical structure 600 of the coding unit, and the size, ie, the height and width, of the coding unit is 64x64. A depth-1 encoding unit 620 having a size of 32x32, a depth-2 encoding unit 620 having a size 16x16, a depth-3 encoding unit 640 having a size 8x8, a depth 4x4 having a size 4x4, There is an encoding unit 650. An encoding unit 650 of depth 4 of size 4x4 is the minimum encoding unit.

The partial data units are arranged as a prediction unit of the encoding unit along the horizontal axis for each depth. That is, the prediction unit of the encoding unit 610 having a size of 64x64 with a depth of 0 is a partial data unit 610 having a size of 64x64, partial data units 612 having a size of 64x32 included in the encoding unit 610 of size 64x64, Partial data units 614 of size 32x64, and partial data units 616 of size 32x32. Conversely, the encoding unit may be a minimum-sized square data unit including the conversion units 610, 612, 614, and 616.

Likewise, the prediction unit of the encoding unit 620 of the size 32x32 of the depth 1 is the partial data unit 620 of the size 32x32, the partial data units 622 of the size 32x16, and the partial data units 620 of the size 32x32 included in the encoding unit 620 of the size 32x32, Partial data units 624 of size 16x32, partial data units 626 of size 16x16.

Likewise, the prediction unit of a 16x16 size 16x16 encoding unit is a 16x16 partial data unit 630, a 16x8 partial data unit 632, and a 16x16 partial data unit 630 included in the 16x16 encoding unit 630, Partial data units 634 of size 8x16, and partial data units 636 of size 8x8.

Similarly, the prediction unit of the coding unit 640 of size 8x8 having a depth of 3 includes the partial data unit 640 of the size 8x8 included in the coding unit 640 of the size 8x8, the partial data units 642 of the size 8x4, It may be partial data units 644 of size 4x8 and partial data units 646 of size 4x4.

Finally, the coding unit 650 of size 4x4 having a depth of 4 is the minimum coding unit and the coding unit of the lowest depth, and the corresponding prediction unit is also the data unit 650 having a size of 4x4.

The coding depth determiner 120 of the video coding apparatus 100 according to an exemplary embodiment of the present invention determines the coding depth of the maximum coding unit 610 by multiplying the coding unit of each depth included in the maximum coding unit 610 Encoding is performed.

The number of deeper coding units according to depths for including data having the same range and size increases as the depth increases. For example, four coding units of depth 2 are required for data included in one coding unit of depth 1. Therefore, in order to compare the encoding results of the same data for each depth, each of the coding units having one depth 1 and four coding units having four depths 2 should be encoded.

For each depth coding, encoding may be performed for each prediction unit of a coding unit according to depths along a horizontal axis of the hierarchical structure 600 of the coding unit, and a representative coding error, which is the smallest coding error at a corresponding depth, may be selected. . In addition, a depth deeper along the vertical axis of the hierarchical structure 600 of the coding unit, the encoding may be performed for each depth, and the minimum coding error may be searched by comparing the representative coding error for each depth. The depth in which the minimum coding error occurs in the maximum coding unit 610 may be selected as the coding depth and the partition type of the maximum coding unit 610.

FIG. 7 shows a relationship between an encoding unit and a conversion unit according to an embodiment of the present invention.

The video encoding apparatus 100 according to an embodiment or the video decoding apparatus 200 according to an embodiment encodes or decodes an image in coding units having a size smaller than or equal to the maximum coding unit for each maximum coding unit. The size of the conversion unit for frequency conversion during encoding can be selected based on data units that are not larger than the respective encoding units.

For example, in the video encoding apparatus 100 or the video encoding apparatus 200 according to an embodiment, when the current encoding unit 710 is 64x64 size, the 32x32 conversion unit 720 The frequency conversion can be performed.

In addition, the data of the 64x64 coding unit 710 is encoded by performing frequency transformation on the 32x32, 16x16, 8x8, and 4x4 transform units having a size of 64x64 or less, and the transform unit having the least error with the original is obtained. Can be selected.

FIG. 8 illustrates depth-specific encoding information, in accordance with an embodiment of the present invention.

The encoding information encoding unit of the video encoding apparatus 100 according to the embodiment is information on an encoding mode, and includes information on a partition type 800, information on a prediction mode 810, Information 820 on the unit size can be encoded and transmitted.

The partition type information 800 indicates information on a type in which the current encoding unit is divided as a prediction unit for predictive encoding of the current encoding unit. For example, the current coding unit CU_0 of depth 0 and size 2Nx2N includes a prediction unit 802 of size 2Nx2N, a prediction unit 804 of size 2NxN, a prediction unit 806 of size Nx2N, and a prediction unit of size NxN 808. ) May be divided into any one type and used as a prediction unit. In this case, the information 800 about the partition type of the current coding unit includes a prediction unit 802 of size 2Nx2N, a prediction unit 804 of size 2NxN, a prediction unit 806 of size Nx2N, and a prediction unit 808 of size NxN. It is set to indicate one of the.

Information 810 about the prediction mode indicates a prediction mode of each prediction unit. For example, through the information 810 about the prediction mode, the prediction unit indicated by the information 800 about the partition type is performed by one of the intra mode 812, the inter mode 814, and the skip mode 816. Can be set.

In addition, the information 820 on the conversion unit size indicates whether to perform frequency conversion on the basis of which conversion unit the current encoding unit is performed. For example, the transform unit may be one of a first intra transform unit size 822, a second intra transform unit size 824, a first inter transform unit size 826, and a second intra transform unit size 828. have.

The encoding information extracting unit of the video decoding apparatus 200 according to the embodiment extracts the information 800 about the partition type, the information 810 about the prediction mode, the information 820 about the conversion unit size ) Can be extracted and used for decoding.

FIG. 9 shows a depth encoding unit according to an embodiment of the present invention.

Partition information may be used to indicate changes in depth. The division information indicates whether the current-depth encoding unit is divided into lower-depth encoding units.

The prediction unit 910 for predictive encoding of coding units having a depth of 0 and 2N_0x2N_0 size includes a partition type 912 having a size of 2N_0x2N_0, a partition type having a size of 2N_0xN_0 (914), a partition type having a size of N_0x2N_0 (916), and a partition having a size of N_0xN_0 Type 918 may be included.

For each partition type, prediction coding must be repeatedly performed for one prediction unit of 2N_0x2N_0 size, two prediction units of 2N_0xN_0 size, two prediction units of N_0x2N_0 size, and four prediction units of N_0xN_0 size. For a prediction unit of size 2N_0x2N_0, size N_0x2N_0, size 2N_0xN_0, and size N_0xN_0, predictive coding may be performed in intra mode and inter mode. The skip mode can be performed only for predictive encoding in a prediction unit of size 2N_0x2N_0.

If the encoding error of the partition type 918 having the size N_0xN_0 is the smallest, the depth 0 is changed to 1 (920), and the depth 2 and the coding units 922, 924, 926, and 928 of the partition type having the size N_0xN_0 are changed. We can repeatedly search for the minimum coding error.

Since encoding is repeatedly performed on the encoding units 922, 924, 926, and 928 of the same depth, encoding of only one of the encoding units of depth 1, for example, will be described. The prediction unit 930 for predictive encoding of a coding unit having a depth of 1 and a size of 2N_1x2N_1 (= N_0xN_0) includes a partition type 932 of size 2N_1x2N_1, a partition type 934 of size 2N_1xN_1, and a partition type 936 of size N_1x2N_1. It may include a partition type 938 of size N_1xN_1. For each partition type, prediction encoding must be repeatedly performed for each prediction unit of size 2N_1x2N_1, prediction units of two sizes 2N_1xN_1, prediction units of two sizes N_1x2N_1, and prediction units of four sizes N_1xN_1.

Also, if the encoding error of the partition type 938 having the size N_1xN_1 is the smallest, the depth 1 is changed to the depth 2 (940), and the coding units 942, 944, 946, and 948 of the depth 2 and the size N_2xN_2 are used. We can repeatedly search for the minimum coding error.

In the case where the maximum depth is d, the split information for each depth may be set until the depth d-1. That is, the prediction unit 950 for the prediction encoding of the coding unit of the depth d-1 and the size 2N_ (d-1) x2N_ (d-1) is a partition having the size 2N_ (d-1) x2N_ (d-1). Partition type 954 of type 952, size 2N_ (d-1) xN_ (d-1), partition type 956 of size N_ (d-1) x2N_ (d-1), size N_ (d- 1) xN_ (d-1) may include a partition type 958.

(D-1) x2N_ (d-1), two predicted units of two sizes 2N_ (d-1) (d-1) x2N_ (d-1), four sizes N_ (d-1) xN_ (d-1). Since the maximum depth is d, the coding unit 952 of the depth d-1 no longer undergoes a division process.

The video coding apparatus 100 according to an exemplary embodiment compares depth-based coding errors to determine the depth of coding for the coding unit 912, and selects the depth at which the smallest coding error occurs.

For example, as the encoding error for the coding unit having a depth of 0, a prediction unit in which the smallest coding error occurs after predictive encoding is performed for each partition type 912, 914, 916, and 918 is determined. Similarly, prediction units having the smallest coding error may be searched for depths 0, 1, ..., d-1. In the depth d, a coding error may be determined through prediction coding based on the prediction unit 960 which is a coding unit having a size of 2N_dx2N_d.

In this way, the depth with the smallest error can be determined by comparing the minimum coding errors for all depths of depths 0, 1, ..., d-1, d, and can be determined as the coding depth. The coded depth and the prediction unit of the corresponding depth may be encoded and transmitted as information about an encoding mode. In addition, since the coding unit must be split from the depth 0 to the coded depth, only the split information of the coded depth is set to '0', and the split information for each depth except the coded depth should be set to '1'.

The encoding information extracting unit 220 of the video decoding apparatus 200 according to an exemplary embodiment may extract information on the encoding depth and prediction unit of the encoding unit 912 and decode the encoding unit 912. [ The video decoding apparatus 200 according to an embodiment may identify a depth having split information of '0' as a coding depth using split information for each depth, and may use the decoding depth by using information about an encoding mode for a corresponding depth. have.

FIGS. 10A, 10B, and 10C illustrate the relationship between an encoding unit, a prediction unit, and a frequency conversion unit according to an embodiment of the present invention.

The coding units 1010 are coding units according to coding depths determined by the video encoding apparatus 100 according to an embodiment with respect to the maximum coding unit. The prediction unit 1060 is prediction units of coding units of respective coding depths of the coding unit 1010, and the transformation unit 1070 is transformation units of coding units of each coding depth.

If the depth-based coding units 1010 have a depth of 0, the coding units 1012 and 1054 have a depth of 1, and the coding units 1014, 1016, 1018, 1028, 1050, and 1052 have depths. 2, coding units 1020, 1022, 1024, 1026, 1030, 1032, and 1048 have a depth of three, and coding units 1040, 1042, 1044, and 1046 have a depth of four.

Some of the prediction units 1060 1014, 1016, 1022, 1032, 1048, 1050, 1052, and 1054 are types in which coding units are split. That is, the prediction units 1014, 1022, 1050, and 1054 are partition types of 2N × N, the prediction units 1016, 1048, and 1052 are partition types of N × 2N, and the prediction units 1032 are partition types of N × N. That is, the prediction unit of the deeper coding units 1010 is smaller than or equal to each coding unit.

The image data of a part 1052 of the conversion units 1070 is subjected to frequency conversion or frequency inverse conversion in units of data smaller in size than the encoding unit. In addition, the transformation units 1014, 1016, 1022, 1032, 1048, 1050, 1052, and 1054 are data units having different sizes or shapes when compared to the corresponding prediction unit among the prediction units 1060. That is, the video encoding apparatus 100 according to the embodiment and the video decoding apparatus 200 according to an embodiment can perform the intra prediction / motion estimation / motion compensation operation for the same encoding unit and the frequency conversion / , Each based on a separate data unit.

11 is a diagram of encoding information according to coding units, according to an embodiment of the present invention.

The output unit 130 of the video encoding apparatus 100 according to an exemplary embodiment outputs encoding information for each encoding unit and the encoding information extracting unit 220 of the video decoding apparatus 200 according to an embodiment extracts encoding information for each encoding unit It is possible to extract the encoding information.

The encoding information may include split information about a coding unit, partition type information, prediction mode information, and transformation unit size information. The encoding information shown in FIG. 11 is an example that can be set in the video encoding apparatus 100 according to the embodiment and the video encoding apparatus 200 according to an embodiment.

The split information may indicate a coded depth of a corresponding coding unit. That is, since a depth that is no longer divided according to the split information is a coded depth, partition type information, a prediction mode, and transform unit size information may be defined for the coded depth. If it is to be further split by the split information, encoding should be performed independently for each coding unit of the divided four lower depths.

The partition type information may indicate a partition type of a transformation unit of a coding unit of a coding depth as one of 2Nx2N, 2NxN, Nx2N, and NxN. The prediction mode may be represented by one of an intra mode, an inter mode, and a skip mode. Intra mode and inter mode may be defined in partition types 2Nx2N, 2NxN, Nx2N and NxN, and a skip mode may be defined only in partition type 2Nx2N. The conversion unit size may be set to two kinds of sizes in the intra mode and two kinds of sizes in the inter mode.

For each of the minimum coding units in the coding unit, encoding information for each coding unit of the coded depth to which they belong may be stored. Therefore, when the encoding information held by the adjacent minimum coding units are respectively checked, it may be determined whether the information is included in the coding unit having the same coding depth. In addition, since the coding unit of the corresponding coding depth may be identified by using the encoding information held by the minimum coding unit, the distribution of the coded depths within the maximum coding unit may be inferred.

In this case, when the current encoding unit is predicted with reference to the neighboring data unit, the encoding information of the minimum encoding unit in the depth encoding unit adjacent to the current encoding unit is directly used, so that the data of the minimum encoding unit can be referred to.

In another embodiment, the encoding information of the coding units according to depths may be stored only for the smallest coding unit that is represented among the coding units according to depths. In this case, when the current coding unit is predicted with reference to the neighboring coding unit, the data adjacent to the current coding unit may be referred to in the depth-wise coding unit by using the encoding information of the adjacent coding units according to depths.

12 shows a flowchart of a video coding method according to an embodiment of the present invention.

In operation 1210, the current picture is split into at least one maximum coding unit. In addition, a maximum depth indicating the total number of possible divisions may be set in advance.

In step 1220, the area of the maximum coding unit is coded at least in one divided area for each depth, and the depth at which the final coding result is output for each of at least one of the divided areas is determined. The maximum encoding unit is divided into stages, and each time the depth is deepened, it is necessary to repeatedly perform encoding for each lower-depth encoding unit.

For each depth-based coding unit, the conversion unit for each partition type having the smallest coding error should be determined. In order to determine the coding depth that causes the minimum coding error of the coding unit, the coding error should be measured and compared for each coding unit of each depth.

In operation 1230, image data, which is a final encoding result of at least one divided region, for each maximum coding unit, and information about an encoding depth and an encoding mode are output. The information on the encoding mode may include information on the encoding depth or division information, partition type information of the encoding depth, prediction mode information, and conversion unit size information. The information about the encoded encoding mode may be transmitted to the decoding end together with the encoded image data.

13 shows a flowchart of a video decoding method according to an embodiment of the present invention.

In step 1310, a bitstream for the encoded video is received and parsed.

In operation 1320, image data of the current picture allocated to the largest coding unit having the maximum size, and information about a coded depth and an encoding mode for each largest coding unit are extracted from the parsed bitstream. The coded depth of each largest coding unit is a depth selected to have the smallest coding error for each largest coding unit in the encoding process of the current picture. In encoding by the largest coding unit, image data is encoded based on at least one data unit obtained by hierarchically dividing the maximum coding unit by depth. Accordingly, the efficiency of encoding and decoding of an image can be improved by decoding the respective image data after determining the coding depth of each coding unit.

In operation 1330, image data of each maximum coding unit is decoded based on the information about the coded depth and the encoding mode for each maximum coding unit. The decoded image data can be reproduced by a reproducing apparatus, stored in a storage medium, or transmitted via a network.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be embodied as a program that can be executed by a computer and operates the program using a computer-readable recording medium. The computer readable recording medium may be a magnetic storage medium such as a ROM, a floppy disk, a hard disk, etc., an optical reading medium such as a CD-ROM or a DVD and a carrier wave such as the Internet Lt; / RTI > transmission).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (4)

In the video decoding method,
Receiving and parsing a bitstream of an encoded image;
Extracting, from the bitstream, maximum coding unit size information indicating a maximum size of a coding unit which is a data unit for decoding an image, and information about a coded depth and an encoding mode for coding units of the image; And
Determining coding units according to a tree structure and decoding the image based on coding units according to the tree structure using the maximum coding unit size information and the information about the coded depth and the encoding mode.
The image is hierarchically divided from a maximum coding unit according to the maximum coding unit size information to coding units of the coded depth according to a depth,
The encoding unit of the current depth is one of the square data units divided from the encoding unit of the higher depth,
The coding unit of the current depth is divided into coding units of a lower depth up to a corresponding coding depth independently of neighboring coding units.
The coding units according to the tree structure may include coding units of the coded depth among coding units split from the maximum coding unit. The method of claim 1, wherein the determining of the coding units of the tree structure and decoding the image comprises:
Among the information about the coded depth and the encoding mode, the coding unit that is no longer divided into coding units of a lower depth, based on split information indicating whether the coding units are split into coding units of a lower depth, the coding unit And determining the depth by the coding unit of the depth. The method of claim 2, wherein the decoding step,
Determining a partition type and a prediction mode of a current coding unit among coding units of the coded depth, based on the information about the coded depth and the coded mode for the coded units according to the tree structure; And
Based on the partition type, a prediction unit having the same size as the current coding unit or partitions in which at least one of a height and a width of the current coding unit are divided are determined, and the prediction unit or the partition is based on the prediction mode. And performing intra prediction or inter prediction for each of them. The method of claim 1, wherein the extracting step,
And extracting, from the bitstream, information on the height and width of the maximum coding unit and information indicating a maximum depth indicating a total number of times that the maximum coding unit can be hierarchically divided. Decryption method.

Images