KR20110112165A - Method and apparatus for video encoding using dynamic-range trasformation, method and apparatus for video decoding using dynamic-range transformation - Google Patents

Abstract

Using image data obtained by converting the dynamic range of the current frame based on the content of the input video, inter prediction through intra prediction and motion estimation for the current frame is performed, and residual data generated by intra prediction and inter prediction. A video encoding method using a content-based dynamic range transform, which performs a frequency transform on the quantized transform, performs quantization on the transform coefficients generated by the frequency transform, and performs entropy encoding on the quantized transform coefficients.

Description

Method and apparatus for video encoding using content-based dynamic range transformation, and apparatus and method and apparatus for video decoding using content-based dynamic range transformation, method and apparatus for video decoding using dynamic-range transformation}

The present invention relates to the encoding and decoding of video.

With the development and dissemination of hardware capable of playing and storing high resolution or high definition video content, there is an increasing need for a video codec for efficiently encoding or decoding high resolution or high definition video content. According to the existing video codec, video is encoded according to a limited encoding method based on a macroblock of a predetermined size.

The digital video capture device obtains video data of pixel values based on a fixed bit-depth. In order to improve the accuracy of operations during the encoding and decoding process, a technique of temporarily extending the bit depth of a pixel value is used.

The present invention relates to encoding and decoding of video having a dynamic range converted based on content.

According to an embodiment of the present invention, a video encoding method using content-based dynamic range transformation uses intra prediction and motion estimation using image data obtained by converting a dynamic range of a current frame based on content of an input video. Performing inter prediction; Performing frequency transform on the residual data generated by the intra prediction and inter prediction, and performing quantization on the transform coefficients generated by the frequency transform; And performing entropy encoding on the quantized transform coefficients, and outputting a bitstream including encoded data for the current frame.

In the performing of the intra prediction and the inter prediction according to an embodiment, the dynamic range may be extended so as to extend the pixel values of the current frame to a range from a maximum value to a minimum value that can be expressed as a current bit depth of a pixel value. The intra prediction and the inter prediction may be performed using the converted image data.

The performing of the intra prediction and the inter prediction according to an embodiment may include converting a value larger than an upper limit among pixel values of the current frame to correspond to the maximum value, and a value smaller than a lower limit among pixel values of the current frame. May be converted to correspond to the minimum value, and pixel values between the upper limit value and the lower limit value among the pixel values of the current frame may be converted to have a one-to-one correspondence with the value between the maximum limit value and the minimum value. The upper limit value and the lower limit value according to an embodiment may be the local maximum and local minimum among pixel values of the current frame, respectively.

The performing of the intra prediction and the inter prediction according to an embodiment may include: multiplying a difference between a current pixel value and the lower limit by the pixel value between the upper limit value and the lower limit value, and a difference between the maximum limit value and the maximum limit value. The value divided by the difference between the upper limit value and the lower limit value may be converted to correspond one-to-one.

According to an exemplary embodiment, a pixel value corresponding one-to-one with respect to pixel values between the upper limit value and the lower limit value may be output through a bit shift operation using a current bit depth.

According to another embodiment, the current bit depth may be an internally extended bit depth during the video decoding process. According to another exemplary embodiment, the current bit depth may be a sum of a first bit depth and an internally extended second bit depth for extending the dynamic range. The performing of the intra prediction and the inter prediction according to another embodiment may include: extending a bit depth of a pixel value of the video by the second bit depth; Extending a dynamic range by the first bit depth with respect to intermediate data of the bit depth extended by the second bit depth; And performing the intra prediction and the inter prediction on the data having the extended dynamic range.

The dynamic range expansion step may include converting a value larger than an upper limit among the intermediate data to correspond to the maximum limit value, and corresponding to the minimum limit value that is smaller than a lower limit among pixel values of the current frame. The data between the upper limit value and the lower limit value of the intermediate data may be converted to correspond to the value between the maximum limit value and the minimum limit value in a one-to-one correspondence. According to another exemplary embodiment, the dynamic range expansion may include performing a pixel shift corresponding to the pixel values between the upper limit value and the lower limit value by bit shifting using the first bit depth and the second bit depth. Can be output via

In the performing of the intra prediction and the inter prediction according to an embodiment, the intra prediction and the inter prediction may be performed by using image data whose dynamic range is converted for each of the luma component and the chroma component of the pixel values of the current frame. have.

According to an embodiment, pixel values between the upper limit value and the lower limit value among the pixel values of the current frame may be converted to have a one-to-one correspondence to the value between the maximum value and the minimum value according to a nonlinear function relationship. . According to an exemplary embodiment, when pixel values between the upper limit value and the lower limit value among the pixel values of the current frame are divided into a predetermined number of sections, the pixel value of each section linearly increases as a function of the section-by-section gradient. Corresponding result values are determined, and the result values of successive sections among the sections are continuous, so that the result values for pixel values of all the sections are one-to-one to the value between the maximum limit value and the minimum limit value. Can be converted to correspond.

According to one embodiment, the current pixel value larger than the upper limit value or smaller than the lower limit value may be truncated.

The upper limit value and the lower limit value according to an embodiment may be determined for each data unit of an image sequence, a frame, a frame set for intra prediction, an area, and a coding unit of the input video.

According to an embodiment, the video encoding method using the dynamic range transformation based on the content may further include encoding and transmitting information on the transformation of the dynamic range.

The information on the conversion of the dynamic range according to an embodiment may include information about an upper limit value and a lower limit value of the content of the input video.

The information on the conversion of the dynamic range according to an embodiment may include information about a lower limit value of the content of the input video and a value obtained by subtracting the upper limit value from a difference between the maximum value and the minimum value of the pixel value. have.

In the intra prediction step and the inter prediction step, the maximum coding unit is hierarchically as the depth is deepened for each maximum coding unit obtained by dividing the current frame into coding units having a predetermined maximum size. The intra prediction and the inter prediction are performed for each of at least one depth-based coding unit for each divided and reduced region, and the performing of the frequency transform and the quantization may include performing at least one depth for each region for each maximum coding unit. The video encoding method performing the frequency transformation and the quantization for each coding unit and using the dynamic range transformation based on the content includes information on at least one coded depth that generates a minimum coding error with an original image of the picture. For a coding unit of the coded depth The method may further include determining an encoding mode, and the performing the entropy encoding may include performing entropy encoding on the encoded image data that is an encoding result according to the determined encoding depth and the encoding mode, thereby determining the encoding depth and the encoding mode. The bitstream including the information about the encoded image data and the encoded information may be output.

According to an embodiment of the present invention, a video decoding method using dynamic range transformation based on content includes: parsing a received bitstream to extract encoded data of a current frame of an original video; Restoring a data symbol by performing entropy decoding on the extracted encoded data; Reconstructing residual data for the current frame by performing inverse quantization and inverse frequency transformation on the reconstructed data symbol, and reconstructing image data by performing intra prediction and motion compensation on the reconstructed residual data step; And reconstructing the current frame by performing deblocking filtering on the reconstructed image data and reconstructing the dynamic range converted based on the content of the original video.

The reconstructing current frame may include reconstructing a pixel value extended from a maximum value to a minimum value that can be expressed as a current bit depth of a pixel value with respect to the restored image data. It may be restored to the dynamic range of the current frame.

The restoring of the current frame according to an embodiment may include restoring the maximum limit value and the minimum limit value of the restored image data to correspond to an upper limit value and a lower limit value of pixel values of the current frame, respectively. Values between the maximum and minimum limits of the data may be converted to have a one-to-one correspondence with the values between the upper limit and the lower limit.

The restoring of the current frame according to an embodiment may include: multiplying the difference between the currently restored data value, the upper limit value, and the lower limit value with respect to the values between the maximum limit value and the minimum limit value by using the maximum limit value and the maximum limit value. The lower limit value plus the value divided by the difference of the value may be converted to correspond one-to-one.

According to an embodiment, a pixel value corresponding one-to-one with respect to values between the maximum and minimum values may be output through a bit shift operation using a current bit depth.

According to another embodiment, the current bit depth may be an internally extended bit depth during the video decoding process. According to another embodiment, the step of restoring a current frame may include: restoring a dynamic range of the restored image data extended by the first bit depth; And reconstructing the bit depth of the pixel value of the current frame by reducing the bit depth of the intermediate data generated by restoring the dynamic range by the second bit depth.

According to another exemplary embodiment, the dynamic range restoration may include: restoring the maximum limit value and the minimum limit value of the restored image data to correspond to an upper limit value and a lower limit value of the intermediate data, respectively; And generating the intermediate data by converting values between the maximum limit value and the minimum limit value of the reconstructed image data to have a one-to-one correspondence to a value between an upper limit value and a lower limit value of the intermediate data.

In the intermediate data generating step according to another embodiment, the pixel value corresponding to the pixel value between the upper limit value and the lower limit value of the intermediate data is calculated by bit shifting using the first bit depth and the second bit depth. Can be output via

The current frame restoration step may restore the pixel value of the current frame by restoring a dynamic range for each of the luma and chroma components with respect to the restored image data.

According to an embodiment, the values between the maximum value and the minimum value of the reconstructed image data may be converted to have a one-to-one correspondence to values between the upper limit value and the lower limit value according to a nonlinear function relationship. According to an exemplary embodiment, when pixel values between the maximum value and the minimum value of the reconstructed image data are divided into a predetermined number of sections, the data for each section is a function of a slope for each section that increases linearly. The pixel values corresponding to the corresponding pixel values are determined, and the pixel values corresponding to the data of the successive sections among the sections are continuous so that the pixel values for the data of all the sections correspond one-to-one to the values between the upper limit value and the lower limit value. Can be converted.

The video decoding method using the dynamic range transformation based on the content according to an embodiment may further include receiving information on the transformation of the dynamic range of the current frame.

According to an embodiment, the data symbol reconstruction step may include: dividing a picture of an input image sequence into coding units having a predetermined maximum size in the encoding step of the received bitstream, and increasing the depth of each maximum coding unit. Information on at least one coded depth that generates a minimum coding error with respect to an original image of the picture by performing encoding on at least one depth-based coding unit for each region in which the maximum coding unit is hierarchically divided and reduced. Extracting information about an encoding mode for a coding unit of the encoding depth, from the bitstream, and reconstructing the image data is based on the encoding depth and an encoding mode based on the information about the encoding mode. Inverse projection, inverse frequency transformation, intra prediction and motion compensation W can decode the encoded image data.

A video encoding apparatus using dynamic range transformation based on content according to an embodiment of the present invention may include an intra prediction unit configured to perform intra prediction using image data obtained by converting a dynamic range of a current frame based on content of an input video. ; An inter predictor configured to perform inter prediction through motion estimation using the dynamic range converted image data; A frequency converter configured to perform frequency conversion on the residual data generated by the intra prediction and the inter prediction; A quantization unit performing quantization on the transform coefficients generated by the frequency transform; An entropy encoder for performing entropy encoding on the quantized transform coefficients; And an output unit configured to output a bitstream including the encoded data of the current frame, generated by the entropy encoding.

According to an embodiment of the present invention, a video decoding apparatus using dynamic range transformation based on content may include: a parser configured to parse the received bitstream and extract encoded data of a current frame of an original video; An entropy decoder configured to reconstruct data symbols by performing entropy decoding on the extracted encoded data; An inverse quantization and inverse frequency converter configured to perform inverse quantization and inverse frequency transformation on the recovered data symbols to restore residual data for the current frame; An intra predictor for performing intra prediction on the reconstructed residual data; A motion compensation unit to perform motion compensation on the restored residual data; And an image reconstruction unit which deblocks the image data reconstructed by the intra prediction and the motion compensation and reconstructs the current frame by reconstructing the dynamic range converted based on the content of the original video. .

The present invention includes a computer readable recording medium having recorded thereon a program for implementing a video encoding method using a dynamic range transformation based on content according to an embodiment of the present invention.

The present invention includes a computer readable recording medium having recorded thereon a program for implementing a video decoding method using content-based dynamic range conversion according to an embodiment of the present invention.

1 is a block diagram of a video encoding apparatus using dynamic range transformation based on content, according to an embodiment of the present invention.
2 is a block diagram of a video decoding apparatus using dynamic range transformation based on content, according to an embodiment of the present invention.
3 illustrates a graph of data in which a dynamic range is converted according to an embodiment.
4 illustrates a graph of data in which a dynamic range is converted based on an extended bit depth, according to an exemplary embodiment.
5 illustrates a probability density function graph of pixel values of original data according to an embodiment.
6 is a flowchart of a video encoding method using dynamic range transformation based on content, according to an embodiment of the present invention.
7 is a flowchart of a video decoding method using dynamic range transformation based on content according to an embodiment of the present invention.
8 is a block diagram of a video encoding apparatus using a dynamic range transformation based on content, based on a hierarchical data unit for each region according to an embodiment of the present invention.
FIG. 9 is a block diagram of a video decoding apparatus using content-based dynamic range transformation based on hierarchical hierarchical data units according to an embodiment of the present invention.
10 illustrates a concept of coding units, according to an embodiment of the present invention.
11 is a block diagram of an image encoder based on coding units, according to an embodiment of the present invention.
12 is a block diagram of an image decoder based on coding units, according to an embodiment of the present invention.
13 is a diagram of deeper coding units according to depths, and prediction units, according to an embodiment of the present invention.
14 illustrates a relationship between a coding unit and transformation units, according to an embodiment of the present invention.
FIG. 15 illustrates encoding information according to depths, according to an embodiment of the present invention.
16 is a diagram of deeper coding units according to depths, according to an embodiment of the present invention.
17, 18, and 19 illustrate a relationship between a coding unit, a prediction unit, and a frequency transform unit, according to an embodiment of the present invention.
20 is a diagram of encoding information according to coding units, according to an embodiment of the present invention.
21 is a flowchart of a video encoding method using dynamic range transformation based on content, based on a hierarchical data unit for each region according to an embodiment of the present invention.
FIG. 22 is a flowchart of a video decoding method using dynamic range transformation based on content, based on a hierarchical data unit for each region according to an embodiment of the present invention.
FIG. 23 illustrates a graph of data with dynamic range transformed based on a nonlinear function, according to one embodiment. FIG.

Hereinafter, a video encoding method and apparatus therefor, and a video decoding method and various embodiments of the apparatus will be described in detail with reference to FIGS. 1 to 22. Specifically, various embodiments related to video encoding and decoding using dynamic range transformation based on content according to an embodiment of the present invention will be described with reference to FIGS. 1 to 7, and with reference to FIGS. 8 to 22, Various embodiments of video encoding and decoding using dynamic range transformation based on content based on hierarchical hierarchical data units according to an embodiment are disclosed.

1 is a block diagram of a video encoding apparatus using dynamic range transformation based on content according to an embodiment of the present invention.

The video encoding apparatus 100 using the dynamic range transformation based on content according to an embodiment may include an intra predictor 110, an inter predictor 120, a frequency converter 130, a quantizer 140, and an entropy encoding. The unit 150 and the output unit 160 is included.

The intra predictor 110 performs intra prediction on image data obtained by converting a dynamic range of a current frame based on the content of the input video, and the inter predictor 120 according to an embodiment The inter prediction is performed through motion estimation on the dynamic range converted image data of the current frame.

The video encoding apparatus 100 using the dynamic range transformation based on the content according to an embodiment may include a dynamic range of pixel values of a current frame in a maximum range of a minimum value to a maximum value that can be expressed as a current bit depth of a pixel value. Can be extended. Accordingly, the video encoding apparatus 100 using the dynamic range transformation based on the content according to an embodiment may encode the current frame using the image data converted so that the dynamic range of the pixel value is extended to the maximum range.

The intra predictor 110 and the inter predictor 120 perform intra prediction and inter prediction on a current frame by using image data obtained by converting the dynamic range, respectively, and output residual data. Can be.

Hereinafter, in the present specification, a large value among pixel values of an original image set to determine a substantial dynamic range of a current frame is referred to as an upper limit, and a small value is referred to as a lower limit. In addition, the maximum value and the minimum value among the pixel values that can be expressed with a predetermined bit depth are referred to as maximum and minimum values, respectively.

According to an embodiment, the dynamic range is extended so that the pixel values between the upper limit value and the lower limit value of the original video of the input video correspond to the values between the minimum value and the maximum value that can be expressed by a predetermined bit depth, so that the content of the input video content is increased. The dynamic range can be extended based on.

The intra predictor 110 and the inter predictor 120 may convert a value greater than an upper limit among pixel values of the current frame to correspond to a maximum value, and then convert the value greater than a lower limit among pixel values of the current frame. Small values can be converted to correspond to minimum values. In addition, the pixel values between the upper limit value and the lower limit value among the pixel values of the current frame may be converted to have a one-to-one correspondence with a value between the maximum limit value and the minimum limit value.

The intra predictor 110 and the inter predictor 120 may determine an upper limit value and a lower limit value as maximum and minimum values of pixel values of the current frame, respectively. Here, if there is no pixel larger than the current pixel value in the interval, the current pixel value is defined as the maximum value. If no pixel is smaller than the current pixel value, the current pixel value is defined as the minimum value.

For example, the intra predictor 110 and the inter predictor 120 according to an embodiment of the present invention may have a difference between the current pixel value and the lower limit and the maximum limit value of the pixel values between the upper limit value and the lower limit value. And a value obtained by dividing the product of the difference between the minimum value and the difference between the upper limit value and the lower limit value.

According to the dynamic range conversion according to an embodiment, a current pixel value larger than an upper limit or smaller than a lower limit of the current frame may be cut.

The intra prediction unit 110 and the inter prediction unit 120 according to an exemplary embodiment may include a pixel value corresponding to one-to-one corresponding to pixel values between an upper limit value and a lower limit value to a bit shift using a current bit depth in order to improve computation speed. Can be determined by calculation.

The video encoding apparatus 100 using the dynamic range transformation based on the content according to an embodiment may internally extend and use a bit depth of an image pixel value during the decoding process of the input video. As the bit depth of the pixel value is extended, the maximum range of the pixel value may also be extended.

When the intra predictor 110 and the inter predictor 120 perform intra prediction and inter prediction on a current frame using image data according to an internally extended bit depth, the extended bit depth is increased. Intra-prediction and inter-prediction of the current frame may be performed using image data of the current frame converted to extend the dynamic range to a maximum range of pixel values.

For example, when the increase amount of the bit depth according to the extension of the dynamic range is the first bit depth, and the internally expanded bit depth is the second bit depth, the bit depth of the first bit depth and the second bit depth is increased. The dynamic range can be extended to the maximum range of pixel values according to the sum. To this end, an operation in which the bit depth of the pixel value of the video extends by the second bit depth and the operation of extending the dynamic range by the first bit depth with respect to the intermediate data of the bit depth extended by the second bit depth may be sequentially performed. It may be.

The intra predictor 110 and the inter predictor 120 determine an upper limit value and a lower limit value for each predetermined data unit such as an image sequence of an input video, a frame, a frame set for intra prediction, an area, and a coding unit. As a result, the dynamic range may be converted based on the content of a predetermined data unit of the input video.

The intra predictor 110 and the inter predictor 120 perform intra prediction and inter prediction on previous frames whose dynamic range has been converted based on the content of the input video, and then register the previous frame. Dual data can be output.

The residual data for the previous frame may be encoded through frequency transformation and quantization, and the quantized data may be restored to a reconstructed frame for the previous frame through inverse quantization, inverse frequency transformation, and in-loop filtering. If the upper limit value and the lower limit value for the dynamic range conversion are set for each frame, the inter predictor 120 according to an embodiment restores the dynamic range of reconstructed frames for the previous frames to the original dynamic range and then reconstructs the current frame. After reconversion based on the dynamic range, inter prediction of the current frame may be performed with reference to a reconstructed frame converted to the same dynamic range as the current frame.

The intra predictor 110 according to an embodiment may perform intra prediction using information on the peripheral region of the current frame having the converted dynamic range.

The frequency converter 130 according to an embodiment performs frequency conversion on residual data generated by intra prediction and inter prediction, and the quantization unit 140 according to an embodiment is generated by frequency conversion. The quantization is performed on the transform coefficients. The entropy encoder 150 according to an embodiment performs entropy encoding on the quantized transform coefficients. The output unit 160 according to an embodiment outputs a bitstream including encoded data of a current frame generated by entropy encoding.

The output unit 160 according to an embodiment may encode information about the transformation of the dynamic range and transmit the encoded information with the encoded data of the current frame. For example, the information about the transformation of the dynamic range may be inserted into a sequence parameter set or a slice header of the bitstream including the encoded data of the current frame.

The information on the conversion of the dynamic range according to an embodiment may include information about an upper limit value and a lower limit value of the content of the input video. Instead of the information on the upper limit in order to save the number of bits, a value obtained by subtracting the upper limit from the difference between the maximum limit value and the minimum limit value of the pixel value may be encoded.

The video encoding apparatus 100 using dynamic range transformation based on content according to an embodiment may be based on a video encoding method based on hierarchical data units for respective regions according to an embodiment. Based on the video encoding method based on the hierarchical data unit for each region according to an embodiment, the video encoding apparatus using the dynamic range transformation based on the content will be described in detail later with reference to FIGS. 8 to 22.

According to the video encoding apparatus 100 using the dynamic range transformation based on the content according to an embodiment, as the dynamic range of the data for internal calculation is extended during encoding, performance such as prediction encoding, frequency transformation, and in-loop filtering are improved. Can be. In addition, since an upper limit value and a lower limit value are set for each data unit for each data unit, the dynamic range may be converted based on the content.

2 is a block diagram of a video decoding apparatus using dynamic range transformation based on content, according to an embodiment of the present invention.

According to an embodiment, the video decoding apparatus 200 using the dynamic range transformation based on the content may include a parser 210, an entropy decoder 220, an inverse quantization and inverse frequency transformer 230, and an intra predictor ( 240, a motion compensator 250, and an image reconstructor 260.

The parsing unit 210 according to an embodiment parses the received bitstream to extract encoded data of the current frame of the original video. The entropy decoding unit 220 according to an embodiment performs entropy decoding on the extracted encoded data to restore data symbols. The inverse quantization and inverse frequency transform unit 230 according to an embodiment performs inverse quantization and inverse frequency transformation on the recovered data symbols to restore residual data for the current frame. The intra predictor 240 performs intra prediction on the reconstructed residual data, and the motion compensator 250 according to the embodiment reconstructs the residual data reconstructed using the motion vector. Perform motion compensation for the The motion vector may be extracted by the parser 210 and provided to the motion compensator 250.

The image reconstructor 260 according to an embodiment performs in-loop filtering such as deblocking filtering on image data reconstructed by intra prediction and motion compensation, and reconstructs the current frame. In particular, if the extracted encoded data is image data obtained by converting the dynamic range based on the content of the original video, the current frame is restored by restoring the dynamic range. If the information about the dynamic range transformation is extracted by the parser 210, it may be inferred that the extracted encoded data is the dynamic range converted data.

The dynamic range of video data reconstructed by in-loop filtering, such as deblocking filtering, is extended to the range of the maximum value to the minimum value that can be expressed by the current bit depth of the pixel value. Accordingly, according to an exemplary embodiment, the image reconstructor 260 may reconstruct the range of the reconstructed image data to the dynamic range of the current frame.

In order to reconstruct the dynamic range of the reconstructed frame, the image reconstructor 260 according to an embodiment may correspond to the maximum and minimum values of the reconstructed pixel values so as to correspond to the upper limit and the lower limit of the pixel values of the current frame, respectively. It is possible to restore and convert the values between the maximum and minimum values to have a one-to-one correspondence to the values between the upper and lower limits. One example of the upper limit value and the lower limit value of the current frame may be a local maximum and a local minimum among pixel values of the current frame.

According to an exemplary embodiment, the image reconstructor 260 multiplies a value between a currently reconstructed data value and a "difference between an upper limit value and a lower limit value" with respect to values between a maximum limit value and a minimum limit value, using a maximum limit value and a minimum limit value. The dynamic range can be inversely transformed by a one-to-one correspondence between a value obtained by dividing the difference by and a lower limit plus one. The image reconstructor 260 according to an embodiment may calculate a one-to-one correspondence result for inverse transformation of the dynamic range by using a bit shift operation.

According to an embodiment, the current bit depth may be an internally extended bit depth during the video decoding process. In this case, the image reconstructor 260 according to an embodiment may reconstruct the internally expanded bit depth to the original bit depth. For example, the image reconstructor 260 reduces the dynamic range of the reconstructed image data in which the dynamic range extended by the first bit depth is expanded by the first bit depth, and restores the dynamic range and generates the dynamic range by restoring the dynamic range. The operation of restoring the bit depth of the intermediate data internally by the second bit depth to restore the bit depth of the pixel value of the original frame may be sequentially performed.

The pixel value of the current frame reconstructed by the image reconstructor 260 according to an exemplary embodiment may be a truncated result value of a pixel value larger than the upper limit value or smaller than the lower limit value of the corresponding original image.

The upper limit value and the lower limit value of the current frame may be determined for each data unit of an image sequence, a frame, a frame set for intra prediction, an area, and a coding unit of the original video, respectively.

The video decoding apparatus 200 using the dynamic range transformation based on the content according to an embodiment may receive information about the transformation of the dynamic range of the current frame.

According to an embodiment, the received information on the conversion of the dynamic range may include information about an upper limit value and a lower limit value of the content of the original video. If the received information includes information on a value obtained by subtracting the upper limit from the difference between the maximum value and the minimum value of the pixel value, the upper limit value may be inferred therefrom. In addition, information on the data unit in which the upper limit value and the lower limit value of the current frame are set may also be received.

The motion compensator 260 according to an embodiment may perform motion compensation on the current frame by referring to reconstructed frames of the previous frame of the original video. In this case, when the information on the dynamic range conversion is set for each frame, by reconverting the dynamic range of the reconstruction frame to synchronize with the converted dynamic range of the current frame, referring to the reconstructed frame converted to the same dynamic range as the current frame, Motion compensation of the current frame may be performed.

The video decoding apparatus 200 using dynamic range transformation based on content according to an embodiment may be based on a video decoding method based on hierarchical data units for respective regions according to an embodiment. Based on the video decoding method based on the hierarchical data unit for each region according to an embodiment, the video decoding apparatus using the dynamic range transformation based on the content will be described in detail later with reference to FIGS. 8 to 22.

According to the video decoding apparatus 200 using the dynamic range transformation based on the content according to an embodiment, the original video may be reconstructed by decoding the encoded data based on the extended dynamic range for internal calculation during encoding.

3 illustrates a pixel value graph of data in which a dynamic range is converted according to an exemplary embodiment.

Pixel values of the original data are shown on the horizontal axis of the graph, and pixel values of data obtained by converting the dynamic range of the video encoding apparatus 100 and the video decoding apparatus 200 according to the exemplary embodiment are illustrated on the vertical axis. In the graph of FIG. 3, since there is no pixel value larger than the upper limit value B, the maximum value is set to the upper limit value B, and since there is no pixel value smaller than the lower limit value A, the minimum value may be set to the lower limit value A. FIG.

The pixel values of the original data are distributed in intervals of [0, 255], and the intra predictor 110 and the inter predictor 120 according to an embodiment may include data obtained by converting a dynamic range of pixel values of the original data. Intra-prediction and inter-prediction may be performed, respectively. In this case, the pixel value of the lower limit value A corresponds to the minimum value 0 according to the current bit depth, the pixel value of the upper limit value B corresponds to the maximum value 255 according to the current bit depth, and the pixel between the lower limit value A and the upper limit value B. The dynamic range is converted so that the value corresponds one-to-one to the value of the data section (0, 255) between the minimum value and the maximum value according to the current bit depth.

According to an exemplary embodiment, the image reconstructor 260 corresponds to the lower limit A of the current frame and the data value 255 to the upper limit B of the current frame with respect to the data in which the dynamic range has been converted. The data values 255) correspond to the pixel values between the lower limit A and the upper limit B, thereby restoring the original dynamic range.

4 illustrates a graph of pixel values of data obtained by converting a dynamic range based on an extended bit depth, according to an exemplary embodiment.

The pixel value of the original data is on the horizontal axis of the graph, and the vertical axis of the graph is on the basis of the bit depths extended by the video encoding apparatus 100 and the video decoding apparatus 200 according to an embodiment. Is shown.

The video encoding apparatus 100 and the video decoding apparatus 200 according to an embodiment may perform various processing by using the data value converted by extending the bit depth of the pixel value. In this case, the intra predictor 110 and the inter predictor 120 according to an embodiment may use the intra prediction and inter prediction with respect to the current frame using data extended to a maximum range according to a bit depth in which the dynamic range is extended. Can be performed. In addition, the image reconstructor 260 according to an embodiment may reconstruct the dynamic range converted data to the dynamic range of the original data based on the extended bit depth. In addition, the image reconstructor 260 according to an embodiment may reconstruct the internally extended bit depth together with the dynamic range.

For example, when the bit depth of the pixel value is extended from 8 bits to 12 bits and by 4 bits, the data section may be extended from [0, 255] to [0, 4095]. In this case, the pixel value of the lower limit value A corresponds to the minimum value 0 according to the bit depth extended to 12 bits, and the pixel value of the upper limit value B corresponds to the maximum value 4095 according to the 12 bit bit depth, and the lower limit value A and the upper limit value B The dynamic range may be converted so that the pixel value between the pixels corresponds one-to-one to the value of the data section [0, 4095] according to the 12-bit bit depth. Accordingly, the intra predictor 110 and the inter predictor 120 may be used for intra prediction and inter prediction by using a data value of which a dynamic range is converted based on a 4-bit extended bit depth. have.

In this case, the image reconstructor 260 according to an exemplary embodiment may include a data value 0 as the lower limit A of the current frame and a data value 4095 as the upper limit B of the current frame, for the data whose dynamic range extended to 12 bits is converted. The data values between the intervals (0, 4095) correspond to the pixel values between the lower limit value A and the upper limit value B, thereby restoring the extended dynamic range to the original dynamic range based on the extended bit depth. In addition, the bit depth internally extended to 12 bits may also be restored to 8 bits. FIG. 5 illustrates a graph of a probability density function (pdf) of pixel values of original data according to an embodiment.

The pixel value of the original frame is shown on the horizontal axis of the graph, and the probability density of the pixel value of the original frame is shown on the vertical axis. In general, the probability density for pixel values below the lower limit and larger than the upper limit of the pixel value of the original frame is extremely small. In consideration of this, the intra prediction unit 110 and the inter prediction unit 120 set a threshold value for a predetermined probability density and set a smaller value among pixel values of an original frame corresponding to the threshold value. As a lower limit, a large value can be set as an upper limit.

The intra predictor 110 and the inter predictor 120 cut a pixel value smaller than the lower limit and an upper limit of the original frame and extend the dynamic range of pixel values between the remaining lower limit and the upper limit. Using one data, intra prediction and inter prediction on a current frame may be performed. In determining the dynamic range of the original frame, a value larger than the upper limit or smaller than the lower limit may be truncated to reduce the difference between the upper limit and the lower limit.

In this case, the image reconstruction unit 260 according to the embodiment corresponds to the lower limit A of the current frame and the maximum limit of the current frame to the upper limit B of the current frame. The data value between the value and the maximum value corresponds to the pixel value between the lower limit A and the upper limit B, thereby restoring the extended dynamic range to the original dynamic range. However, a pixel value smaller than the lower limit or larger than the upper limit among the pixel values of the original frame cut in the dynamic range extension process cannot be restored.

Therefore, the smaller the lower limit value or the larger the upper limit value, the less the dynamic range is extended, but less pixel values of the original frame may be lost during the restoration of the dynamic range. On the contrary, the larger the lower limit value or the smaller the upper limit value, the more the dynamic range is extended to be advantageous for detailed image processing, but the loss rate of the pixel value of the original frame may be large during the restoration of the dynamic range. Accordingly, the video encoding apparatus 100 using the dynamic range transformation based on the content according to an embodiment may adjust the cutting rate of the pixel value of the original frame by variably setting the lower limit and the upper limit as necessary, thereby extending the dynamic range. You can also adjust the resiliency of the original and original pixels.

Hereinafter, in order for the intra prediction unit 110 and the inter prediction unit 120 according to an embodiment to use for intra prediction and inter prediction, a method of converting a dynamic range of pixel values of an original frame is included in a relational expression. Will be detailed accordingly.

In general, the video format of a digital video capture device uses a fixed bit depth. Currently, the most widely used video format is allocated 8 bits per pixel for each color. In the N-bit bit depth video format, the pixel value is distributed in the interval of [0, 2 ^N -1], that is, from 0 to 2 ^N -1, so that the pixel value interval of the 8-bit bit depth video sequence is [0, 2 ⁸ -1]. In general, the pixel value of a video sequence is more likely to be distributed near the center of the pixel value interval, and the probability of being distributed near the outer edge of the interval is rare.

That is, the distribution of pixel values of the original video sequence may be based on the following equation (1).

0 ≤ OrgMin ≤ Org ≤ OrgMax ≤ 2 ^N -1 ... (1)

Since the original pixel value Org, the minimum of the original pixel values OrgMin, and the maximum value OrgMax are greater than or equal to 0 and less than or equal to 2 ^N -1, the original pixel value is a range of pixel values narrower than the actual maximum possible range 2 ^N -1 Can be distributed in

According to the dynamic range conversion according to an embodiment, the minimum value OrgMin and the maximum value OrgMax of the original pixel value are respectively set as the lower limit and the upper limit based on the content, and the upper limit OrgMax is the difference between the upper limit and the lower limit Range (= OrgMin-OrgMax). Can be replaced by OrgMin + Range. In this case, the dynamic range conversion method based on the content according to an embodiment may be according to the following conversion equations (2), (3) and (4).

if (Org <OrgMin) OrgTransformed = 0; ...(2)

else if (Org> (OrgMin + Range)) OrgTransformed = ^2N -1; ... (3)

else OrgTransformed = (Org-OrgMin) * ( ^2N -1) / Range; ...(4)

When the original pixel value Org is smaller than the lower limit OrgMin according to the conversion equation (2), the data OrgTransformed converted by the dynamic range transformation is determined to be 0, and the original pixel value Org is smaller than the upper limit OrgMin + Range according to the conversion equation (3). The transformed data OrgTransformed is determined as 2 ^N -1, and when the original pixel value Org is lower than the lower limit OrgMin or higher than the upper limit OrgMin + Range according to the conversion equation (4), the transformed data OrgTransformed is (Org-OrgMin) with respect to the original pixel value Org. ) * (2 ^N -1) / Range can be one-to-one correspondence.

The dynamic range conversion method according to the conversion equations (2), (3), and (4) is based on the linear function of the slope (2 ^N -1) / Range when the original pixel value Org is lower limit OrgMin or higher and upper limit OrgMin + Range. The original pixel value Org corresponds to the transformed data OrgTransformed.

The conversion equations (2), (3), and (4) are merely examples of the conversion equation for the dynamic range conversion based on the content according to an embodiment, and the dynamic range conversion method of the present invention is not limited to the conversion equation based on the linear function. . For example, if the original pixel value Org is less than the lower limit OrgMin, it is converted to the minimum value of the extended dynamic range. If the original pixel value Org is greater than the upper limit OrgMin + Range, the condition is converted to the maximum value of the extended dynamic range. Is preferably maintained. If the original pixel value Org is lower than the lower limit OrgMin or more than the upper limit OrgMin + Range, if the dynamic range can be transformed so that the corresponding transformed data OrgTransformed monotonically increases as the original pixel value Org increases, a transform equation can be variably determined as necessary. Can be. However, for convenience of description, a dynamic range conversion method based on content according to an embodiment will be described below based on exemplary conversion equations (2), (3), and (4).

In the dynamic range conversion based on content according to an embodiment, a division operation may be performed using bit shift to improve a calculation speed. In order to represent the conversion equation using bit shift, the offset OFFSET may be set to 1 << (CADR_BITS -1). Bit depth CADR_BITS for dynamic range transformation based on content may be determined for each data unit of an image sequence of an input video, a frame, a frame set for intra prediction, an area, and a coding unit. For convenience of explanation, the following description will be made based on the embodiment in which the bit depth CADR_BITS is eight. In addition, the constant s = (((2 ^N -1) << BITS) + (Range >> 1)) / Range can be set. Accordingly, the conversion equation (4) is changed to the conversion equation (5) as follows, whereby the conversion equation using the bit shift can be determined.

if (Org <OrgMin) OrgTransformed = 0; ...(2)

else if (Org> (OrgMin + Range)) OrgTransformed = ^2N -1; ... (3)

else OrgTransformed = (s * (Org- OrgMin) + OFFSET) >> CADR_BITS ... (5)

As a result of the dynamic range transformation equations (2), (3), and (4) (or (5)) based on the content, the interval of the transformed data OrgTransformed may be transformed into 0 ≦ OrgTransformed ≦ ^2N −1. That is, the section [OrgMin, OrgMax] of the original pixel value can be extended to the section [0, 2 ^N -1] of the transformed data OrgTransformed.

The intra prediction unit 110 and the inter prediction unit 120 of the video encoding apparatus 100 using the dynamic range transformation based on the content may perform intra prediction and inter prediction using the transformed data OrgTransformed. Can be. Compared to the distribution of residuals generated by intra prediction and inter prediction using the original pixel value Org, when the transformed data OrgTransformed is used, the generated residuals are wider, and thus the accuracy of internal calculations may be improved.

Hereinafter, a method of reconstructing the dynamic range of the data restored by decoding by the image reconstructor 260 according to an embodiment to the dynamic range of the original frame will be described in detail according to a relational expression.

By inversely transforming the dynamic range of the data reconstructed through the decoding process, the dynamic range of the original frame may be restored. That is, by applying the dynamic range inverse transform based on the content according to the following Equation (6) to the data Rec restored through decoding, the data RecInvTransformed having the inverse transformed dynamic range may be output.

Rec Inv Transformed = Rec * Range / (2 ^N -1) + OrgMin; ... (6)

The inverse transform equation (6) of the dynamic range based on the content according to an embodiment may also be changed to a transform equation using bit shift as shown in the transform equation (7) below.

RecInvTransformed = (si * Rec + (OrgMin << CADR_BITS) + OFFSET) >> CADR_BITS ... (7)

In this case, the constant si may be set to si = ((Range << CADR_BITS) + ((2 ^N -1) >> 1)) / (2 ^N -1).

When the current sequence follows the YUV color format, the dynamic range transformation based on the content according to the embodiment may be performed on the luma component and the chroma component, respectively. If the dynamic range transformation and inverse transformation described above with reference to the relations (1) to (7) are applied to the Y component pixel value that is the luma component of the pixel value, the chroma is described with reference to relations (8) to (23) below. Dynamic range transformations and inverse transformations based on content for component U component pixel values and V component pixel values are described.

In relations (8) to (17) below, Org_U and Org_V represent original U component pixel values and original V component pixel values, respectively. OrgMin_U and OrgMax_U represent the minimum and maximum values of the original U component pixel values, respectively, and OrgMin_V and OrgMax_V represent the minimum and maximum values of the original V component pixel values, respectively. OrgMin_U and OrgMin_V may be different, and OrgMax_U and OrgMax_V may also be different. For convenience of explanation, in the following relations (8) to (17), OrgMin_U and OrgMax_U are assumed to be the lower and upper limits of the U component, and OrgMin_V and OrgMax_V, respectively, as the lower and upper limits of the V component pixel values.

In addition, iRange_U represents the difference (OrgMax_U-OrgMin_U) between the upper limit OrgMax_U and the lower limit OrgMin_U of the U component pixel value, and iRange_V represents the difference (OrgMax_V-OrgMin_V) between the upper limit OrgMax_V and the lower limit OrgMin_V. The dynamic range conversion method based on the content of the pixel value of the chroma U component according to an embodiment may be according to the following conversion equations (8), (9), and (10).

if (Org_U <OrgMin_U) OrgTransformed_U = 0; ...(8)

else if (Org_U> (OrgMin_U + iRange_U)) OrgTransformed_U = 2 ^N -1; ... (9)

else OrgTransformed_U = (Org_U-OrgMin_U) * (2 ^N -1) / iRange_U; ... (10)

The dynamic range conversion method based on the content of the pixel value of the chroma V component may be according to the following conversion equations (11), (12) and (13).

if (Org_V <OrgMin_V) OrgTransformed_V = 0; ... (11)

else if (Org_V> (OrgMin_V + iRange_V)) OrgTransformed_V = 2 ^N -1; ... (12)

else OrgTransformed_V = (Org_V-OrgMin_V) * ( ^2N -1) / iRange_V; ... (13)

In order to improve the computation speed of the dynamic range transformation based on the content and reduce the complexity of the computation, the division of the relations (8) to (10) and (11) to (13) may be performed through bit shift operations. For example, the constants s_U and s_V for the U component and the V component may be predefined according to relations (14) and (15), respectively.

s_U = (((2 ^N -1) << CADR_BITS) + (iRange_U >> 1)) / iRange_U; ... (14)

s_V = (((2 ^N -1) << CADR_BITS) + (iRange_V >> 1)) / iRange_V; ... (15)

The dynamic lane conversion equations (8), (9), and (10) based on the content of the pixel value of the chroma U component may be transformed into the following equation (16) by using the constant s_U and bit operation.

if (Org_U <OrgMin_U) OrgTransformed_U = 0; ...(8)

else if (Org_U> (OrgMin_U + iRange_U)) OrgTransformed_U = 2 ^N -1; ... (9)

else OrgTransformed_U = (s_U * (Org_U -OrgMin_U) + OFFSET) >> CADR_BITS ... (16)

Similarly, (13) of the dynamic range transformation equations (11), (12), and (13) based on the content of the pixel value of the chroma V component can be transformed into the following equation (17) using the constant s_V and the bitwise operation. have.

if (Org_V < OrgMin_V) OrgTransformed_V = 0; ... (11)

else if (Org_V> (OrgMin_V + iRange_V)) OrgTransformed_V = 2 ^N -1; ... (12)

else OrgTransformed_V = (s_V * (Org_V -OrgMin_V) + OFFSET) >> CADR_BITS ... (17)

Like the dynamic range transformation based on the content, the inverse transformation process may be performed for each chroma component. If the dynamic range inverse transforms (6) and (7) based on the above-described content were inverse transforms for the luma component, then the dynamic range inverse transforms based on the content for chroma U and Y components are expressed in relations (18) to (23). It is described with reference to.

Through the relations (18) and (19), the U component pixel value Rec_U and the V component pixel value Rec_V of the decoded and restored data are input, and the U component pixel value RecInvTransformed_U and the inverse transformed V component pixel value whose dynamic range is inversely transformed. RecInvTransformed_V may be output.

RecInvTransformed_U = Rec_U * iRange_U / (2 ^N -1) + OrgMin_U; ... (18)

RecInvTransformed_V = Rec_V * iRange_V / (2 ^N -1) + OrgMin_V; ... (19)

The inverse transform equations (18) and (19) of the dynamic range based on the content according to an embodiment may also be changed to the fast equations (22) and (23) using bit shift, respectively. First, the constants si_U and si_V for inverse transformation of the U component and the V component may be previously defined according to the relations (20) and (21), respectively.

si_U = ((iRange_U << CADR_BITS) + ((2 ^N -1) >> 1)) / (2 ^N -1); ... (20)

si_V = ((iRange_V << CADR_BITS) + ((2 ^N -1) >> 1)) / (2 ^N -1); ... (21)

Using the constants si_U and si_V, the dynamic lane inverse equations (18) and (19) based on the content for the pixel values of the chroma U component and the V component can be defined by the following relations (22) and (23), respectively.

RecInvTransformed_U = (si_U * Rec_U + (OrgMin_U << CADR_BITS) + OFFSET) >> CADR_BITS ... (22)

RecInvTransformed_V = (si_V * Rec_V + (OrgMin_V << CADR_BITS) + OFFSET) >> CADR_BITS ... (23)

Similar to dynamic range and inverse transforms based on content for luma components, the ranges iRange_U and iRange_V between the upper and lower limits of pixel values of chroma components are the video sequence, frames, framesets for intra prediction, regions, and coding units of the input video. It may be determined for each data unit.

Even in dynamic range conversion based on content for chroma components, a cutting process may be additionally performed to improve accuracy. Since most pixel values of the chroma component are concentrated in a much narrower band than the luma component, the dynamic range can be effectively extended according to the dynamic range transformation based on the content according to an embodiment, thereby achieving accurate prediction encoding. Can be.

As described above with reference to FIG. 4, the video encoding apparatus 100 and the video decoding apparatus 200 according to an embodiment may provide a dynamic range based on content for data whose bit depth of a pixel value is internally extended. Various processing can be performed using the converted data value. In this case, the intra predictor 110 and the inter predictor 120 according to an embodiment may intra-prediction and inter-prediction a current frame using data having an extended dynamic range based on the extended bit depth and content. Can be performed.

In addition, the image reconstructor 260 according to an embodiment may inversely convert the decoded data dynamic range to restore data having an internally extended bit depth, and inversely convert the extended bit depth data to perform data conversion of the original dynamic range. Can be restored. Alternatively, video data of the original dynamic range may be reconstructed by simultaneously inversely converting the dynamic range of the decoded data and the internally extended bit depth.

Hereinafter, the transform equation and the inverse transform equation of the dynamic range based on the content considering the internally expanded bit depth are described in detail.

According to an embodiment, the process of converting the dynamic range based on the content considering the internally extended bit depth may include the process of internally extending the bit depth and converting the dynamic range based on the content. In this case, the total bit depth Nbits after the conversion may be determined as the sum of the BitDepth and the extended bit increase BitIncrement for the content-based dynamic range conversion according to the relation (24).

Nbits = BitDepth + BitIncrement ... (24)

First, the data having the bit depth extended internally by applying the extended bit depth to the original pixel value Org may be output according to the relation (25).

Org = Org << BitIncrement ... (25)

Since the result Org according to relation (25) is internally extended bit depth data, both the upper limit value OrgMax and the lower limit value OrgMin may be determined in the range of the data of the internally extended bit depth.

As a result of extending the dynamic range of the pixel value in which the bit depth is internally extended, data may be determined according to Equation (26) below.

OrgTranformed = (Org-OrgMin) * (2 ^{BitDepth + BitIncrement} -1) / (OrgMax-OrgMin)

Org (Org-OrgMin) * 2 ^BitIncrement * 2 ^BitDepth / (OrgMax-OrgMin + 1) ... (26)

The constant RangeFast is defined according to the relation (27) below.

RangeFast = OrgMax-OrgMin + 1 ... (27)

According to an embodiment, the relation (26) of the relations (25) and (26) for transforming the dynamic range based on the content considering the internally expanded bit depth may be changed to the relation (28) using the bit operation.

Org = Org << BitIncrement ... (25)

OrgTranformed = (((Org-OrgMin) << BitIncrement) << BitDepth) / RangeFast ... (28)

As the internal extension step of the bit depth according to the relation (25) and the dynamic range transformation step based on the content according to the relation (26) or (28) are performed, the content considering the internally extended bit depth according to an embodiment Based on the dynamic range conversion may be performed.

According to an exemplary embodiment, an inverse transform process for restoring a dynamic range and an internally extended bit depth with respect to data whose dynamic range is converted based on content in consideration of an internally extended bit depth may include inverse transform (restore) of the dynamic range. ) And internally extended bit depth recovery.

First, the inverse transformation process of the dynamic range may be according to relation (29).

RecInvTransformed = ((Rec * RangeFast + CADR_OFFSET) >> BitDepth) + (OrgMin << BitIncrement); ... (29)

Here, the constant CADR_OFFSET may be defined according to relation (30).

CADR_OFFSET = (1 << (BitDepth -1)) ... (30)

The reconstructed dynamic range data RecInvTransformed output according to relation (29) is data of an internally extended bit depth. Accordingly, the bit depth may be restored to the original bit depth according to the relation (31).

Rec = (RecInvTransformed + (1 << (BitIncrement-1))) >> BitIncrement ... (31)

As described above, the output data Rec according to relation (31) may be video data of the restored dynamic range.

In the case where the bit depth is not extended internally, the variable BitIncrement is set to 0 in the above-described relations (24) to (31), and the relational expression (25) and the relational expression (31) during the inverse transformation may be omitted. have.

According to dynamic range transformation and inverse transformation based on content considering internally extended bit depths according to relations (24) to (31), loss of original data may occur, but when the advantage of high-speed operation versus loss is needed more Can be useful.

As described above, the conversion equations (25) to (28) and the inverse conversion equations (29) to (31) of the dynamic range based on the content considering the internally expanded bit depth according to an embodiment are the original pixel value Org of the luma component and the reconstructed pixel. Exemplary relations for the value Rec. Dynamic range transformation and inverse transformation based on content considering an internally extended bit depth according to an embodiment for video data of a chroma component may be performed.

In dynamic range conversion based on content considering internally extended bit depth for chroma components, most of the data of the chroma components are concentrated in a narrow band near the offset CADR_OFFSET (= 1 < <(BitDepth-1)). Accordingly, according to an embodiment, an exemplary relation for dynamic range transformation based on content considering internally extended bit depths for chroma components U and V components may include relations (32) for internal expansion of bit depths, and (33) and relations (34) and (35) for dynamic range transformation based on content.

Org_U = Org_U << BitIncrement ... (32)

Org_V = Org_V << BitIncrement ... (33)

OrgTranformed_U = ((Org_U + CADR_OFFSET) << BitDepth) / RangeFast + CADR_OFFSET ... (34)

OrgTranformed_V = ((Org_V + CADR_OFFSET) << BitDepth) / RangeFast + CADR_OFFSET ... (35)

The constant RangeFast in the relations (34) and (35) can be used as the constant RangeFast of the relations (27) and (28) during the dynamic range conversion for the luma component. Because of this, additional overhead for chroma components can be minimized.

An exemplary relationship for the inverse transform process for reconstructing the dynamic range and the internally extended bit depth for the data of the chroma component U and V components according to one embodiment according to relations (32) to (35), Based on equations (36) and (37) for dynamic range inverse transform, and relations (38) and (39) for reconstruction of an internally extended bit depth.

RecInvTransformed_U = (((Rec_U-(CADR_OFFSET << BitIncrement)) * RangeFast) >> BitDepth) + (CADR_OFFSET << BitIncrement) ... (36)

RecInvTransformed_V = (((Rec_V-(CADR_OFFSET << BitIncrement)) * RangeFast) >> BitDepth) + (CADR_OFFSET << BitIncrement) ... (37)

Rec_U = (RecInvTransformed_U + (1 << (Bit_Increment-1))) >> Bit_Increment ... (38)

Rec_V = (RecInvTransformed_V + (1 << (Bit_Increment-1))) >> Bit_Increment ... (39)

Therefore, for chroma components concentrated in a narrow band, the bit depth may be extended internally, and the dynamic range may be maximized based on the content to accurately predict the chroma component data during video encoding. In addition, by using the data predicted based on the maximized dynamic range in video decoding, more accurate reconstruction of the video data is possible.

FIG. 23 illustrates a graph of data with dynamic range transformed based on a non-linear function, according to one embodiment. FIG.

As described above, since the dynamic range transformation method of the present invention is not limited to the mapping relationship based on the linear function, a nonlinear function may be used. However, when the nonlinear function is used, the computational burden may be greater and the rate-distortion cost may be worse than the accuracy is improved.

In addition, a mapping relationship based on a piece-wise linear function, which is a linear function for each section and the sections have continuous characteristics, illustrated in FIG. 23 may be used. The horizontal axis of the interval nonlinear function of FIG. 23 represents a current pixel value with respect to the maximum pixel value of the input pixel, and the vertical axis represents a current value with respect to the maximum value of the converted data. If the interval nonlinear function approximates a cumulative distributioin function of the original pixel value, the dynamic range transformation based on this may be more effective. In addition, since each slope is constant for each section, the rate-distortion cost for each section can be determined, and the burden of computation amount can be reduced with the improvement of accuracy.

6 is a flowchart of a video encoding method using dynamic range transformation based on content, according to an embodiment of the present invention.

In step 61, frame-by-frame intra prediction and inter prediction of the input video sequence are performed. In this case, intra prediction and inter prediction on the current frame are performed using image data obtained by converting the dynamic range of the current frame based on the content of the input video.

Intra-prediction and inter-prediction of the current frame may be performed based on the image data of the dynamic range transformed so that the pixel values of the current frame may be extended, ranging from the maximum value to the minimum value that can be expressed as the current bit depth of the pixel value. Can be. For example, a value larger than an upper limit among pixel values of a current frame is converted to correspond to a maximum value, a value smaller than a lower limit among pixel values of a current frame is converted to correspond to a minimum value and a pixel of the current frame Pixel values between the upper limit value and the lower limit value among the values may be converted to have a one-to-one correspondence with the value between the maximum limit value and the minimum limit value.

Pixel values greater than the upper limit or smaller than the lower limit among the current pixel values may be truncated. The upper limit value and the lower limit value may be determined for each data unit of an image sequence of an input video, a frame, a frame set for intra prediction, an area, and a coding unit.

The current bit depth may be an internally extended bit depth during the video decoding process.

In step 63, a frequency transform is performed on the residual data generated by intra prediction and inter prediction, and quantization is performed on the transform coefficients generated by the frequency transform.

In step 65, entropy encoding is performed on the quantized transform coefficients, and a bitstream including the encoded data for the current frame is output. In addition, information on the transformation of the dynamic range may be encoded and transmitted. For example, the information on the dynamic range transformation may include information about an upper limit value and a lower limit value of the content of the input video. In addition, the number of bits can be reduced by expressing information about a value obtained by subtracting the upper limit from the difference between the maximum limit value and the minimum limit value of the pixel value instead of the upper limit information.

7 is a flowchart of a video decoding method using dynamic range transformation based on content according to an embodiment of the present invention.

In step 71, the received bitstream is parsed and the encoded data of the current frame of the original video is extracted from the bitstream. For inverse transformation of the original frame into the dynamic range, information about an upper limit value and a lower limit value may be received along with a bitstream for the image data.

In step 73, entropy decoding is performed on the extracted encoded data to recover the data symbols.

In step 75, inverse quantization and inverse frequency transform on the recovered data symbols are performed to restore the residual data for the current frame. Intra prediction and motion compensation are performed on the reconstructed residual data to reconstruct image data.

In step 77, deblocking filtering is performed on the reconstructed image data, and the current frame is reconstructed by reconstructing the converted dynamic range based on the content of the original video. For example, the dynamic range of the reconstructed pixel value extended in the range of the maximum value to the minimum value that can be expressed by the current bit depth of the pixel value may be restored to the dynamic range of the current frame.

In order to restore the dynamic range of the reconstruction frame to the dynamic range of the current frame, the maximum and minimum values of the reconstructed image data may be reconstructed so as to correspond to the upper and lower limits of the pixel values of the current frame, respectively. Values between the maximum value and the minimum value in the image data may be converted to correspond one-to-one to values between the upper limit value and the lower limit value.

The bit depth of the reconstructed dynamic range may be an internally extended bit depth during the video decoding process. In this case, the bit depth may be restored to the bit depth of the original video.

The upper limit value and the lower limit value of the dynamic range of the original frame may be determined for each data unit of an image sequence, a frame, a frame set for intra prediction, an area, and a coding unit of the original video.

According to an embodiment of the present invention, there is provided a video encoding method using a dynamic lane transformation based on content and a device thereof, or a video decoding method using a dynamic lane transformation based on a content and an apparatus according to an embodiment. Since the image processing such as intra prediction and inter prediction can be performed more precisely by extending the dynamic range of, image quality due to encoding and decoding can be improved.

Hereinafter, various embodiments of video encoding and decoding using dynamic range transformation based on content based on hierarchical data units for respective regions according to an embodiment of the present invention will be described with reference to FIGS. 8 to 22.

8 is a block diagram of a video encoding apparatus using a dynamic range transformation based on content, based on a hierarchical data unit for each region according to an embodiment of the present invention.

The video encoding apparatus 80 according to an embodiment includes a maximum coding unit splitter 81, a dynamic range transformed image encoder 83, and a bitstream output unit 85.

For convenience of description below, the video encoding apparatus 80 using the dynamic range transformation based on the content based on the hierarchical data unit for each region according to an embodiment is abbreviated as the video encoding apparatus 80 according to the embodiment. Refer to.

The maximum coding unit splitter 81 according to an embodiment may partition the current picture based on the maximum coding unit that is a coding unit of the maximum size for the current picture of the image. If the current picture is larger than the maximum coding unit, image data of the current picture may be split into at least one maximum coding unit. The maximum depth and the maximum size of the coding unit that limit the total number of times of hierarchically dividing the height and the width of the maximum coding unit may be preset. The image data may be output to the image encoder 83 which is dynamic range converted for each of at least one maximum coding unit.

The dynamic range transformed image encoder 83 according to an embodiment performs encoding on a hierarchical coding unit for each region, for each maximum coding unit obtained by dividing the current picture into coding units having a predetermined maximum size. The encoding for each hierarchical coding unit for each region according to an embodiment refers to encoding for deeper coding units that are hierarchically divided according to a deeper depth, for each region in which a maximum coding unit is divided as a depth deepens. .

The coding unit according to an embodiment may be characterized by a maximum size and depth. The depth is an index according to the number of times the coding unit is hierarchically divided from the maximum coding unit, and as the depth increases, the coding units according to depths may be split from the maximum coding unit to the minimum coding unit. The depth of the largest coding unit is the highest depth and the minimum coding unit may be defined as the lowest coding unit. As the maximum coding unit decreases as the depth increases, the size of the coding unit for each depth decreases, and thus, the coding unit of the higher depth may include coding units of a plurality of lower depths.

The maximum depth according to an embodiment is an index related to the number of divisions from the maximum coding unit to the minimum coding unit. The first maximum depth according to an embodiment may represent the total number of divisions from the maximum coding unit to the minimum coding unit. The second maximum depth according to an embodiment may represent the total number of depth levels from the maximum coding unit to the minimum coding unit. For example, when the depth of the largest coding unit is 0, the depth of the coding unit obtained by dividing the largest coding unit once may be set to the depth of the coding unit divided once or twice. In this case, if the coding unit divided four times from the maximum coding unit is the minimum coding unit, since depth levels of 0, 1, 2, 3, and 4 exist, the maximum depth according to an embodiment is 4, according to another embodiment. The maximum depth may be set to five.

As described above, the image data of the current picture may be divided into maximum coding units according to the maximum size of the coding unit, and each maximum coding unit may include coding units divided by depths. Since the maximum coding unit is divided according to depths, image data of a spatial domain included in the maximum coding unit may be hierarchically classified according to depths. Technical features of the coding units according to depths are described below with reference to FIGS. 10 to 19.

The dynamic range transformed image encoder 83 according to an embodiment may perform intra prediction, inter prediction, frequency transformation, and quantization based on at least one depth-based coding unit of the largest coding unit.

The dynamic range-converted image encoder 83 according to an embodiment may perform intra prediction and inter prediction on the current frame using image data obtained by converting the dynamic range of the current frame based on content. Here, the transformation of the dynamic range based on the content corresponds to the above-described transformation of the dynamic range through the video encoder 100 according to an embodiment.

According to an embodiment, the dynamic range may be converted so that the pixel values of the current picture are extended to a range of a minimum value to a maximum value that can be expressed as the current bit depth of the pixel value. Accordingly, the dynamic range-converted image encoder 83 according to an embodiment may encode the current picture by using the image data of which the dynamic range is converted into a range of minimum to maximum pixel values. .

Therefore, intra prediction and inter prediction for the current frame may be performed using image data obtained by converting the dynamic range of the current picture. In this case, the dynamic range of the current picture is converted so that a value larger than an upper limit among pixel values of the current picture corresponds to a maximum value, a value smaller than a lower limit is converted to correspond to a minimum value, and between an upper limit value and a lower limit value. The pixel values may be converted to have a one-to-one correspondence with a value between the maximum value and the minimum value.

The upper limit value and the lower limit value of the pixel values of the current picture may be determined as the maximum and minimum values of the pixel values of the current picture, respectively. Alternatively, when the upper limit value and the lower limit value of the pixel value of the current picture are set separately, a result in which the current pixel value larger than the upper limit value or smaller than the lower limit value of the current picture may be cut according to the dynamic range conversion according to an exemplary embodiment.

According to an embodiment, an upper limit value and a lower limit value may be determined for each data unit of a picture sequence, a frame, a frame set for intra prediction, an area, and a coding unit of an input video.

The dynamic range-converted image encoder 85 according to an embodiment performs intra prediction and inter prediction on previous pictures whose dynamic range is converted based on the content of the input video, and performs residual data on the previous picture. You can print The residual data for the previous picture may be reconstructed as a reconstructed picture for the previous picture through frequency transform, quantization, inverse quantization, inverse frequency transform, and deblocking filtering.

According to an embodiment, when the dynamic range is converted from picture to picture, the dynamic range of the reconstructed pictures for the previous pictures is preferably reconstructed based on the dynamic range of the current picture once reconstructed to the original dynamic range. In this way, the current picture inter prediction may be performed with reference to a reconstructed picture generated to be based on the same dynamic range as the current picture.

The dynamic range transformed image encoder 83 according to an embodiment may perform intra prediction on the current picture by using the peripheral region information of the current picture having the converted dynamic range.

The video encoding apparatus 80 according to an embodiment may extend and use a bit depth of an image pixel value internally in the decoding process of the input video. Accordingly, the dynamic range of the image data according to the internally extended bit depth may be converted, and intra prediction and inter prediction may be performed on the current picture using the converted dynamic range image data.

The dynamic range transformed image encoder 83 according to an embodiment performs frequency transform on residual data generated by intra prediction and inter prediction, and quantizes transform coefficients generated by frequency transform. do.

When the video encoding apparatus 80 according to an embodiment is based on a video encoding method based on hierarchical data units for each region according to an embodiment, the dynamic range transformed image encoder 83 according to an embodiment may include: A hierarchical coding unit for each region is encoded to determine a depth at which the final encoding result is output. That is, the dynamic range-converted image encoder 83 according to an embodiment encodes the image data in coding units according to depths for each maximum coding unit of the current picture, and selects a depth at which the smallest coding error occurs to determine the coding depth. Can be. For each of at least one coded depth, an encoding mode for a corresponding coding unit may be determined. The determined coded depth and the image data for each maximum coding unit are output to the bitstream output unit 85.

Image data in the largest coding unit is encoded based on coding units according to depths according to at least one depth less than or equal to the maximum depth, and encoding results based on the coding units for each depth are compared. As a result of comparing the encoding error of the coding units according to depths, a depth having the smallest encoding error may be selected. At least one coding depth may be determined for each maximum coding unit.

As the depth of the maximum coding unit increases, the coding unit is divided into hierarchically and the number of coding units increases. In addition, even in the case of coding units having the same depth included in one largest coding unit, a coding error of each data is measured, and whether or not division into a lower depth is determined. Therefore, even in the data included in one largest coding unit, since the encoding error for each depth is different according to the position, the coding depth may be differently determined according to the position. Accordingly, one or more coding depths may be set for one maximum coding unit, and data of the maximum coding unit may be partitioned according to coding units of one or more coding depths.

Predictive coding and frequency transformation of the largest coding unit may be performed. Similarly, the prediction encoding and the frequency transformation are performed based on depth-wise coding units for each maximum coding unit and for each depth below the maximum depth.

Since the number of coding units for each depth increases each time the maximum coding unit is divided for each depth, encoding including prediction coding and frequency transform should be performed on all the coding units for each depth generated as the depth deepens. For convenience of explanation, the prediction encoding and the frequency transformation will be described based on the coding unit of the current depth among at least one maximum coding unit.

The video encoding apparatus 80 according to an embodiment may variously select a size or shape of a data unit for encoding image data. In order to encode the image data, a process of predictive encoding, frequency transformation, entropy encoding, and the like is performed. The same data unit may be used in all stages, or the data unit may be changed in stages.

For example, the video encoding apparatus 80 may select not only a coding unit for encoding the image data, but also a data unit different from the coding unit in order to perform predictive encoding of the image data in the coding unit.

For prediction encoding of the largest coding unit, prediction encoding may be performed based on a partial data unit of the coding unit according to depths of the maximum coding unit. The partial data unit of the coding unit may include a data unit in which at least one of a height and a width of the coding unit and depth-specific coding units are divided.

For example, when the size of the coding unit is 2Nx2N (where N is a positive integer), the size of the partial data unit may be 2Nx2N, 2NxN, Nx2N, NxN, or the like. The prediction encoding may be performed based on variously divided data units in addition to a data unit having half of at least one of a height or a width of the coding unit. Hereinafter, the data unit on which the prediction encoding is based may be referred to as a 'prediction unit'.

The prediction mode of the coding unit may be at least one of an intra mode, an inter mode, and a skip mode. For example, the intra mode and the inter mode may be performed with respect to prediction units having sizes of 2N × 2N, 2N × N, N × 2N, and N × N. In addition, the skip mode may be performed only for a prediction unit having a size of 2N × 2N. The encoding may be performed independently for each prediction unit within the coding unit to select a prediction mode having the smallest encoding error.

Also, the video encoding apparatus 80 according to an embodiment may perform frequency conversion of image data of a coding unit based on not only a coding unit for encoding image data, but also a data unit different from the coding unit.

For frequency transformation of a coding unit, frequency transformation may be performed based on a data unit having a size smaller than or equal to the coding unit. For example, the data unit for frequency conversion may include a data unit for an intra mode and a data unit for an inter mode. Hereinafter, the data unit on which the frequency conversion is based may be referred to as a 'conversion unit'.

The encoded information for each coded depth requires not only the coded depth but also prediction related information and frequency transform related information. Accordingly, the dynamic range-converted image encoder 83 according to an embodiment not only has a coded depth that generates a minimum coding error, but also a partition type obtained by dividing a coding unit of a coded depth into prediction units, a prediction mode for each prediction unit, and a frequency. The size of a transformation unit for transformation may be determined.

The dynamic range transformed image encoder 83 according to an embodiment may measure encoding errors of coding units according to depths using a Lagrangian Multiplier-based rate-distortion optimization technique. have.

The bitstream output unit 85 according to an embodiment may output a bitstream including information about a coded depth and an encoding mode and encoded image data. The bitstream output unit 85 according to an embodiment may perform entropy encoding on encoded image data that is a result of encoding according to an encoding depth and an encoding mode determined based on hierarchical data units for each region, and then output the bitstream output unit. The processor 85 may output a bitstream including information about an encoded depth and an encoding mode and encoded image data.

The bitstream output unit 85 according to an embodiment may include information about image data and a coding mode according to depths, which are encoded based on at least one coded depth determined by the dynamic range-converted image encoder 85. Output in the form of a bitstream. The image data encoded by the dynamic range-converted image encoder 85 may be converted into a bitstream form through entropy encoding and then inserted into a bitstream for transmission.

According to an embodiment, the encoded image data may be a result of encoding residual data of an image. The information about the encoding modes according to depths according to an embodiment may include coded depth information, partition type information of a prediction unit of a coding unit of a coding depth, prediction mode information for each prediction unit, and size information of a transformation unit.

The coded depth information may be defined using depth-specific segmentation information indicating whether to encode to a coding unit of a lower depth without encoding to the current depth. If the current depth of the current coding unit is a coding depth, since the current coding unit is encoded in a coding unit of the current depth, split information of the current depth may be defined so that it is no longer divided into lower depths. On the contrary, if the current depth of the current coding unit is not the coding depth, encoding should be attempted using the coding unit of the lower depth, and thus split information of the current depth may be defined to be divided into coding units of the lower depth.

If the current depth is not the coded depth, encoding is performed on the coding unit divided into the coding units of the lower depth. Since at least one coding unit of a lower depth exists in the coding unit of the current depth, encoding may be repeatedly performed for each coding unit of each lower depth, and recursive coding may be performed for each coding unit of the same depth.

Since at least one coding depth is determined in one maximum coding unit and information about at least one coding mode should be determined for each coding depth, information about at least one coding mode may be determined for one maximum coding unit. In addition, since the data of the largest coding unit is divided hierarchically according to the depth, the coding depth may be different for each location, and thus information about the coded depth and the coding mode may be set for the data.

The smallest square data unit among coding units, prediction units, and transformation units set within a predetermined range of video units such as a current frame or a current sequence may be referred to as a 'minimum unit'. The bitstream output unit 85 according to an embodiment may set corresponding encoding information for each minimum unit included in the maximum coding unit. That is, the coding unit of the coded depth includes one or more minimum units having the same coding information. By using this, if the neighboring minimum units have the same depth-specific encoding information, it may be the minimum unit included in the same maximum coding unit.

For example, the encoding information output through the bitstream output unit 85 according to an embodiment may be classified into encoding information according to depth coding units and encoding information according to prediction units. The encoding information for each coding unit according to depth may include prediction mode information and partition size information. The encoding information transmitted for each prediction unit includes information about an estimation direction of the inter mode, information about a reference image index of the inter mode, information about a motion vector, information about a chroma component of an intra mode, and information about an inter mode of an intra mode. And the like. In addition, information about a maximum size and information about a maximum depth of a coding unit defined for each picture, slice, or GOP may be inserted in a header of a bitstream.

The bitstream output unit 85 according to an embodiment may encode and output filter coefficients used for adaptive loop filtering. In addition, for adaptive loop filtering according to an embodiment, the type, number, size, quantization bit, coefficient, filtering direction, whether to perform filtering and whether to perform running filtering, etc. may be set. Information about the one-dimensional filter set of filtering may be encoded and transmitted.

According to an embodiment of the simplest form of the video encoding apparatus 80 according to an embodiment, a coding unit for each depth is a coding unit having a size that is half the height and width of the coding unit of a higher layer depth. That is, if the size of the coding unit of the current depth is 2Nx2N, the size of the coding unit of the lower depth is NxN. In addition, the current coding unit having a size of 2N × 2N may include up to four lower depth coding units having a size of N × N.

In addition, the bitstream output unit 85 according to an embodiment may encode information about the dynamic range for dynamic range restoration and insert the encoded information about the dynamic range for insertion into the transmission bitstream. The bitstream output unit 85 according to an embodiment may receive additional information about the dynamic range transformation from the dynamic range transformed image encoder 85, encode the same, and insert the additional information into the bitstream.

For example, the information on the dynamic range transformation may be inserted into a sequence parameter set or a slice header of a bitstream including encoded data of the current picture. The information on the dynamic range transformation according to an embodiment may include information about an upper limit value and a lower limit value of the content of the input video. In order to save the number of bits, instead of the information on the upper limit, a value obtained by subtracting the upper limit from the difference between the maximum value and the minimum value of the pixel value may be encoded.

Accordingly, the video encoding apparatus 80 according to an embodiment determines a coding unit having an optimal shape and size for each maximum coding unit based on the size and the maximum depth of the maximum coding unit determined in consideration of the characteristics of the current picture. Can be. In addition, since each of the maximum coding units may be encoded in various prediction modes, frequency transform schemes, or the like, an optimal coding mode may be determined in consideration of image characteristics of coding units having various image sizes.

Therefore, if an image having a very high resolution or a very large data amount is encoded in an existing macroblock unit, the number of macroblocks per picture is excessively increased. Accordingly, since the compressed information generated for each macroblock increases, the transmission burden of the compressed information increases, and the data compression efficiency tends to decrease. Therefore, the video encoding apparatus according to an embodiment may adjust the coding unit in consideration of the image characteristics while increasing the maximum size of the coding unit in consideration of the size of the image, thereby increasing image compression efficiency.

Further, according to an exemplary embodiment, image processing such as intra prediction and inter prediction may be performed more precisely for each coding unit by using image data having an extended dynamic range based on content, so that image quality due to encoding and decoding may be improved. Can be improved.

FIG. 9 is a block diagram of a video decoding apparatus using content-based dynamic range transformation based on hierarchical hierarchical data units according to an embodiment of the present invention.

Based on the hierarchical data unit for each region according to an embodiment, the video decoding apparatus 90 using the dynamic range transformation based on the content may include a receiver 91, encoded data, encoded information, and dynamic range information extractor 93. ), An image data decoder 95 and a dynamic range reconstructed image reconstructor 97. For convenience, the video decoding apparatus 90 using the dynamic range transformation based on the content based on the hierarchical data unit for each region according to an embodiment may be shortened to the video decoding apparatus 90 according to an embodiment. The encoded data, the encoded information, and the dynamic range information extractor 93 according to the embodiment are abbreviated to the information extractor 93 according to an embodiment, and the dynamic range restored image restorer 97 according to an embodiment. Is abbreviated to an image reconstructor 97 according to an embodiment.

Definitions of various terms such as coding units, depths, prediction units, transformation units, and information about various encoding modes for various processings of the video decoding apparatus 90 according to an embodiment may include video encoding according to FIGS. 8 and 1. Same as described above with reference to device 80.

The receiver 91 according to an embodiment receives and parses a bitstream of an encoded video, and the information extractor 93 according to an embodiment encodes data, encoded information, and the like encoded from the bitstream from the parsed bitstream. Extract the dynamic range information. The extracted encoded data and encoded information are output to the image data decoder 95. The dynamic range information may be output to the image restoration unit 97 according to an embodiment.

The information extractor 93 according to an embodiment may extract information about a maximum size of a coding unit of the current picture from a header for the current picture.

In addition, the information extractor 93 extracts information about a coded depth and an encoding mode for each largest coding unit from the parsed bitstream. The extracted information about the coded depth and the coding mode is output to the image data decoder 95. That is, the image data of the bit string may be divided into maximum coding units so that the image data decoder 95 may decode the image data for each maximum coding unit.

The information about the coded depth and the encoding mode for each largest coding unit may be set for one or more coded depth information, and the information about the coding mode for each coded depth may include partition type information of a prediction unit for each coding unit, prediction mode information, and the like. The size information of the transformation unit may be included. In addition, split information for each depth may be extracted as the coded depth information.

The information about the coded depth and the encoding mode for each maximum coding unit extracted by the information extractor 93 according to an embodiment may be determined according to the depths according to the maximum coding units in the encoding stage, such as the video encoding apparatus 80 according to the embodiment. Information about a coded depth and an encoding mode determined to repeatedly perform encoding for each coding unit to generate a minimum encoding error. Therefore, the video decoding apparatus 90 according to an embodiment may decode the encoded data to generate the minimum encoding error, and reconstruct the original image according to the corresponding encoding scheme.

The information extractor 93 according to an embodiment may extract information about a coded depth and an encoding mode for each minimum unit. For each minimum unit, if information about the coded depth and the encoding mode of the corresponding maximum coding unit is recorded, the minimum units having the information about the same coded depth and the encoding mode may be inferred as data units included in the same maximum coding unit. have. That is, when the minimum unit of the same information is collected and decoded, decoding based on the coding unit of the coded depth having the smallest encoding error is possible.

The image data decoder 95 according to an embodiment decodes image data of each maximum coding unit based on information about a coded depth and an encoding mode for each maximum coding unit. Based on the coded depth information for each largest coding unit, the image data decoder 95 according to an embodiment may decode the image data for each coding unit of at least one coded depth. The decoding process may include a prediction process including intra prediction and motion compensation, and an inverse frequency transform process.

The image data decoder 95 according to an embodiment may perform prediction units for each coding unit on the basis of partition type information and prediction mode information of the prediction unit of the coding unit for each coding depth, for prediction encoding for each coding unit. In the prediction mode, intra prediction or motion compensation may be performed.

In addition, the image data decoder 95 according to an embodiment may perform inverse transformation on each transformation unit for each coding unit based on the size information of the transformation unit of the coding unit for each coding depth, for inverse frequency transformation for each coding unit. Frequency conversion can be performed.

The image data decoder 95 according to an embodiment may determine a coded depth of a current maximum coding unit using split information according to depths. If the split information indicates decoding to the current depth, the current depth is the coded depth. Therefore, the decoder 95 according to an embodiment may decode the coding unit of the current depth with respect to the image data of the current maximum coding unit by using the partition type, the prediction mode, and the transformation unit size information of the prediction unit.

That is, the encoding information set with respect to the minimum unit can be analyzed, and the minimum unit holding encoding information including the same split information can be collected and decoded into one data unit.

Image data decoded based on depth-based coding units by the image data decoder 95 according to an embodiment may be input to the image reconstructor 97 according to an embodiment. The image reconstructor 97 according to an embodiment may reconstruct the current picture by reconstructing the decoded image data for each largest coding unit. In order to improve reconstructed picture quality of the current picture, in-loop filtering such as deblocking filtering and adaptive loop filtering may be performed.

In addition, when image data encoded based on a dynamic lane converted based on content according to an embodiment is decoded, the image reconstructor 97 according to an embodiment may include an information extracting unit ( By using the dynamic range information extracted by 93), the original dynamic range of the current picture can be restored.

The dynamic range information extracted by the information extractor 93 according to an embodiment may include information on the conversion of the dynamic range of the current picture. For example, the received information about the conversion of the dynamic range may include information about an upper limit value and a lower limit value of the content of the original video. Instead of the information on the upper limit among the extracted information, the information on the value obtained by subtracting the upper limit from the difference between the maximum value and the minimum value of the pixel value may be included.

In addition, information on the data unit in which the upper limit value and the lower limit value of the current frame are set may also be received. For example, the upper limit value and the lower limit value of the current frame may be set for each data unit of an image sequence, a picture, a frame, a frame set for intra prediction, an area, and a coding unit of the original video.

The image reconstructor 97 according to an exemplary embodiment may include a reconstructed pixel extended with respect to image data reconstructed by decoding, loop filtering, etc., in the range of the maximum value to the minimum value that can be expressed by the current bit depth of the pixel value. You can restore the value to the dynamic range of the current frame.

The image reconstructor 97 restores the maximum value and the minimum value among the reconstructed pixel values to correspond to the upper limit value and the lower limit value among the pixel values of the current frame, respectively. The values between the values can be converted to have a one-to-one correspondence to the values between the upper and lower limits. One example of the upper limit value and the lower limit value of the current frame may be a local maximum and a local minimum among pixel values of the current frame.

According to an exemplary embodiment, the image reconstructor 97 multiplies a value between a currently reconstructed data value and a 'difference between an upper limit value and a lower limit value' with respect to values between a maximum limit value and a minimum limit value. A value obtained by adding a lower limit to a value divided by a difference of 'can be converted to have a one-to-one correspondence. The image reconstructor 97 according to an embodiment may calculate and output a one-to-one corresponding pixel value with respect to values between the maximum limit value and the minimum limit value, using a bit shift operation using the current bit depth. . According to an embodiment, the current bit depth may be an internally extended bit depth during the video decoding process.

The pixel value of the current frame reconstructed by the image reconstructor 97 according to an exemplary embodiment may be a truncated result value of a pixel value greater than or equal to an upper limit and a pixel value less than or equal to a lower limit of the corresponding original image.

The image data decoder 95 according to an embodiment may perform motion compensation on the current picture by referring to reconstructed pictures of the previous picture of the original video. If information on dynamic range conversion is set on a picture basis, the image data decoder 95 according to an embodiment receives the reconstructed pictures of previous pictures from the image reconstructor 97 according to an embodiment. In addition, by reconverting the dynamic range of the reconstructed pictures based on the converted dynamic range of the current picture, motion compensation of the current picture may be performed with reference to the reconstructed picture based on the same extended dynamic range.

The video decoding apparatus 90 according to an embodiment obtains information on a coding unit that generates a minimum coding error by recursively encoding each maximum coding unit in the encoding process, and uses the same to decode the current picture. Can be performed. That is, the image data can be decoded in an optimal coding unit for each maximum coding unit. In addition, in the case of decoding the encoded image data after the dynamic range has been transformed, the dynamic range of the original picture is reconstructed using the received information on the dynamic range transform, and thus encoding is performed through more detailed intra prediction and inter prediction for each coding unit. The obtained data can be decoded in an optimal coding unit for each maximum coding unit.

Hereinafter, with reference to FIGS. 10 through 20, hierarchical hierarchical data units are described in detail for video encoding and decoding using content-based dynamic range transformation based on hierarchical hierarchical data units according to an embodiment.

10 illustrates a concept of coding units, according to an embodiment of the present invention.

Examples of coding units may include 32x32, 16x16, 8x8, and 4x4, starting from a coding unit having a width x height of 64x64. In addition to the square coding units, there may be coding units having a width x height of 64x32, 32x64, 32x16, 16x32, 16x8, 8x16, 8x4, and 4x8.

As for the video data 310, the resolution is set to 1920x1080, the maximum size of the coding unit is 64, and the maximum depth is 2. As for the video data 320, the resolution is set to 1920x1080, the maximum size of the coding unit is 64, and the maximum depth is 4. For the video data 330, the resolution is set to 352x288, the maximum size of the coding unit is 16, and the maximum depth is 2. The maximum depth illustrated in FIG. 10 represents the total number of divisions from the maximum coding unit to the minimum coding unit.

When the resolution is high or the amount of data is large, it is preferable that the maximum size of the coding size is relatively large not only to improve coding efficiency but also to accurately shape an image characteristic. Accordingly, the video data 310 or 320 having a higher resolution than the video data 330 may be selected to have a maximum size of 64.

The maximum depth represents the total number of layers in the hierarchical coding unit. Therefore, since the maximum depth of the video data 310 is 2, the coding unit 315 of the video data 310 is encoded from the largest coding unit having a long axis size of 64 by two layers of depth and having a long axis size of 32 or 16. It can include up to units. On the other hand, since the maximum depth of the video data 330 is 2, the coding unit 335 of the video data 330 has two long depths of encoding from the coding units having the long axis size of 16, so that the long axis sizes are 8 and 4 It can include up to units.

Since the maximum depth of the video data 320 is 4, the coding unit 325 of the video data 320 has a depth of four layers deeper from the maximum coding unit having a long axis size of 64, so that the long axis size is 32, 16, 8, 4 Up to coding units may be included. As the depth increases, the expressive power of the detailed information may be improved.

11 is a block diagram of an image encoder based on coding units, according to an embodiment of the present invention.

The image encoder 400 according to an embodiment includes operations performed by the dynamic range adjusted image encoder 83 of the video encoding apparatus 80 according to an embodiment to encode image data. That is, the intra predictor 410 performs intra prediction on the coding unit of the intra mode among the current frame 405, and the motion estimator 420 and the motion compensator 425 are the current frame 405 of the inter mode. And the inter frame estimation and motion compensation using the reference frame 495.

Data output from the intra predictor 410, the motion estimator 420, and the motion compensator 425 is output as a quantized transform coefficient through the frequency transformer 430 and the quantizer 440. The quantized transform coefficients are restored to the data of the spatial domain through the inverse quantizer 460 and the inverse frequency converter 470, and the restored data of the spatial domain are deblocked 480 and the loop filter 490. After processing, the data is output to the reference frame 495. The quantized transform coefficients may be output to the bitstream 455 via the entropy encoder 450.

In order to be applied to the video encoding apparatus 80 according to an embodiment, an intra predictor 410, a motion estimator 420, a motion compensator 425, and a frequency converter that are components of the image encoder 400 may be used. 430, quantization unit 440, entropy encoding unit 450, inverse quantization unit 460, inverse frequency transformer 470, deblocking unit 480, and loop filtering unit 490 are all coded at maximum. Work should be performed based on depth-based coding units considering the maximum depth for each unit.

In particular, the intra predictor 410, the motion estimator 420, and the motion compensator 425 determine the prediction unit and the prediction mode in the coding unit in consideration of the maximum size and the depth of the coding unit, and the frequency converter 430. ) Should consider the size of the transform unit in consideration of the maximum size and depth of the coding unit.

12 is a block diagram of an image decoder based on coding units, according to an embodiment of the present invention.

The bitstream 505 is parsed through the parser 510, and the encoded video data to be decoded and information about encoding necessary for decoding are parsed. The encoded video data is output as dequantized data through the entropy decoding unit 520 and the inverse quantization unit 530, and the image data of the spatial domain is reconstructed through the inverse frequency converter 540.

For the image data of the spatial domain, the intra prediction unit 550 performs intra prediction on the coding unit of the intra mode, and the motion compensator 560 uses the reference frame 585 together to apply the coding unit of the inter mode. Perform motion compensation for the

Data in the spatial domain that has passed through the intra predictor 550 and the motion compensator 560 may be post-processed through the deblocking unit 570 and the loop filtering unit 580 to be output to the reconstructed frame 595. In addition, the post-processed data through the deblocking unit 570 and the loop filtering unit 580 may be output as the reference frame 585.

In order to decode the image data by the image data decoder 95 of the video decoding apparatus 90 according to an embodiment, step-by-step operations after the parser 510 of the image decoder 500 according to an embodiment are performed. Can be.

In order to be applied to the video decoding apparatus 90 according to an exemplary embodiment, the parser 510, the entropy decoder 520, the inverse quantizer 530, and the inverse frequency converter, which are components of the image decoder 500, may be used. 540, the intra predictor 550, the motion compensator 560, the deblocking unit 570, and the loop filtering unit 580 all perform operations based on the coding unit of the coding depth for each largest coding unit. do.

In particular, the intra predictor 550 and the motion compensator 560 determine a coding unit and a prediction mode in consideration of the maximum size and depth of the coding unit, and the inverse frequency transformer 540 determines the maximum size and depth of the coding unit. Consider the size of the transformation unit.

13 is a diagram of deeper coding units according to depths, and prediction units, according to an embodiment of the present invention.

The video encoding apparatus 80 and the video decoding apparatus 90 according to an embodiment use hierarchical coding units to consider image characteristics. The maximum height, width, and maximum depth of the coding unit may be adaptively determined according to the characteristics of the image, and may be variously set according to a user's request. According to the maximum size of the preset coding unit, the size of the coding unit for each depth may be determined.

The hierarchical structure 600 of a coding unit according to an embodiment illustrates a case in which a maximum height and a width of a coding unit are 64 and a maximum depth is five. The maximum depth illustrated in FIG. 13 represents the total number of depth levels from the maximum coding unit to the minimum coding unit.

Since the depth deepens along the vertical axis of the hierarchical structure 600 of the coding unit according to an embodiment, the height and the width of the coding unit for each depth are divided. Also, a prediction unit is illustrated, which is a partial data unit that is the basis of prediction encoding of each deeper coding unit along the horizontal axis of the hierarchical structure 600 of the coding unit.

That is, the coding unit 610 has a depth of 0 as the largest coding unit of the hierarchical structure 600 of the coding unit, and the size, ie, the height and width, of the coding unit is 64x64. The depth is deeper along the vertical axis, the coding unit 620 of depth 1 having a size of 32x32, the coding unit 630 of depth 2 having a size of 16x16, the coding unit 640 of depth 3 having a size of 8x8, and the depth 4 of depth 4x4. The coding unit 650 exists. A coding unit 650 having a depth of 4 having a size of 4 × 4 is a minimum coding unit.

Partial data units are arranged as prediction units of the coding unit along the horizontal axis for each depth. That is, the prediction unit of the coding unit 610 of size 64x64 having a depth of 0 includes the partial data unit 610 of the size 64x64 included in the coding unit 610 of the size 64x64, the partial data units 612 of the size 64x32, It may be partial data units 614 of size 32x64 and partial data units 616 of size 32x32. In contrast, the coding unit may be a data unit of a square size of a minimum size including the prediction units 610, 612, 614, and 616.

Similarly, the prediction unit of the coding unit 620 having a size of 32x32 having a depth of 1 includes the partial data unit 620 having a size of 32x32, the partial data units 622 having a size of 32x16, included in the coding unit 620 having a size of 32x32. It may be partial data units 624 of size 16x32 and partial data units 626 of size 16x16.

Similarly, the prediction unit of the coding unit 630 of size 16x16 having a depth of 2 may include the partial data unit 630 of the size 16x16 included in the coding unit 630 of the size 16x16, the partial data units 632 of the size 16x8, The partial data units 634 of size 8x16 and the partial data units 636 of size 8x8.

Similarly, the prediction unit of the coding unit 640 of size 8x8 having a depth of 3 includes the partial data unit 640 of the size 8x8 included in the coding unit 640 of the size 8x8, the partial data units 642 of the size 8x4, It may be partial data units 644 of size 4x8 and partial data units 646 of size 4x4.

Finally, the coding unit 650 of size 4x4 having a depth of 4 is a minimum coding unit and the coding unit of the lowest depth, and the prediction unit of the coding unit 650 of size 4x4 is a partial data unit 650 having a size of 4x4, and the size thereof. 4x2 partial data units 652, size 2x4 partial data units 654, size 2x2 partial data units 656.

The coded depth and the coded mode determiner 83 of the video encoding apparatus 80 according to an exemplary embodiment may determine depths of the maximum coding unit 610 in order to determine the coded depth of the maximum coding unit 610. Encoding must be performed for each coding unit of.

The number of deeper coding units according to depths for including data having the same range and size increases as the depth increases. For example, four coding units of depth 2 are required for data included in one coding unit of depth 1. Therefore, in order to compare the encoding results of the same data for each depth, each of the coding units having one depth 1 and four coding units having four depths 2 should be encoded.

For each depth coding, encoding may be performed for each prediction unit of a coding unit according to depths along a horizontal axis of the hierarchical structure 600 of the coding unit, and a representative coding error, which is the smallest coding error at a corresponding depth, may be selected. . In addition, a depth deeper along the vertical axis of the hierarchical structure 600 of the coding unit, the encoding may be performed for each depth, and the minimum coding error may be searched by comparing the representative coding error for each depth. The depth in which the minimum coding error occurs in the maximum coding unit 610 may be selected as the coding depth and the partition type of the maximum coding unit 610.

14 illustrates a relationship between a coding unit and transformation units, according to an embodiment of the present invention.

The video encoding apparatus 80 or the video decoding apparatus 90 according to the embodiment encodes or decodes an image in coding units having a size smaller than or equal to the maximum coding unit for each maximum coding unit. The size of a transform unit for frequency transformation during the encoding process may be selected based on a data unit that is not larger than each coding unit.

For example, in the video encoding apparatus 80 or the video decoding apparatus 90 according to the embodiment, when the current coding unit 710 is 64x64 size, the 32x32 size transformation unit 720 is selected. Frequency conversion can be performed using the above.

In addition, the data of the 64x64 coding unit 710 is encoded by performing frequency transformation on the 32x32, 16x16, 8x8, and 4x4 size transformation units of 64x64 size or less, and then the transformation unit having the least error with the original is obtained. Can be selected.

FIG. 15 illustrates encoding information according to depths, according to an embodiment of the present invention.

The encoding information encoder of the video encoding apparatus 80 according to an embodiment is information about an encoding mode, and information about a partition type 800, information about a prediction mode 810, and transformation for each coding unit of each coded depth. Information 820 about the unit size may be encoded and transmitted.

The information about the partition type 800 indicates information about a type in which the current coding unit is split as a prediction unit for prediction encoding of the current coding unit. For example, the current coding unit CU_0 of depth 0 and size 2Nx2N includes a prediction unit 802 of size 2Nx2N, a prediction unit 804 of size 2NxN, a prediction unit 806 of size Nx2N, and a prediction unit of size NxN 808. ) May be divided into any one type and used as a prediction unit. In this case, the information 800 about the partition type of the current coding unit includes a prediction unit 802 of size 2Nx2N, a prediction unit 804 of size 2NxN, a prediction unit 806 of size Nx2N, and a prediction unit 808 of size NxN. It is set to indicate one of the.

Information 810 about the prediction mode indicates a prediction mode of each prediction unit. For example, through the information 810 about the prediction mode, the prediction unit indicated by the information 800 about the partition type is performed by one of the intra mode 812, the inter mode 814, and the skip mode 816. Can be set.

In addition, the information about the transform unit size 820 indicates whether to transform the current coding unit based on the transform unit. For example, the transform unit may be one of a first intra transform unit size 822, a second intra transform unit size 824, a first inter transform unit size 826, and a second intra transform unit size 828. have.

The information extractor 93 of the video decoding apparatus 90 according to an embodiment may include information about a partition type 800, information 810 about a prediction mode, and a transform unit size for each depth-based coding unit. The information 820 may be extracted and used for decoding.

16 is a diagram of deeper coding units according to depths, according to an embodiment of the present invention.

Segmentation information may be used to indicate a change in depth. The split information indicates whether a coding unit of a current depth is split into coding units of a lower depth.

The prediction unit 910 for predictive encoding of coding units having a depth of 0 and 2N_0x2N_0 size includes a partition type 912 having a size of 2N_0x2N_0, a partition type having a size of 2N_0xN_0 (914), a partition type having a size of N_0x2N_0 (916), and a partition having a size of N_0xN_0 Type 918 may be included.

For each partition type, prediction coding must be repeatedly performed for one prediction unit of 2N_0x2N_0 size, two prediction units of 2N_0xN_0 size, two prediction units of N_0x2N_0 size, and four prediction units of N_0xN_0 size. For prediction units of size 2N_0x2N_0, size N_0x2N_0, size 2N_0xN_0, and size N_0xN_0, prediction encoding may be performed in intra mode and inter mode. The skip mode may be performed only for prediction encoding on a prediction unit having a size of 2N_0x2N_0.

If the encoding error of the partition type 918 having the size N_0xN_0 is the smallest, the depth 0 is changed to 1 (920), and the depth 2 and the coding units 922, 924, 926, and 928 of the partition type having the size N_0xN_0 are changed. We can repeatedly search for the minimum coding error.

Since encoding is repeatedly performed on coding units 922, 924, 926, and 928 having the same depth, only one of them will be described, for example, encoding of a coding unit having a depth of 1. The prediction unit 930 for predictive encoding of a coding unit having a depth of 1 and a size of 2N_1x2N_1 (= N_0xN_0) includes a partition type 932 of size 2N_1x2N_1, a partition type 934 of size 2N_1xN_1, and a partition type 936 of size N_1x2N_1. It may include a partition type 938 of size N_1xN_1. For each partition type, prediction encoding must be repeatedly performed for each prediction unit of size 2N_1x2N_1, prediction units of two sizes 2N_1xN_1, prediction units of two sizes N_1x2N_1, and prediction units of four sizes N_1xN_1.

Also, if the encoding error of the partition type 938 having the size N_1xN_1 is the smallest, the depth 1 is changed to the depth 2 (940), and the coding units 942, 944, 946, and 948 of the depth 2 and the size N_2xN_2 are used. We can repeatedly search for the minimum coding error.

In the case where the maximum depth is d, the split information for each depth may be set until the depth d-1. That is, the prediction unit 950 for the prediction encoding of the coding unit of the depth d-1 and the size 2N_ (d-1) x2N_ (d-1) is a partition having the size 2N_ (d-1) x2N_ (d-1). Partition type 954 of type 952, size 2N_ (d-1) xN_ (d-1), partition type 956 of size N_ (d-1) x2N_ (d-1), size N_ (d- 1) xN_ (d-1) may include a partition type 958.

For each partition type, one prediction unit of size 2N_ (d-1) x2N_ (d-1), two prediction units of 2N_ (d-1) xN_ (d-1), two sizes N_ (d-1) It is necessary to repeatedly perform prediction coding on each prediction unit of) x2N_ (d-1) and prediction units of four sizes N_ (d-1) xN_ (d-1). Since the maximum depth is d, the coding unit 952 of the depth d-1 no longer undergoes a division process.

The video encoding apparatus 80 according to an embodiment compares the encoding error for each depth to select a depth at which the smallest encoding error occurs in order to determine a coding depth for the coding unit 912.

For example, as the encoding error for the coding unit having a depth of 0, a prediction unit in which the smallest coding error occurs after predictive encoding is performed for each partition type 912, 914, 916, and 918 is determined. Similarly, prediction units having the smallest coding error may be searched for depths 0, 1, ..., d-1. Since the data unit 960 having a size of 2N_dx2N_d is a minimum unit and is a prediction unit of the coding unit 952 having a depth d-1, a coding error may be determined based on the prediction encoding.

In this way, the depth with the smallest error can be determined by comparing the minimum coding errors for all depths of depths 0, 1, ..., d-1 and determined as the coding depth. The coded depth and the prediction unit of the corresponding depth may be encoded and transmitted as information about an encoding mode. In addition, since the coding unit must be split from the depth 0 to the coded depth, only the split information of the coded depth is set to '0', and the split information for each depth except the coded depth should be set to '1'.

The information extractor 93 of the video decoding apparatus 90 according to an embodiment may extract information about a coding depth and a prediction unit of the coding unit 912 and use the same to decode the coding unit 912. According to an embodiment, the video decoding apparatus 90 may identify a depth having split information of '0' as a coding depth by using split information for each depth, and may use the decoding depth by using information about an encoding mode for a corresponding depth. have.

17, 18, and 19 illustrate a relationship between a coding unit, a prediction unit, and a frequency transform unit, according to an embodiment of the present invention.

The coding units 1010 are coding units according to coding depths determined by the video encoding apparatus 80 according to an embodiment with respect to the maximum coding unit. The prediction unit 1060 is prediction units of coding units of respective coding depths of the coding unit 1010, and the transformation unit 1070 is transformation units of coding units of each coding depth.

If the depth-based coding units 1010 have a depth of 0, the coding units 1012 and 1054 have a depth of 1, and the coding units 1014, 1016, 1018, 1028, 1050, and 1052 have depths. 2, coding units 1020, 1022, 1024, 1026, 1030, 1032, and 1048 have a depth of three, and coding units 1040, 1042, 1044, and 1046 have a depth of four.

Some of the prediction units 1060 1014, 1016, 1022, 1032, 1048, 1050, 1052, and 1054 are types in which coding units are split. That is, the prediction units 1014, 1022, 1050, and 1054 are partition types of 2N × N, the prediction units 1016, 1048, and 1052 are partition types of N × 2N, and the prediction units 1032 are partition types of N × N. That is, the prediction unit of the deeper coding units 1010 is smaller than or equal to each coding unit.

The image data of the part 1052 of the transformation units 1070 is subjected to frequency transformation or inverse frequency transformation in a data unit having a smaller size than the coding unit. In addition, the transformation units 1014, 1016, 1022, 1032, 1048, 1050, 1052, and 1054 are data units having different sizes or shapes when compared to the corresponding prediction unit among the prediction units 1060. That is, the video encoding apparatus 80 according to an embodiment and the video decoding apparatus 90 according to the embodiment may be an intra prediction / motion estimation / motion compensation operation and a frequency conversion / inverse transformation operation for the same coding unit. Each can be performed based on separate data units.

20 is a diagram of encoding information according to coding units, according to an embodiment of the present invention.

The bitstream output unit 85 of the video encoding apparatus 80 according to an embodiment outputs encoding information for each coding unit, and the information extractor 93 of the video decoding apparatus 90 according to an embodiment may be encoding unit. Star coding information can be extracted.

The encoding information may include split information about a coding unit, partition type information, prediction mode information, and transformation unit size information. The encoding information illustrated in FIG. 11 is an example that may be set in the video encoding apparatus 80 and the video decoding apparatus 90 according to an embodiment.

The split information may indicate a coded depth of a corresponding coding unit. That is, since a depth that is no longer divided according to the split information is a coded depth, partition type information, a prediction mode, and transform unit size information may be defined for the coded depth. If it is to be further split by the split information, encoding should be performed independently for each coding unit of the divided four lower depths.

The partition type information may indicate a partition type of a transformation unit of a coding unit of a coding depth as one of 2Nx2N, 2NxN, Nx2N, and NxN. The prediction mode may be represented by one of an intra mode, an inter mode, and a skip mode. Intra mode and inter mode may be defined in partition types 2Nx2N, 2NxN, Nx2N and NxN, and a skip mode may be defined only in partition type 2Nx2N. The conversion unit size may be set to two kinds of sizes in the intra mode and two kinds of sizes in the inter mode.

For each minimum unit in the coding unit, encoding information for each coding unit of the coded depth to which the code belongs may be stored. Therefore, if the encoding information held by the adjacent minimum units, respectively, can be confirmed, it can be confirmed whether it is included in the coding unit of the same coded depth. In addition, since the coding unit of the corresponding coding depth may be identified by using the encoding information held by the minimum unit, the distribution of the coded depths within the maximum coding unit may be inferred.

Therefore, in this case, when the current coding unit is predicted with reference to the neighboring data unit, the minimum unit data may be referred to by using the encoding information of the minimum unit in the depth-specific coding unit adjacent to the current coding unit.

In another embodiment, the encoding information of the coding units according to depths may be stored only for the smallest representative unit among the coding units according to depths. In this case, when the current coding unit is predicted with reference to the neighboring coding unit, the data adjacent to the current coding unit may be referred to in the depth-wise coding unit by using the encoding information of the adjacent coding units according to depths.

21 is a flowchart of a video encoding method using dynamic range transformation based on content, based on a hierarchical data unit for each region according to an embodiment of the present invention.

In operation 2110, the current picture among the input video sequences is divided into maximum coding units.

In operation 2120, the dynamic range is converted based on the content of the image data of the current picture, and intra prediction, inter prediction, frequency conversion, and quantization are performed based on hierarchical data units for each region on the image data having the converted dynamic range. do. As a result of performing recursive encoding for each region of the largest coding unit, a coding depth and an encoding mode may be determined. According to an embodiment, intra prediction and inter prediction may be performed on image data whose dynamic range is converted based on content.

According to the conversion of the dynamic range based on the content according to an embodiment, the dynamic range is extended so that the pixel values of the upper limit value or the lower limit value range of the original image correspond to the minimum or maximum value range that can be expressed by a predetermined bit depth. Can be.

In operation 2130, entropy encoding is performed on the data encoded for each coding unit according to depths, and a bitstream including encoded data, information about a coded depth, an encoding mode, and dynamic range information is output.

FIG. 22 is a flowchart of a video decoding method using dynamic range transformation based on content, based on a hierarchical data unit for each region according to an embodiment of the present invention.

In operation 2210, the encoded video bitstream is received. In operation 2220, the received bitstream is parsed to extract encoded data, encoding information, and dynamic range information from the parsed bitstream. In operation 2230, data symbols are reconstructed by performing entropy decoding on the encoded data.

In operation 2240, image data is decoded for each largest coding unit by performing inverse quantization, inverse frequency transformation, intra prediction, and motion compensation on a hierarchical data unit for each region based on encoding information.

In operation 2250, the current picture is reconstructed by performing in-loop filtering such as deblocking filtering on the decoded picture, and the dynamic range of the current picture is reconstructed. When the information on the dynamic range transformation is extracted from the bitstream, the original dynamic range of the current picture may be reconstructed based on the information on the dynamic range transformation.

According to the restoration of the dynamic range based on the content according to an exemplary embodiment, the dynamic range may be reduced such that the minimum value or the maximum value range of the reconstructed image data corresponds to the pixel value of the upper limit value or the lower limit value of the original image. Can be.

A video encoding method using dynamic lane transformation based on content based on hierarchical data units of regions according to an embodiment, or dynamic lane based on content based on hierarchical data units of regions according to an embodiment According to the video decoding method using the transformation, since the dynamic range of the input image can be extended to further perform image processing such as intra prediction and inter prediction, the image quality according to encoding and decoding can be improved.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium includes storage media such as magnetic storage media (eg, ROM, floppy disk, hard disk, etc.) and optical reading media (eg, CD-ROM, DVD, etc.).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

Claims (44)

Performing inter prediction through intra prediction and motion estimation on the current frame using image data obtained by converting a dynamic range of a current frame based on content of an input video;
Performing frequency transform on the residual data generated by the intra prediction and inter prediction, and performing quantization on the transform coefficients generated by the frequency transform; And
Performing entropy encoding on the quantized transform coefficients, and outputting a bitstream including encoded data of the current frame. The method of claim 1, wherein performing intra prediction and inter prediction comprises:
The intra prediction and the inter prediction using image data whose dynamic range is converted to extend the pixel values of the current frame in a range of a maximum value to a minimum value that can be expressed as a current bit depth of a pixel value. And a video encoding method using dynamic range transformation based on content. The method of claim 2, wherein performing the intra prediction and the inter prediction is performed.
Convert a value larger than an upper limit among pixel values of the current frame to correspond to the maximum value, and convert a value smaller than a lower limit among pixel values of the current frame to correspond to the minimum value,
Video encoding using a dynamic range transformation based on content, characterized by converting pixel values between the upper limit value and the lower limit value among the pixel values of the current frame to have a one-to-one correspondence with the value between the maximum limit value and the minimum limit value. Way. The method of claim 3, wherein
The upper limit value and the lower limit value are video encoding methods using dynamic range transformation based on content, wherein a maximum value and a minimum value of pixel values of the current frame are respectively. The method of claim 3, wherein performing intra prediction and inter prediction comprises:
For pixel values between the upper limit value and the lower limit value, a product of a difference between a current pixel value and the lower limit value, and a difference between the maximum limit value and the maximum limit value divided by the difference between the upper limit value and the lower limit value is one-to-one corresponded. A video encoding method using dynamic range transformation based on content, characterized in that for converting. The method of claim 5, wherein
And a one-to-one corresponding pixel value for the pixel values between the upper limit value and the lower limit value is output through a bit shift operation using a current bit depth. The method of claim 2,
The current bit depth is a video encoding method using a dynamic range transformation based on content, characterized in that the bit depth is internally extended during the decoding process of the video. The method of claim 7, wherein
The current bit depth is a video encoding method using the dynamic range transformation based on the content, characterized in that the sum of the first bit depth and the internally extended second bit depth for the extension of the dynamic range. The method of claim 8, wherein performing intra prediction and inter prediction comprises:
Extending the bit depth of the pixel value of the video by the second bit depth;
Extending a dynamic range by the first bit depth with respect to intermediate data of the bit depth extended by the second bit depth; And
And performing the intra prediction and the inter prediction on the data of which the dynamic range has been expanded. 10. The method of claim 9, wherein the dynamic range expanding step comprises:
A value larger than an upper limit among the intermediate data is converted to correspond to the maximum value, and a value smaller than a lower limit among pixel values of the current frame is converted to correspond to the minimum value,
And converting the data between the upper limit value and the lower limit value among the intermediate data so as to have a one-to-one correspondence with the value between the maximum limit value and the minimum limit value. The method of claim 10, wherein the dynamic range expansion step,
Dynamic range based on content, characterized by outputting a pixel value corresponding to the pixel value between the upper limit value and the lower limit value through a bit shift operation using the first bit depth and the second bit depth. Video coding method using transform. The method of claim 2, wherein performing the intra prediction and the inter prediction is performed.
And the intraframe and the inter prediction are performed on the pixel values of the current frame by using the image data whose dynamic range is converted for each of the luma component and the chroma component. The method of claim 3, wherein
Wherein the pixel values between the upper limit value and the lower limit value among the pixel values of the current frame are converted one-to-one corresponding to the value between the maximum limit value and the minimum limit value according to a nonlinear function relationship. Video coding method using range transformation. The method of claim 13,
If the pixel values between the upper limit value and the lower limit value among the pixel values of the current frame are divided into a predetermined number of sections, the pixel value for each section corresponds to a function relationship of the slope for each section that increases linearly. A value is determined, and the result values of successive sections of the sections are continuous so that the result values for pixel values of all the sections are converted to correspond one-to-one to the value between the maximum limit value and the minimum limit value. A video encoding method using dynamic range transformation based on a characteristic content. The method of claim 3, wherein
And a current pixel value larger than the upper limit or smaller than the lower limit is truncated. The method of claim 3, wherein
The upper limit value and the lower limit value are determined according to one data unit of an image sequence, a frame, a frame set for intra prediction, an area, and a coding unit of the input video. . The method of claim 1, wherein performing intra prediction and inter prediction comprises:
For previous frames whose dynamic range is transformed based on the content of the input video, inter prediction, frequency transformation, quantization, inverse quantization, inverse frequency transformation, and deblocking filtering through intra prediction, motion estimation and compensation are generated. Restoring the dynamic range of the reconstructed frames;
Converting a dynamic range of a reconstructed frame in which the dynamic range is restored, based on the dynamic range of the current frame; And
The video encoding method using the dynamic range transformation based on content, wherein the inter frame prediction is performed on the current frame with reference to the reference frame having the converted dynamic range. The video encoding method of claim 1, wherein the video encoding method using the dynamic range transformation based on the content includes:
And encoding and transmitting the information about the transformation of the dynamic range. The video encoding method using the dynamic range transformation based on the content. The method of claim 18,
And the information on the conversion of the dynamic range includes information about an upper limit value and a lower limit value of the content of the input video. The method of claim 1,
The intra prediction step and the inter prediction step may include: a region in which the maximum coding unit is hierarchically divided and reduced as the depth increases with respect to each maximum coding unit in which the current frame is divided into coding units having a predetermined maximum size. The intra prediction and the inter prediction are performed for each coding unit of at least one depth.
In the performing of the frequency transform and quantization, the frequency transform and the quantization are performed for at least one depth-based coding unit for each region for each maximum coding unit,
The video encoding method using the dynamic range transformation based on the content may include: determining an encoding mode for a coding unit of the coded depth including information on at least one coded depth for generating a minimum coding error with an original image of the picture. Further comprising:
The entropy encoding may include performing entropy encoding on encoded image data that is an encoding result according to the determined encoding depth and an encoding mode, and including information about the determined encoding depth and an encoding mode and encoded image data. A video encoding method using dynamic range transformation based on content, characterized by outputting a stream. Parsing the received bitstream to extract encoded data of the current frame of the original video;
Restoring a data symbol by performing entropy decoding on the extracted encoded data;
Reconstructing residual data for the current frame by performing inverse quantization and inverse frequency transformation on the reconstructed data symbol, and reconstructing image data by performing intra prediction and motion compensation on the reconstructed residual data step; And
Restoring the current frame by performing deblocking filtering on the restored image data and restoring the converted dynamic range based on the content of the original video. Video decoding method using. The method of claim 21, wherein the current frame recovery step,
The reconstructed pixel value of the reconstructed image data may be reconstructed to the dynamic range of the current frame. The reconstructed pixel value extended in the range of the maximum value to the minimum value that can be expressed as the current bit depth of the pixel value is restored. A video decoding method using dynamic range transformation based on content to be described. The method of claim 22, wherein the current frame recovery step,
The maximum limit value and the minimum limit value of the restored image data are restored to correspond to an upper limit value and a lower limit value among pixel values of the current frame, respectively.
And converting values between the maximum value and the minimum value of the reconstructed image data so as to have a one-to-one correspondence with values between the upper limit value and the lower limit value. The method of claim 23,
The upper limit value and the lower limit value are video decoding methods using dynamic range transformation based on content, wherein the maximum value and the minimum value of pixel values of the current frame are respectively. The method of claim 23, wherein the current frame recovery step,
For values between the maximum value and the minimum value, the product of the difference between the currently restored data value, the upper limit value and the lower limit value, divided by the difference between the maximum limit value and the maximum limit value plus the lower limit value A video decoding method using dynamic range transformation based on content, characterized in that the one-to-one correspondence is converted. The method of claim 25,
And a pixel value corresponding one-to-one with respect to the values between the maximum value and the minimum value is output through a bit shift operation using a current bit depth. The method of claim 22,
The current bit depth is a video decoding method using a dynamic range transformation based on content, characterized in that the bit depth is internally extended during the decoding process of the video. The method of claim 27,
Wherein the current bit depth is a sum of a first bit depth for expanding the dynamic range and a second bit depth extended internally. The method of claim 28, wherein the current frame recovery step,
Restoring a dynamic range of the reconstructed image data extended by the first bit depth; And
Restoring the bit depth of the intermediate data generated by restoring the dynamic range by the second bit depth to restore the bit depth of the pixel value of the current frame. Video decoding method. The method of claim 29, wherein the dynamic range restoration step,
Restoring the maximum limit value and the minimum limit value of the restored image data to correspond to an upper limit value and a lower limit value of the intermediate data, respectively; And
And generating the intermediate data by converting the values between the maximum and minimum values of the reconstructed image data to have a one-to-one correspondence to the values between the upper limit value and the lower limit value of the intermediate data. A video decoding method using based dynamic range transformation. The method of claim 30, wherein the intermediate data generation step,
And outputs a pixel value corresponding one to one with respect to pixel values between the upper limit value and the lower limit value of the intermediate data through a bit shift operation using the first bit depth and the second bit depth. Video decoding method using range transformation. The method of claim 22, wherein the current frame recovery step,
And reconstructing the dynamic range for each of the luma and chroma components of the reconstructed image data, thereby reconstructing a pixel value of the current frame. The method of claim 23,
Using the dynamic range transformation based on the content, the values between the maximum value and the minimum value of the reconstructed image data are converted one-to-one corresponding to the value between the upper limit value and the lower limit value according to a nonlinear function relationship. Video decoding method. The method of claim 33, wherein
If the pixel values between the maximum value and the minimum value of the reconstructed image data are divided into a predetermined number of sections, the data for each section correspond to a function corresponding to a function of a slope for each section that increases linearly. A value is determined, and pixel values corresponding to data of consecutive sections among the sections are successively converted, so that pixel values for data of all the sections are converted one-to-one to a value between the upper limit value and the lower limit value. A video decoding method using dynamic range transformation based on content to be described. The method of claim 23,
And a current pixel value larger than the upper limit value or smaller than the lower limit value is truncated. The method of claim 23,
And the upper limit value and the lower limit value are determined for each data unit of an image sequence, a frame, a frame set for intra prediction, an area, and a coding unit of the original video. The video decoding method of claim 21, wherein the video decoding method using the dynamic range transformation based on the content includes:
And receiving information on the original video for converting the dynamic range of the current frame. 39. The method of claim 37,
And the information on the conversion of the dynamic range includes information about an upper limit value and a lower limit value of the content of the original video. The method of claim 21, wherein the image data restoration step,
Restoring the dynamic range of the reconstructed frame relative to the previous frame of the original video;
Converting a dynamic range of the reconstructed frame based on the dynamic range of the current frame; And
And a motion compensation for the current frame by referring to the reconstructed frame of which the dynamic range has been converted. The method of claim 21,
The data symbol reconstruction step may include: dividing a picture of an input image sequence into coding units having a predetermined maximum size in the encoding step of the received bitstream, and increasing the depth of each maximum coding unit as the depth increases. Encoding including information on at least one coded depth that generates a minimum coding error with respect to an original image of the picture by performing encoding on at least one depth-by-depth coding unit for each region that is hierarchically divided and reduced. Further extracting information about an encoding mode for a coding unit of depth from the bitstream,
The image data reconstructing step may decode the encoded image data by performing inverseization, inverse frequency transformation, intra prediction, and motion compensation based on the encoding depth and the encoding mode based on the information about the encoding mode. And a video encoding apparatus using dynamic range transformation based on content. An intra predictor configured to perform intra prediction on the current frame by using image data obtained by converting a dynamic range of a current frame based on content of an input video;
An inter prediction unit using the dynamic range converted image data to perform inter prediction through motion estimation on the current frame;
A frequency converter configured to perform frequency conversion on the residual data generated by the intra prediction and the inter prediction;
A quantization unit performing quantization on the transform coefficients generated by the frequency transform;
An entropy encoder for performing entropy encoding on the quantized transform coefficients; And
And an output unit configured to output a bitstream including the encoded data of the current frame, generated by the entropy encoding. A parser configured to parse the received bitstream and extract encoded data of a current frame of the original video;
An entropy decoder configured to reconstruct data symbols by performing entropy decoding on the extracted encoded data;
An inverse quantization and inverse frequency converter configured to perform inverse quantization and inverse frequency transformation on the recovered data symbols to restore residual data for the current frame;
An intra predictor for performing intra prediction on the reconstructed residual data;
A motion compensation unit to perform motion compensation on the restored residual data; And
And an image reconstruction unit for reconstructing the current frame by performing deblocking filtering on the image data reconstructed by the intra prediction and the motion compensation, and reconstructing the dynamic range converted based on the content of the original video. A video decoding apparatus using a dynamic range transformation based on content characterized in that. A computer-readable recording medium having recorded thereon a program for implementing a video encoding method using a dynamic range transformation based on the content of any one of claims 1 to 20. 41. A computer readable recording medium having recorded thereon a program for implementing a video decoding method using dynamic range transformation based on the content of any of claims 21-40.

Images