KR102511611B1 - Image encoding method/apparatus, image decoding method/apparatus and and recording medium for storing bitstream - Google Patents

Abstract

The present invention provides an image encoding method and an image decoding method. An image encoding method of the present invention includes a first division step of dividing a current image into a plurality of blocks, and dividing a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks. and a second dividing step, wherein the sub-block including the boundary of the current image is divided until the sub-block including the boundary of the current image does not exist among the sub-blocks. It is performed recursively with the target block.

Description

Image encoding method/device, image decoding method/device, and recording medium storing bitstream

The present invention relates to a video encoding/decoding method and apparatus. In particular, it relates to a video boundary processing method and apparatus in video encoding/decoding. More particularly, it relates to a method and apparatus for efficiently processing image padding and borders of images in a tree structure.

Recently, the demand for multimedia data such as moving pictures is rapidly increasing on the Internet. However, it is difficult to keep up with the rapidly increasing amount of multimedia data at the rate at which the bandwidth of a channel develops. As part of this trend, ITU-T's Video Coding Expert Group (VCEG) and ISO/IEC's Moving Picture Expert Group (MPEG) are studying video compression standards through steady joint research.

Video compression is largely composed of intra-prediction, inter-prediction, transform, quantization, entropy coding, and in-loop filter. Among them, intra-prediction refers to a technique of generating a prediction block for a current block using reconstructed pixels existing around the current block. At this time, since the encoding process is performed in units of blocks, there is a problem in that it is difficult to perform encoding in units of blocks unless the size of the image is a multiple of the block size.

An object of the present invention is to provide a padding method and apparatus for adjusting the size of an image in units of blocks in encoding/decoding of an image.

In addition, an object of the present invention is to provide a block division method and apparatus that are efficient in video boundary processing in video encoding/decoding.

Another object of the present invention is to provide a computer-readable recording medium storing a bitstream generated by the image encoding method/apparatus according to the present invention.

An image encoding method according to the present invention includes a first division step of dividing a current image into a plurality of blocks, and dividing a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks. and a second division step of dividing the sub-blocks including the boundary of the current image until the sub-block including the boundary of the current image does not exist among the sub-blocks. It can be performed recursively using a partitioning target block.

In the image encoding method according to the present invention, the first dividing step may be a step of dividing the current image into a plurality of largest blocks having the same size.

In the video encoding method according to the present invention, the second splitting step may be a step of performing quad tree splitting or binary tree splitting on the splitting target block.

In the video encoding method according to the present invention, when the division target block has a size capable of performing the quad tree division, the second division step performs the quad tree division on the division target block, and the division target block When the quad tree splitting is not possible, the binary tree splitting may be performed on the splitting target block in the second splitting step.

In the video encoding method according to the present invention, when the quad-tree splitting or the binary tree splitting is performed on the splitting target block in the second splitting step, a second step indicating whether the splitting block is quad-tree splitted or not 1 split information or 2 split information indicating whether the split target block is binary tree split may not be encoded.

In the video encoding method according to the present invention, when the binary tree splitting is performed on the splitting target block in the second splitting step, the splitting direction of the binary tree splitting is included in the splitting block and simultaneously the current splitting target block. It may be determined based on the shape of the encoding target block, which is an area within the boundary of the image.

In the video encoding method according to the present invention, when the binary tree splitting is performed on the splitting target block in the second splitting step, splitting direction information indicating a splitting direction of the binary tree splitting may not be encoded.

In the video encoding method according to the present invention, the second segmentation step further comprises comparing the size of a remaining area included in the segmentation target block and at the same time, which is an area outside the boundary of the current video, with a predetermined threshold value; , Based on the comparison result, one of the quad tree partitioning and the binary tree partitioning may be performed on the partitioning target block.

In the video encoding method according to the present invention, the size of the remaining area is the smaller of the horizontal length and the vertical length of the remaining area, and in the second dividing step, the size of the remaining area is greater than the predetermined threshold value. , the quad tree splitting may be performed on the splitting target block, and if the size of the remaining area is smaller than the predetermined threshold value, the binary tree splitting may be performed on the splitting target block.

In the video encoding method according to the present invention, the second splitting step is a step of performing binary tree splitting on the splitting object block, and the splitting direction of the binary tree splitting is included in the splitting object block and simultaneously It may be determined based on the shape of the remaining area, which is an area outside the boundary of the current image.

In the video encoding method according to the present invention, the splitting direction of the binary tree partitioning is the vertical direction when the shape of the remaining area is a vertically long block, and the horizontal direction when the shape of the remaining area is a horizontally long block. direction can be

An image decoding method according to the present invention includes a first division step of dividing a current image into a plurality of blocks, and dividing a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks. and a second division step of dividing the sub-blocks including the boundary of the current image until the sub-block including the boundary of the current image does not exist among the sub-blocks. It can be performed recursively using a partitioning target block.

In the video decoding method according to the present invention, the second splitting step may be a step of performing quad tree splitting or binary tree splitting on the splitting target block.

In the video decoding method according to the present invention, when the division target block has a size capable of performing the quad tree division, the second division step performs the quad tree division on the division target block, and the division target block When the quad tree splitting is not possible, the binary tree splitting may be performed on the splitting target block in the second splitting step.

In the video decoding method according to the present invention, when the quad-tree splitting or the binary tree splitting is performed on the splitting target block in the second splitting step, the second indicating whether the splitting block is quad-tree splitted or not 1 split information or second split information indicating whether the split target block is binary tree split may be derived as a predetermined value without being decoded from the bitstream.

In the video decoding method according to the present invention, when the binary tree splitting is performed on the splitting object block in the second splitting step, the splitting direction of the binary tree splitting is included in the splitting object block and simultaneously the current splitting direction. It may be determined based on the shape of the decoding target block, which is an area within the boundary of the image.

In the video decoding method according to the present invention, when the binary tree splitting is performed on the splitting target block in the second splitting step, splitting direction information indicating the splitting direction of the binary tree splitting is not decoded from the bitstream. , can be derived to a predetermined value.

In the video decoding method according to the present invention, the second segmentation step further comprises comparing the size of a remaining area included in the segmentation target block and at the same time, which is an area outside the boundary of the current video, with a predetermined threshold value; , Based on the comparison result, one of the quad tree partitioning and the binary tree partitioning may be performed on the partitioning target block.

In the video decoding method according to the present invention, the size of the remaining area is the smaller of the horizontal length and the vertical length of the remaining area, and in the second dividing step, the size of the remaining area is larger than the predetermined threshold value. , the quad tree splitting may be performed on the splitting target block, and if the size of the remaining area is smaller than the predetermined threshold value, the binary tree splitting may be performed on the splitting target block.

In the video decoding method according to the present invention, the second splitting step is a step of performing binary tree splitting on the splitting object block, and the splitting direction of the binary tree splitting is included in the splitting object block and simultaneously It may be determined based on the shape of the remaining area, which is an area outside the boundary of the current image.

In the video decoding method according to the present invention, the division direction of the binary tree partitioning is the vertical direction when the shape of the remaining area is a vertically long block, and the horizontal direction when the shape of the remaining area is a horizontally long block. direction can be

A computer-readable recording medium according to the present invention may store a bitstream generated by the image encoding method according to the present invention.

According to the present invention, in encoding/decoding of an image, a padding method and apparatus for adjusting the size of an image in units of blocks may be provided.

In addition, according to the present invention, in video encoding/decoding, an efficient block division method and apparatus for video boundary processing can be provided.

In addition, according to the present invention, a computer readable recording medium for storing a bitstream generated by the image encoding method/apparatus according to the present invention may be provided.

The compression efficiency of an image can be improved by using the padding method and device or the block division method and device according to the present invention.

1 is a block diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.
2 illustrates types of intra-prediction modes defined in an image encoding/decoding apparatus.
3 illustrates an intra-prediction method based on a planar mode.
4 illustrates reference pixels for intra-prediction.
5 illustrates an intra-prediction method based on a horizontal mode and a vertical mode.
6 is a schematic block diagram of an image decoding apparatus according to an embodiment of the present invention.
7 is a diagram for explaining a padding process according to an embodiment of the present invention.
8A and 8B are diagrams for explaining a padding process according to the present invention.
FIG. 9 is a diagram for explaining the operation of the image segmentation unit 101 of FIG. 1 .
10 is a diagram for explaining the operation of a decoder according to the present invention.
11 is a diagram exemplarily illustrating a case in which the size of a current image is not a multiple of the size of a maximum block.
12A to 12E are exemplary diagrams for explaining an image segmentation method according to an embodiment of the present invention.
13A to 13E are exemplary diagrams for explaining an image segmentation method according to another embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant descriptions of the same components are omitted.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , an image encoding apparatus 100 includes an image division unit 101, an intra prediction unit 102, an inter prediction unit 103, a subtraction unit 104, a transform unit 105, and a quantization unit. 106, an entropy encoding unit 107, an inverse quantization unit 108, an inverse transform unit 109, a multiplication unit 110, a filter unit 111, and a memory 112.

Each component shown in FIG. 1 is shown independently to represent different characteristic functions in the video encoding device, and does not mean that each component is made of separate hardware or a single software component. That is, each component is listed and included as each component for convenience of explanation, and at least two components of each component can be combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separated embodiments of components are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

In addition, some of the components may be optional components for improving performance rather than essential components that perform essential functions in the present invention. The present invention can be implemented by including only components essential to implement the essence of the present invention, excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement. Also included in the scope of the present invention.

The image segmentation unit 100 may divide an input image into at least one block. In this case, the input image may have various shapes and sizes such as picture, slice, tile, and segment. A block may mean a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The division may be performed based on at least one of a quadtree or a binary tree. A quad tree is a method in which a parent block is divided into four sub-blocks whose width and height are half of the parent block. The binary tree is a method in which an upper block is divided into lower blocks whose width or height is half of the upper block. Through the binary tree-based partitioning described above, a block may have a non-square shape as well as a square shape.

Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit for performing encoding or a unit for performing decoding.

The prediction units 102 and 103 may include an inter prediction unit 103 performing inter prediction and an intra prediction unit 102 performing intra prediction. It is possible to determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific details are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit.

A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 105 . In addition, prediction mode information and motion vector information used for prediction may be encoded in the entropy encoding unit 107 together with residual values and transmitted to the decoder. When a specific encoding mode is used, it is also possible to encode an original block as it is and transmit it to a decoder without generating a prediction block through the prediction units 102 and 103.

The intra-prediction unit 102 may generate a prediction block based on reference pixel information around the current block, which is pixel information in the current picture. When a prediction mode of a neighboring block of a current block to which intra prediction is to be performed is inter prediction, a reference pixel included in a neighboring block to which inter prediction is applied may be replaced with a reference pixel in another neighboring block to which intra prediction is applied. That is, when a reference pixel is unavailable, the unavailable reference pixel information may be replaced with at least one reference pixel among available reference pixels.

In intra prediction, a prediction mode may include a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional prediction mode in which directional information is not used during prediction. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and intra prediction mode information or predicted luminance signal information used to predict luminance information may be used to predict chrominance information.

The intra-prediction unit 102 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a filter that performs filtering on reference pixels of the current block, and can adaptively determine whether to apply the filter according to the prediction mode of the current prediction unit. When the prediction mode of the current block is a mode in which AIS filtering is not performed, AIS filter may not be applied.

The reference pixel interpolator of the intra prediction unit 102 interpolates the reference pixel when the intra prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the interpolated pixel value of the reference pixel, and interpolates the reference pixel at the position of the fractional unit. can create When the prediction mode of the current prediction unit is a prediction mode for generating a prediction block without interpolating reference pixels, the reference pixels may not be interpolated. The DC filter may generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

A residual block including residual value information that is a difference value between a prediction unit generated by the prediction units 102 and 103 and an original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130 and transformed.

2 is a diagram for explaining an example of an intra prediction mode. The intra prediction mode shown in FIG. 2 has a total of 35 modes. Mode 0 represents a planar mode, mode 1 represents a DC mode, and modes 2 through 34 represent angular modes.

3 is a diagram for explaining intra-prediction in a planar mode.

3 is a diagram for explaining a planar mode. To generate the predicted value of the first pixel P1 in the current block, the reconstructed pixel at the same position along the Y-axis and the reconstructed pixel T existing at the upper right of the current block are linearly interpolated as shown. Similarly, in order to generate the predicted value of the second pixel P2, the reconstructed pixel at the same position along the X axis and the reconstructed pixel L existing at the lower left of the current block are generated by linear interpolation as shown. A value obtained by averaging the two predicted pixels P1 and P2 becomes a final predicted pixel. In the planar mode, a prediction block of the current block is generated by inducing prediction pixels in the same manner as above.

4 is a diagram for explaining the DC mode. After calculating the average of the reconstructed pixels around the current block, the average value is used as a predicted value of all pixels in the current block.

FIG. 5 is a diagram for explaining an example of generating a prediction block using the 10th mode (horizontal mode) and the 26th mode (vertical mode) of FIG. 2 . In the case of using mode 10, a prediction block of the current block is generated by copying each reference pixel adjacent to the left side of the current block in the right direction. Similarly, in mode 26, a prediction block of the current block is generated by copying each reference pixel adjacent to the upper side of the current block in a downward direction.

Referring back to FIG. 1 , the inter-prediction unit 103 may predict a prediction unit based on information on at least one picture among pictures before or after the current picture, and in some cases, coding within the current picture is completed. A prediction unit may be predicted based on information of a partial region. The inter-prediction unit 103 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolator may receive reference picture information from the memory 112 and generate pixel information of an integer pixel or less in the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/4 pixels. In the case of a color difference signal, a DCT-based 4-tap interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.

The motion predictor may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. As a method for calculating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. The motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on interpolated pixels. The motion prediction unit may predict the current prediction unit by using a different motion prediction method. Various methods such as a skip method, a merge method, and an advanced motion vector prediction (AMVP) method may be used as motion prediction methods.

The subtraction unit 104 subtracts a block to be currently encoded and a prediction block generated by the intra prediction unit 102 or the inter prediction unit 103 to generate a residual block of the current block.

The transform unit 105 may transform a residual block including residual data using a transform method such as DCT, DST, or Karhunen Loeve Transform (KLT). In this case, a transform method may be determined based on an intra prediction mode of a prediction unit used to generate the residual block. For example, according to the intra prediction mode, DCT may be used in the horizontal direction and DST may be used in the vertical direction.

The quantization unit 106 may quantize the values converted into the frequency domain by the transform unit 105 . A quantization coefficient may change according to a block or an importance of an image. The value calculated by the quantization unit 106 may be provided to the inverse quantization unit 108 and the entropy encoding unit 107 .

The transform unit 105 and/or the quantization unit 106 may be selectively included in the image encoding apparatus 100 . That is, the image encoding apparatus 100 may encode the residual block by performing at least one of transformation or quantization on the residual data of the residual block, or skipping both transformation and quantization. A block input to the input of the entropy encoding unit 107 is generally referred to as a transform block, even if either transform or quantization is not performed in the image encoding apparatus 100 or both transform and quantization are not performed.

The entropy encoding unit 107 entropy encodes the input data. Entropy encoding may use various encoding methods such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).

The entropy encoding unit 107 receives residual value coefficient information and block type information of a coding unit from the prediction units 102 and 103, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, and reference frame information. , various information such as block interpolation information and filtering information can be encoded. In the entropy encoding unit 107, for the coefficients of the transform block, in units of partial blocks within the transform block, various types of flags indicating coefficients other than 0, coefficients with absolute values greater than 1 or 2, and signs of the coefficients are encoded. It can be. A coefficient that is not coded only with the flag may be coded using an absolute value of a difference between a coefficient coded with the flag and a coefficient of an actual transform block. The inverse quantization unit 108 and the inverse transform unit 109 inversely quantize the values quantized by the quantization unit 106 and inversely transform the values transformed by the transform unit 105 . The residual generated by the inverse quantization unit 108 and the inverse transform unit 109 is predicted through the motion estimation unit, motion compensation unit, and intra prediction unit 102 included in the prediction units 102 and 103. Combined with a prediction unit, a reconstructed block may be generated. The multiplier 110 multiplies the prediction block generated by the prediction units 102 and 103 and the residual block generated through the inverse transform unit 109 to generate a reconstructed block.

The filter unit 111 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by a boundary between blocks in a reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, when vertical filtering and horizontal filtering are performed, horizontal filtering and vertical filtering may be processed in parallel.

The offset correction unit may correct an offset of the deblocked image from the original image in units of pixels. In order to perform offset correction for a specific picture, pixels included in the image are divided into a certain number of areas, then the area to be offset is determined and the offset is applied to the area, or the offset is performed considering the edge information of each pixel method can be used.

Adaptive Loop Filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image with the original image. After dividing the pixels included in the image into predetermined groups, filtering may be performed differentially for each group by determining one filter to be applied to the corresponding group. Information related to whether or not to apply ALF may be transmitted for each coding unit (CU) of a luminance signal, and the shape and filter coefficients of an ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the block to be applied.

The memory 112 may store a reconstructed block or picture calculated through the filter unit 111, and the stored reconstructed block or picture may be provided to the prediction units 102 and 103 when performing inter prediction.

6 is a block diagram illustrating an image decoding apparatus 600 according to an embodiment of the present invention.

Referring to FIG. 6 , an image decoding apparatus 600 includes an entropy decoding unit 601, an inverse quantization unit 602, an inverse transform unit 603, a multiplication unit 604, a filter unit 605, a memory 606, and Prediction units 607 and 608 may be included.

When the video bitstream generated by the video encoding apparatus 100 is input to the video decoding apparatus 600, the input bitstream may be decoded according to a process opposite to that performed by the video encoding apparatus 100. .

The entropy decoding unit 601 may perform entropy decoding by a procedure opposite to that of the entropy encoding performed by the entropy encoding unit 107 of the image encoding apparatus 100 . For example, various methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be applied corresponding to the method performed by the image encoder. In the entropy decoding unit 601, the coefficients of the transform block are based on various types of flags indicating non-zero coefficients, coefficients with an absolute value greater than 1 or 2, and signs of coefficients in units of partial blocks within the transform block. can be decrypted. A coefficient not expressed only by the flag may be decoded through a sum of a coefficient expressed through the flag and a signaled coefficient.

The entropy decoding unit 601 may decode information related to intra prediction and inter prediction performed by the encoder. The inverse quantization unit 602 generates a transform block by performing inverse quantization on the quantized transform block. It operates substantially the same as the inverse quantization unit 108 of FIG. 1 .

The inverse transform unit 603 generates a residual block by performing an inverse transform on the transform block. In this case, the transformation method may be determined based on information about a prediction method (inter or intra prediction), block size and/or shape, intra prediction mode, and the like. It operates substantially the same as the inverse transform unit 109 of FIG. 1 .

The multiplication unit 604 multiplies the prediction block generated by the intra prediction unit 607 or the inter prediction unit 608 and the residual block generated by the inverse transform unit 603 to generate a reconstructed block. It operates substantially the same as the evaporation unit 110 of FIG. 1 .

The filter unit 605 reduces various types of noise generated in restored blocks.

The filter unit 605 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to a corresponding block or picture and information on whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the video encoding apparatus 100. The deblocking filter of the video decoding apparatus 600 may receive information related to the deblocking filter provided from the video encoding apparatus 100, and the video decoding apparatus 600 may perform deblocking filtering on a corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type and offset value information of the offset correction applied to the image during encoding.

ALF may be applied to a coding unit based on ALF application information and ALF coefficient information provided from the video encoding apparatus 100 . Such ALF information may be included in a specific parameter set and provided. The filter unit 605 operates substantially the same as the filter unit 111 of FIG. 1 .

The memory 606 stores the reconstruction block generated by the augmentation unit 604. It operates substantially the same as memory 112 of FIG.

The prediction units 607 and 608 may generate a prediction block based on information related to prediction block generation provided from the entropy decoding unit 601 and previously decoded block or picture information provided from the memory 606 .

The prediction units 607 and 608 may include an intra prediction unit 607 and an inter prediction unit 608 . Although not separately shown, the prediction units 607 and 608 may further include a prediction unit determining unit. The prediction unit determination unit receives various information such as prediction unit information input from the entropy decoding unit 601, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, classifies the prediction unit from the current coding unit, and predicts It is possible to determine whether a unit performs inter prediction or intra prediction. The inter-prediction unit 608 uses information necessary for inter-prediction of the current prediction unit provided from the video encoding apparatus 100 to use information included in at least one picture among pictures before or after the current picture that includes the current prediction unit. Inter prediction may be performed on the current prediction unit based on . Alternatively, inter prediction may be performed based on information of a pre-reconstructed partial region in the current picture including the current prediction unit.

Whether or not the motion prediction method of the prediction unit included in the coding unit is skip mode, merge mode, or AMVP mode based on the coding unit to perform inter-prediction. can judge

The intra-prediction unit 607 generates a prediction block using previously reconstructed pixels located around a block to be currently encoded.

The intra-prediction unit 607 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a filter that performs filtering on reference pixels of the current block, and can adaptively determine whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on reference pixels of the current block using the prediction mode of the prediction unit and AIS filter information provided by the image encoding apparatus 100 . When the prediction mode of the current block is a mode in which AIS filtering is not performed, AIS filter may not be applied.

The reference pixel interpolator of the intra-prediction unit 607 interpolates the reference pixel to obtain a reference pixel at a fractional unit position when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating the reference pixel. can create The generated reference pixel at the location of the fractional unit may be used as a prediction pixel of a pixel in the current block. When the prediction mode of the current prediction unit is a prediction mode for generating a prediction block without interpolating reference pixels, the reference pixels may not be interpolated. The DC filter may generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

The intra-prediction unit 607 operates substantially the same as the intra-prediction unit 102 of FIG. 1 .

The inter-prediction unit 608 generates an inter-prediction block using reference pictures and motion information stored in the memory 606. The inter-prediction unit 608 operates substantially the same as the inter-prediction unit 103 of FIG. 1 .

The present invention relates in particular to padding and boundary processing of an image, and various embodiments of the present invention are described in more detail below with reference to drawings.

According to an embodiment of the present invention, a pre-processing process may be performed before an encoding target image is input to the image segmentation unit 101 of FIG. 1 . An encoding target image may have various horizontal and vertical sizes in pixel units. However, since encoding and decoding of an image is performed in units of arbitrary blocks rather than units of pixels, a padding process may be required to adjust the size of an image to be encoded in units of blocks.

7 is a diagram for explaining a padding process according to an embodiment of the present invention.

In the example shown in FIG. 7 , an encoding target image includes a block area and a non-block area. By performing the preprocessing process on the encoding target image, a padded area may be added. The encoding target image to which the padded region is added may be input to the image segmentation unit 101 of FIG. 1 .

In the example shown in FIG. 7 , the unit of the minimum block may be 4x4. The horizontal or vertical length of a block may be 2 ⁿ . Therefore, for example, when the length and width of the encoding target image are 9, respectively, the first column on the right side and the lower row of the encoding target image may be non-block regions that are not included in the block region. In this way, when the video to be encoded includes the non-block region, padding may be performed in consideration of the size of the minimum block. In this way, by performing padding in consideration of the block size, the padded video to be encoded does not include the non-block region. The padded encoding target image is input to the image divider 101 of FIG. 1 and divided into a plurality of blocks, so that block-by-block encoding can be performed.

8A and 8B are diagrams for explaining a padding process according to the present invention.

First, padding may be performed in a horizontal direction using a pixel closest to a pixel to be padded. In the example shown in FIG. 8A , the top three pixels to be padded may be padded using the pixel A of the non-block area adjacent to the left.

Thereafter, padding may be performed in a vertical direction using a pixel closest to the pixel to be padded. In the example shown in FIG. 8B , the leftmost three pixels to be padded may be padded using the pixel I of the non-block area adjacent to the top.

In the above example, the padding process from the horizontal direction to the vertical direction has been described, but it is not limited thereto, and padding from the vertical direction to the horizontal direction may be performed. The size of the minimum block may be a block unit or a sub-block unit, which will be described with reference to FIG. 9 .

FIG. 9 is a diagram for explaining the operation of the image segmentation unit 101 of FIG. 1 .

An input image may be divided into a plurality of largest blocks. The maximum block size may be preset or signaled through a bitstream. For example, when the size of the input image is 128x128 and the size of the maximum block is 64x64, the input image may be divided into four maximum blocks.

Each largest block may be input to the image segmentation unit 101 of FIG. 1 and divided into a plurality of blocks. The input image may be directly input to the image segmentation unit 101 without being divided into a plurality of largest blocks. The minimum size of a divided block and/or the maximum size of a block that can be divided into sub-blocks may be preset or signaled in a header at a higher level of the block. The upper level may be at least one of video, sequence, picture, slice, tile, and largest coding unit (LCU).

The current block can be divided into 4 sub-blocks or 2 sub-blocks.

Splitting a current block into four sub-blocks may be referred to as quad-tree splitting. The current block can be divided into four sub-blocks having the same size by quad-tree splitting. The first partitioning information is information indicating whether the current block is quad-tree partitioned. The first partition information may be, for example, a 1-bit flag.

Splitting the current block into two sub-blocks may be referred to as binary-tree splitting. The current block can be split into two sub-blocks with identical or different sizes by binary tree splitting. The second partition information is information indicating whether the current block is binary tree partitioned. The second partition information may be, for example, a 1-bit flag. When the current block is binary tree split, splitting direction information may be further signaled. The division direction information may indicate whether the division of the current block is horizontal division or vertical division. The division direction information may be, for example, a 1-bit flag. When the current block is divided into two sub-blocks having different sizes, partition type information may be further signaled. The splitting type information may indicate a splitting ratio of two subblocks.

The current block can be partitioned recursively using quad tree partitioning and binary tree partitioning. For example, when a current block is quad-tree partitioned into four sub-blocks, each sub-block may be recursively quad-tree partitioned, binary tree partitioned, or undivided. When the current block is binary tree split into two sub-blocks, each sub-block may be recursively quad-tree split, binary tree split, or not split. Alternatively, quad tree partitioning may not be performed on a subblock generated by binary tree partitioning.

The encoder may determine whether or not to split the quad tree with respect to the input current block. Based on the determination, first partition information may be encoded (S901). If the current block is quad-tree partitioned (Yes in S902), the current block may be quad-tree partitioned into four blocks (S903). Each of the four blocks generated by quad tree splitting may be input again to step S901 according to a predetermined order (S904 and S916). The predetermined order may be a Z-scan order. In step S904, the first block according to the predetermined order among the four blocks may be specified and input to step S901 as the current block.

If the current block is not quad-tree split (No in S902), the encoder may determine whether or not to split the binary tree for the current block. Based on the determination, second partition information may be encoded (S905). If the current block is binary tree split (Yes in S906), the encoder may determine whether to horizontally split or vertically split the current block, and encode splitting direction information based on the decision (S907). When the current block is horizontally divided (Yes in S908), horizontal division of the current block may be performed (S909). Otherwise (No in S908), vertical division of the current block may be performed (S910). When the binary tree partitioning of the current block is asymmetric partitioning, a partitioning ratio may be determined and partitioning type information may be further encoded. In this case, steps S909 and S910 may be performed in consideration of division type information. Each of the two sub-blocks generated by binary tree splitting may be input again to step S905 according to a predetermined order (S911 and S914). The predetermined order may be left to right or top to bottom. In step S911, a first sub-block according to the predetermined order among the two sub-blocks may be specified and input as a current block in step S905. If quad-tree splitting is allowed again for sub-blocks generated by binary tree splitting, each of the two blocks generated by binary tree splitting may be input to step S901 according to a predetermined order.

If the current block is not binary tree partitioned (No in S906), encoding may be performed on the current block or current sub-block (S912). Encoding in step S912 may include prediction, transformation, quantization, and the like.

If the subblock encoded in step S912 is not the last subblock generated by binary tree splitting (No in S913), the next subblock generated by binary tree splitting can be specified and input as the current block in step S905. Yes (S914).

If the coded subblock in step S912 is the last subblock generated by binary tree splitting (Yes in step S913), whether the block to which the coded subblock belongs is the last block among blocks generated by quad tree splitting is determined. It can be judged (S915). In the case of the last block (Yes in S915), encoding of the largest block input in step S901 or the current image can be ended. If it is not the last block (No in S915), the next block generated by quad tree splitting can be specified and input as the current block in step S901 (S916).

10 is a diagram for explaining the operation of a decoder according to the present invention.

The decoder may decode the input first partition information of the current block (S1001). If the current block is quad-tree partitioned (Yes in S1002), the current block may be quad-tree partitioned into four blocks (S1003). Each of the four blocks generated by quad tree splitting may be input again to step S1001 according to a predetermined order (S1004 and S1016). The predetermined order may be a Z-scan order. In step S1004, the first block according to the predetermined order among the four blocks may be specified and input to step S1001 as the current block.

If the current block is not quad-tree split (No in S1002), the decoder can decode second partition information of the current block (S1005). If the current block is binary tree split (Yes in S1006), the decoder can decode splitting direction information (S1007). When the current block is horizontally divided (Yes in S1008), horizontal division of the current block may be performed (S1009). Otherwise (No in S1008), vertical division of the current block may be performed (S1010). When the binary tree partitioning of the current block is an asymmetric partitioning, partitioning type information may be further decoded. In this case, steps S1009 and S1010 may be performed in consideration of division type information. Each of the two sub-blocks generated by binary tree splitting may be input again to step S1005 according to a predetermined order (S1011 and S1014). The predetermined order may be left to right or top to bottom. In step S1011, a first sub-block according to the predetermined order among the two sub-blocks may be specified and input as a current block in step S1005. If quad-tree splitting is allowed again for sub-blocks generated by binary tree splitting, each of the two blocks generated by binary tree splitting may be input to step S1001 according to a predetermined order.

If the current block is not binary tree partitioned (No in S1006), decoding of the current block or current sub-block may be performed (S1012). Decoding in step S1012 may include prediction, inverse quantization, inverse transformation, and the like.

If the subblock decoded in step S1012 is not the last subblock generated by binary tree splitting (No in S1013), the next subblock generated by binary tree splitting can be specified and input as the current block in step S1005. Yes (S1014).

If the subblock decoded in step S1012 is the last subblock generated by binary tree splitting (Yes in step S1013), whether the block to which the decoded subblock belongs is the last block among blocks generated by quadtree splitting is checked. It can be determined (S1015). In the case of the last block (Yes in S1015), decoding of the largest block input in step S1001 or the current image may be terminated. If it is not the last block (No in S1015), the next block generated by quad tree splitting can be specified and input as the current block in step S1001 (S1016).

11 is a diagram exemplarily illustrating a case in which the size of a current image is not a multiple of the size of a maximum block.

As shown in FIG. 11 , when the current image is divided into a plurality of maximum blocks, an area corresponding to a part of the maximum block size remains on the right side or bottom of the current image. That is, in the case of maximum block 2, maximum block 5, maximum block 6, maximum block 7, or maximum block 8, pixels of the current image exist only in some regions.

Hereinafter, an image division method according to the present invention for efficiently dividing a current image as shown in FIG. 11 will be described.

12A to 12E are exemplary diagrams for explaining an image segmentation method according to an embodiment of the present invention. Hereinafter, a method for dividing the largest block 2 in FIG. 11 and/or encoding of division information will be described with reference to FIGS. 12A to 12E. In the following description with reference to FIGS. 12A to 12E, it is assumed that the size of the current image is 146x146, the size of the largest block is 64x64, the minimum size of a block that can be divided into a quad tree is 16x16, and the minimum size of a sub-block is 2x2. .

12A is an enlarged view of the largest block 2, and a portion surrounded by a thick line is an area within the current image. The maximum block 2 may be a block to be divided as described with reference to FIG. 9 . As shown in FIG. 12A, the rightmost position and the lowest position of the largest block 2 are not completely included in the region of the current image. In addition, the maximum size of block 2 is 64x64, which is a size capable of quad tree partitioning. In this case, the maximum block 2 may be quad-tree partitioned (S903). That is, step S901 for the largest block 2 can be omitted. Thereafter, segmentation and encoding may be performed on block A0, which is the first block among four blocks (A0, A1, A2, and A3) generated by quad tree splitting block 2 at the maximum. Block A1 and block A3 are not completely included in the region of the current video, and thus segmentation and encoding may be omitted. Block A2, like block A0, can be segmented and coded according to the present invention.

FIG. 12B is an enlarged block A0 of FIG. 12A, and a portion surrounded by a thick line is an area within the current image. Block A0 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 12B, the rightmost position (B1) and the lowest position (B3) of block A0 are not completely included in the area of the current image. Also, the size of block A0 is 32x32, which is a size capable of partitioning a quad tree. In this case, block A0 may be quad-tree partitioned (S903). That is, step S901 for block A0 may be omitted. Thereafter, segmentation and encoding may be performed on block B0, which is the first block among four blocks (B0, B1, B2, and B3) generated by quad tree partitioning of block A0. Block B0 may be divided and coded using the method described with reference to FIG. 9 . That is, the block B0 may be coded with or without splitting by quad tree splitting and/or binary tree splitting. Block B2, like block B0, can be segmented and coded according to the present invention. Block B1 and block B3 are not completely included in the region of the current video, and thus segmentation and encoding may be omitted. Block B2 can be divided and coded in the same way as block B0. That is, step S901 cannot be omitted for blocks B0 and B2. However, for blocks B1 and B3, as will be described later, step S901 can be omitted.

FIG. 12C is an enlarged view of block B1 of FIG. 12B, and a portion surrounded by a thick line is an area within the current image. Block B1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 12C, the rightmost position and the lowest position of block B1 are not completely included in the area of the current image. In addition, the size of block B1 is 16x16, which is a size in which quad tree partitioning is impossible. In this case, block B1 cannot be quad-tree partitioned, but can be binary-tree partitioned. Also, since the encoding target block is a vertically long rectangle, the division direction may be determined in the vertical direction. Therefore, steps S901 to S907 may be omitted for the block B1, and binary tree splitting in step S909 or S910 may be performed according to the shape of the encoding target block. Thereafter, segmentation and encoding may be performed on block C1, which is a first block among two blocks C1 and C2 generated by binary tree partitioning of block B1. Since block C2 is not completely included in the region of the current video, segmentation and encoding may be omitted.

FIG. 12D is an enlarged view of block C1 of FIG. 12C , and a portion surrounded by a thick line is an area within the current image. Block C1 may be a division target block described with reference to FIG. 9 . As shown in FIG. 12D, the rightmost position and the lowest position of block C1 are not completely included in the area of the current image. In addition, since binary tree splitting is applied to block B1, which is an upper block of block C1, block C1 may also be binary tree split. Also, since the encoding target block is a vertically long rectangle, the division direction may be determined in the vertical direction. Accordingly, for the block C1, steps S905 to S907 may be omitted, and binary tree splitting of step S909 or S910 may be performed according to the shape of the encoding target block. Thereafter, segmentation and encoding may be performed on block D1, which is a first block among two blocks D1 and D2 generated by binary tree partitioning of block C1. Since the block D2 is not completely included in the region of the current image, segmentation and encoding may be omitted.

FIG. 12E is an enlarged view of block D1 of FIG. 12D, and a portion surrounded by a thick line is an area within the current image. Block D1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 12E, the rightmost position and the lowest position of block D1 are not completely included in the area of the current image. Also, since binary tree splitting is applied to block C1, which is an upper block of block D1, block D1 may also be binary tree split. Also, since the encoding target block is a vertically long rectangle, the division direction may be determined in the vertical direction. Accordingly, for the block D1, steps S905 to S907 may be omitted, and binary tree splitting of step S909 or S910 may be performed according to the shape of the encoding target block. Thereafter, segmentation and encoding may be performed on block E1, which is a first block among two blocks E1 and E2 generated by binary tree partitioning of block D1. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical division cannot be performed on the block E1. That is, the block E1 may be encoded without being split or encoded after being horizontally split. In this case, only information on whether to divide block E1 may be signaled. Since block E2 is not completely included in the region of the current video, segmentation and encoding may be omitted.

The partition information omitting method of the largest block 2 and the largest block 5 shown in FIG. 11 may be the same. In addition, the method for omitting partition information of the largest block 6 and the largest block 7 may be the same as the method of omitting partition information of the largest block 2 and maximum block 5 except for the horizontal or vertical direction. For the largest block 8, division information omission may be performed in the same manner as for other largest blocks by using a horizontally or vertically preset criterion.

Hereinafter, with reference to FIGS. 12A to 12E , a method for dividing the largest block 2 in FIG. 11 and/or decoding of division information will be described.

12A is an enlarged view of the largest block 2, and a portion surrounded by a thick line is an area within the current image. The maximum block 2 may be a block to be divided as described with reference to FIG. 9 . As shown in FIG. 12A, the rightmost position and the lowest position of the largest block 2 are not completely included in the region of the current image. In addition, the maximum size of block 2 is 64x64, which is a size capable of quad tree partitioning. In this case, the maximum block 2 may be quad-tree partitioned (S1003). That is, step S1001 for the largest block 2 can be omitted. Thereafter, partitioning and decoding may be performed on block A0, which is the first block among four blocks (A0, A1, A2, and A3) generated by quad tree splitting block 2 at the maximum. Block A1 and block A3 are not completely included in the region of the current video, and thus segmentation and decoding may be omitted. Block A2 can be segmented and decoded according to the present invention like block A0.

FIG. 12B is an enlarged block A0 of FIG. 12A, and a portion surrounded by a thick line is an area within the current image. Block A0 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 12B, the rightmost position and the lowest position of block A0 are not completely included in the area of the current video. Also, the size of block A0 is 32x32, which is a size capable of partitioning a quad tree. In this case, block A0 may be quad-tree partitioned (S1003). That is, step S1001 for block A0 may be omitted. Thereafter, partitioning and decoding may be performed on block B0, which is the first block among four blocks (B0, B1, B2, and B3) generated by quad tree partitioning of block A0. Block B0 may be divided and decoded using the method described with reference to FIG. 9 . That is, the block B0 may be decoded with or without splitting by quad tree splitting and/or binary tree splitting. Block B2 can be segmented and decoded according to the present invention as well as block B0. Block B1 and block B3 are not completely included in the region of the current video, and thus segmentation and decoding may be omitted. Block B2 can be divided and decoded in the same way as block B0. That is, step S1001 cannot be omitted for block B0 and block B2. However, for blocks B1 and B3, as will be described later, step S1001 can be omitted.

FIG. 12C is an enlarged view of block B1 of FIG. 12B, and a portion surrounded by a thick line is an area within the current image. Block B1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 12C, the rightmost position and the lowest position of block B1 are not completely included in the area of the current image. In addition, the size of block B1 is 16x16, which is a size in which quad tree partitioning is impossible. In this case, block B1 cannot be quad-tree partitioned, but can be binary-tree partitioned. Also, since the decoding object block is a vertically long rectangle, the division direction may be determined in the vertical direction. Therefore, steps S1001 to S1007 may be omitted for block B1, and binary tree splitting of step S1009 or S1010 may be performed according to the shape of the block to be decoded. Thereafter, partitioning and decoding may be performed on block C1, which is the first block among two blocks C1 and C2 generated by binary tree partitioning of block B1. Since block C2 is not completely included in the region of the current video, segmentation and decoding may be omitted.

FIG. 12D is an enlarged view of block C1 of FIG. 12C , and a portion surrounded by a thick line is an area within the current image. Block C1 may be a division target block described with reference to FIG. 9 . As shown in FIG. 12D, the rightmost position and the lowest position of block C1 are not completely included in the area of the current image. In addition, since binary tree splitting is applied to block B1, which is an upper block of block C1, block C1 may also be binary tree split. Also, since the decoding object block is a vertically long rectangle, the division direction may be determined in the vertical direction. Therefore, steps S1005 to S1007 may be omitted for the block C1, and binary tree splitting of step S1009 or S1010 may be performed according to the shape of the block to be decoded. Thereafter, partitioning and decoding may be performed on block D1, which is the first block among two blocks D1 and D2 generated by binary tree partitioning of block C1. Since block D2 is not completely included in the region of the current video, segmentation and decoding may be omitted.

FIG. 12E is an enlarged view of block D1 of FIG. 12D , and a portion surrounded by a thick line is an area within the current image. Block D1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 12E, the rightmost position and the lowest position of block D1 are not completely included in the area of the current image. Also, since binary tree splitting is applied to block C1, which is an upper block of block D1, block D1 may also be binary tree split. Also, since the decoding object block is a vertically long rectangle, the division direction may be determined in the vertical direction. Accordingly, for the block D1, steps S1005 to S1007 may be omitted, and binary tree splitting of step S1009 or S1010 may be performed according to the shape of the block to be decoded. Thereafter, partitioning and decoding may be performed on block E1, which is a first block among two blocks E1 and E2 generated by binary tree partitioning of block D1. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical division cannot be performed on the block E1. That is, the block E1 may be decoded without being split or decoded after being horizontally split. In this case, only information on whether to divide block E1 may be signaled. Since the block E2 is not completely included in the region of the current video, segmentation and decoding may be omitted.

In the embodiment described with reference to FIG. 12 , binary tree splitting may be performed when a predetermined condition is satisfied after quad tree splitting is performed on a split target block. In the above embodiment, whether or not the size of the block to be split is a size capable of splitting a quad tree corresponds to the predetermined condition. The predetermined condition may be set using a minimum size and/or threshold of a block capable of quad tree splitting and a block capable of binary tree splitting.

When a threshold value is used, one of quad tree division and binary tree division based on the result of comparing the threshold value with the size of a region not completely included in the current image (hereinafter referred to as “remaining region”) among blocks to be divided. can be performed. In the following description, it is assumed that the threshold value is 32.

For example, in FIG. 12A, the size of the encoding target block is 18x64, and the size of the remaining area is 46x64. Since the remaining area is a vertically long rectangle, the horizontal length of the remaining area and the threshold can be compared. Since the horizontal length of the remaining region is 46, which is greater than the threshold value of 32, quad tree partitioning can be performed on a 64x64 partitioning target block. In FIG. 12B, since the horizontal length of the remaining area is 14, which is smaller than the threshold value of 32, binary tree splitting can be performed on the 32x32 split target block. That is, even if the size of the block to be split is such that quad tree splitting is possible, binary tree splitting may be performed instead of quad tree splitting according to the above condition. At this time, the splitting direction of the binary tree splitting is based on the smaller one of the horizontal and vertical sides of the 18x32 encoding target block. Thus, block A0 of FIG. 12B can be divided into two vertically long 16x32 blocks. Thereafter, binary tree partitioning in the vertical direction may be repeatedly performed until the block size is such that binary tree partitioning is not possible.

The threshold value may be signaled through a higher-level header of the aforementioned block. The threshold value must be a value between the maximum size and minimum size of a block capable of quad tree splitting. Also, the threshold value may be set smaller than the maximum size of a block capable of binary tree partitioning. When the threshold value is set to be greater than the maximum size of a block capable of binary tree partitioning, the threshold value may be changed to the maximum size of a block capable of binary tree partitioning.

13A to 13E are exemplary diagrams for explaining an image segmentation method according to another embodiment of the present invention.

13A is an enlarged view of the largest block 2, and a portion surrounded by a thick line is an area within the current image. The maximum block 2 may be a block to be divided as described with reference to FIG. 9 . As shown in FIG. 13A, the rightmost position and the lowest position of the largest block 2 are not completely included in the region of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13A is a vertically long 46x64 block, binary tree partitioning in the vertical direction can be performed for the maximum block 2. That is, steps S901 to S907 for block 2 may be omitted, and steps S909 or S910 may be performed according to the shape of the remaining area.

FIG. 13B is an enlarged view of block A0 of FIG. 13A, and a portion surrounded by a thick line is an area within the current image. Block A0 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 13B, the rightmost position and the lowest position of block A0 are not completely included in the area of the current video. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13B is a vertically long 14x64 block, binary tree partitioning in the vertical direction can be performed on the block A0. Since block B0 is a block within the current video region, it can be coded using sub-block partitioning. Steps S905 to S908 cannot be omitted for block B0.

FIG. 13C is an enlarged block B1 of FIG. 13B , and a portion surrounded by a thick line is an area within the current image. Block B1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 13C, the rightmost position and the lowest position of block B1 are not completely included in the area of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13C is a vertically long 14x64 block, binary tree partitioning in the vertical direction can be performed on block B1. That is, steps S901 to S907 for block B1 may be omitted, and steps S909 or S910 may be performed according to the shape of the remaining area. Since block C2 is not completely included in the region of the current video, segmentation and encoding may be omitted.

FIG. 13D is an enlarged block C1 of FIG. 13C , and a portion surrounded by a thick line is an area within the current image. Block C1 may be a division target block described with reference to FIG. 9 . As shown in FIG. 13D, the rightmost position and the lowest position of block C1 are not completely included in the area of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13C is a vertically long 6x64 block, binary tree partitioning in the vertical direction can be performed on block C1. That is, steps S901 to S907 for block C1 may be omitted, and steps S909 or S910 may be performed according to the shape of the remaining area. Since the block D2 is not completely included in the region of the current image, segmentation and encoding may be omitted.

FIG. 13E is an enlarged view of block D1 of FIG. 13D , and a portion surrounded by a thick line is an area within the current image. Block D1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 13E, the rightmost position and the lowest position of block D1 are not completely included in the area of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13C is a vertically long 2x64 block, binary tree partitioning in the vertical direction can be performed on the block D1. That is, steps S901 to S907 for the block D1 may be omitted, and steps S909 or S910 may be performed according to the shape of the remaining area. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical division cannot be performed on the block E1. That is, the block E1 may be encoded without being split or encoded after being horizontally split. In this case, only information on whether to divide block E1 may be signaled. Since block E2 is not completely included in the region of the current video, segmentation and encoding may be omitted.

Hereinafter, with reference to FIGS. 13A to 13E , a method for splitting the largest block 2 in FIG. 11 and/or decoding of split information will be described.

13A is an enlarged view of the largest block 2, and a portion surrounded by a thick line is an area within the current image. The maximum block 2 may be a block to be divided as described with reference to FIG. 9 . As shown in FIG. 13A, the rightmost position and the lowest position of the largest block 2 are not completely included in the region of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13A is a vertically long 46x64 block, binary tree partitioning in the vertical direction can be performed for the maximum block 2. That is, steps S1001 to S1007 for block 2 may be omitted, and steps S1009 or S1010 may be performed according to the shape of the remaining area.

FIG. 13B is an enlarged view of block A0 of FIG. 13A, and a portion surrounded by a thick line is an area within the current image. Block A0 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 13B, the rightmost position and the lowest position of block A0 are not completely included in the area of the current video. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13B is a vertically long 14x64 block, binary tree partitioning in the vertical direction can be performed on the block A0. Since block B0 is a block within the current video region, it can be decoded using sub-block partitioning. Steps S1005 to S1008 cannot be omitted for block B0.

FIG. 13C is an enlarged block B1 of FIG. 13B , and a portion surrounded by a thick line is an area within the current image. Block B1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 13C, the rightmost position and the lowest position of block B1 are not completely included in the area of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13C is a vertically long 14x64 block, binary tree partitioning in the vertical direction can be performed on block B1. That is, steps S1001 to S1007 for the block B1 may be omitted, and steps S1009 or S1010 may be performed according to the shape of the remaining area. Since block C2 is not completely included in the region of the current video, segmentation and decoding may be omitted.

FIG. 13D is an enlarged block C1 of FIG. 13C , and a portion surrounded by a thick line is an area within the current image. Block C1 may be a division target block described with reference to FIG. 9 . As shown in FIG. 13D, the rightmost position and the lowest position of block C1 are not completely included in the area of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13C is a vertically long 6x64 block, binary tree partitioning in the vertical direction can be performed on block C1. That is, steps S1001 to S1007 for block C1 may be omitted, and steps S1009 or S1010 may be performed according to the shape of the remaining area. Since block D2 is not completely included in the region of the current video, segmentation and decoding may be omitted.

FIG. 13E is an enlarged view of block D1 of FIG. 13D , and a portion surrounded by a thick line is an area within the current image. Block D1 may be a partition target block described with reference to FIG. 9 . As shown in FIG. 13E, the rightmost position and the lowest position of block D1 are not completely included in the area of the current image. In this case, a block division method may be selected based on the shape of the remaining area. Since the remaining area in FIG. 13C is a vertically long 2x64 block, binary tree partitioning in the vertical direction can be performed on the block D1. That is, steps S1001 to S1007 for the block D1 may be omitted, and steps S1009 or S1010 may be performed according to the shape of the remaining area. The horizontal size of block E1 is equal to 2, which is assumed to be the minimum size of a sub-block. Therefore, vertical division cannot be performed on the block E1. That is, the block E1 may be decoded without being split or decoded after being horizontally split. In this case, only information on whether to divide block E1 may be signaled. Since the block E2 is not completely included in the region of the current video, segmentation and decoding may be omitted.

In the embodiment described with reference to FIG. 13 , only binary tree splitting is performed on a block to be split, and the direction of binary tree splitting may be determined according to the shape of the remaining area.

In the embodiment described with reference to FIGS. 9 and 10 , segmentation processing may be performed on the boundary of an image in units of sub-blocks or units of blocks. Information on whether or not to perform division processing on the boundary of an image in which unit may be signaled. For example, the information may be signaled from an encoder to a decoder through a header of a higher level of the aforementioned block.

Exemplary methods of this disclosure are presented as a series of operations for clarity of explanation, but this is not intended to limit the order in which steps are performed, and each step may be performed concurrently or in a different order, if desired. In order to implement the method according to the present disclosure, other steps may be included in addition to the exemplified steps, other steps may be included except for some steps, or additional other steps may be included except for some steps.

Various embodiments of the present disclosure are intended to explain representative aspects of the present disclosure, rather than listing all possible combinations, and matters described in various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented by a processor (general processor), controller, microcontroller, microprocessor, or the like.

The scope of the present disclosure is software or machine-executable instructions (eg, operating systems, applications, firmware, programs, etc.) that cause operations according to methods of various embodiments to be executed on a device or computer, and such software or It includes a non-transitory computer-readable medium in which instructions and the like are stored and executable on a device or computer.

Claims (22)

a first dividing step of dividing the current image into a plurality of blocks; and
A second division step of dividing a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks;
The second dividing step is recursively performed by using a sub-block including the boundary of the current image as the division target block until there is no sub-block including the boundary of the current image among the sub-blocks. become,
The second splitting step divides the splitting block by selectively applying quad tree splitting or binary tree splitting based on the size and threshold of the splitting block,
The quad tree splitting is performed based on a result of comparing the size of the split target block with the size of a block capable of quad tree splitting,
The binary tree splitting is performed based on a result of comparing the size of the split target block with the size of a block capable of binary tree splitting and the threshold value when the quad tree splitting is not performed,
The threshold value is different from the size of a block capable of binary tree partitioning and is a value between a maximum size and a minimum size of a block capable of quad tree partitioning. According to claim 1,
In the first division step,
The step of dividing the current image into a plurality of largest blocks having the same size. According to claim 1,
If any one of the rightmost position of the division target block and the lowest position of the division target block is not included in the region of the current image,
An image encoding method for determining whether to apply binary tree partitioning to the partitioning target block based on a maximum block size capable of binary tree partitioning. ◈Claim 4 was abandoned when the registration fee was paid.◈ According to claim 1,
The video encoding method of omitting the second dividing step and encoding for a sub-block among the sub-blocks that does not include a boundary of the current video and is located outside the current video. ◈Claim 5 was abandoned when the registration fee was paid.◈ According to claim 1,
When the quad tree split or the binary tree split is performed on the split target block in the second split step,
First splitting information indicating whether the splitting target block is quad-tree splitting or second splitting information indicating whether the splitting block is binary tree splitting is not encoded. ◈Claim 6 was abandoned when the registration fee was paid.◈ According to claim 1,
When the binary tree splitting is performed on the splitting target block in the second splitting step,
The splitting direction of the binary tree splitting is determined based on which one of the rightmost position of the partitioning target block and the lowest position of the partitioning target block is not included in the region of the current video. ◈Claim 7 was abandoned when the registration fee was paid.◈ According to claim 1,
When the binary tree splitting is performed on the splitting target block in the second splitting step,
Splitting direction information indicating a splitting direction of the binary tree splitting is not encoded. delete delete delete ◈Claim 11 was abandoned when the registration fee was paid.◈ According to claim 6,
The splitting direction of the binary tree splitting is
When the rightmost position of the division target block is not included in the region of the current image, it is in the vertical direction;
When the lowermost position of the division target block is not included in the region of the current image, the video encoding method is in the horizontal direction. a first dividing step of dividing the current image into a plurality of blocks; and
A second division step of dividing a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks;
The second dividing step is recursively performed by using a sub-block including the boundary of the current image as the division target block until there is no sub-block including the boundary of the current image among the sub-blocks. become,
The second splitting step divides the splitting block by selectively applying quad tree splitting or binary tree splitting based on the size and threshold of the splitting block,
The quad tree splitting is performed based on a result of comparing the size of the split target block with the size of a block capable of quad tree splitting,
The binary tree splitting is performed based on a result of comparing the size of the split target block with the size of a block capable of binary tree splitting and the threshold value when the quad tree splitting is not performed,
The threshold value is different from the size of a block capable of partitioning the binary tree and is a value between a maximum size and a minimum size of a block capable of quad tree partitioning. According to claim 12,
If any one of the rightmost position of the division target block and the lowest position of the division target block is not included in the region of the current image,
An image decoding method for determining whether to apply binary tree partitioning to the partition target block based on a maximum block size capable of binary tree partitioning. According to claim 12,
The video decoding method of omitting the second dividing step and decoding for a sub-block that does not include a boundary of the current video among the sub-blocks and is located outside the current video. ◈Claim 15 was abandoned when the registration fee was paid.◈ According to claim 12,
When the quad tree split or the binary tree split is performed on the split target block in the second split step,
The first splitting information indicating whether the splitting target block is quad-tree splitting or the second splitting information indicating whether the splitting target block is binary tree splitting is not decoded from the bitstream and is derived as a predetermined value. decryption method. ◈Claim 16 was abandoned when the registration fee was paid.◈ According to claim 12,
When the binary tree splitting is performed on the splitting target block in the second splitting step,
The splitting direction of the binary tree splitting is determined based on which one of the rightmost position of the partitioning target block and the lowest position of the partitioning target block is not included in the region of the current video. ◈Claim 17 was abandoned when the registration fee was paid.◈ According to claim 12,
When the binary tree splitting is performed on the splitting target block in the second splitting step,
The splitting direction information indicating the splitting direction of the binary tree splitting is not decoded from the bitstream and derived as a predetermined value. delete delete delete ◈Claim 21 was abandoned when the registration fee was paid.◈ According to claim 16,
The splitting direction of the binary tree splitting is
When the rightmost position of the division target block is not included in the region of the current image, it is in the vertical direction;
When the lowermost position of the division target block is not included in the region of the current image, the image decoding method is in the horizontal direction. A computer-readable recording medium storing a bitstream generated by an image encoding method,
The video encoding method,
a first dividing step of dividing the current image into a plurality of blocks; and
A second division step of dividing a division target block including a boundary of the current image among the plurality of blocks into a plurality of sub-blocks;
The second dividing step is recursively performed by using a sub-block including the boundary of the current image as the division target block until there is no sub-block including the boundary of the current image among the sub-blocks. become,
The second splitting step divides the splitting block by selectively applying quad tree splitting or binary tree splitting based on the size and threshold of the splitting block,
The quad tree splitting is performed based on a result of comparing the size of the split target block with the size of a block capable of quad tree splitting,
The binary tree splitting is performed based on a result of comparing the size of the split target block with the size of a block capable of binary tree splitting and the threshold value when the quad tree splitting is not performed,
The threshold value is different from the size of a block capable of binary tree partitioning and is a value between a maximum size and a minimum size of a block capable of quad tree partitioning.

Images