KR102520405B1 - Method of processing video, video encoding and decoding thereof - Google Patents

Abstract

An image processing method according to the present invention includes the steps of dividing a picture of an image into a plurality of coding units, which are basic units in which inter prediction or intra prediction is performed; and selectively constructing a prediction mode list for deriving a prediction direction of the decoding object block from an intra prediction direction of neighboring blocks adjacent to the decoding object block, for an intra-predicted unit among the divided coding units. And, the coding unit splitting step includes splitting the picture or the split coding unit into a binary tree structure.

Description

Video processing method, video decoding and encoding method using the same {Method of processing video, video encoding and decoding thereof}

The present invention relates to video encoding and decoding, and more particularly, to a method of performing intra prediction by dividing a video picture into a plurality of blocks.

In the video compression method, encoding is performed by dividing one picture into a plurality of blocks having a predetermined size. In addition, inter-prediction and intra-prediction techniques that remove redundancy between pictures are used to increase compression efficiency.

In this case, a residual signal is created using intra prediction and inter prediction, and the reason for obtaining the residual signal is that the amount of data is small when coding with the residual signal, so the data compression rate increases, and the better the prediction, the better the residual signal. because the value of is small.

In the intra prediction method, data of a current block is predicted using pixels around the current block. The difference between the actual value and the predicted value is called a residual signal block. In the case of HEVC, the intra prediction method is increased from 9 prediction modes used in the existing H.264/AVC to 35 prediction modes, and prediction is further subdivided.

In the case of the inter-prediction method, the most similar block is found by comparing the current block with blocks in neighboring pictures. At this time, the location information (Vx, Vy) of the found block is called a motion vector. A difference between pixel values within a block between a current block and a prediction block predicted by a motion vector is called a residual signal block (motion-compensated residual block).

In this way, although intra prediction and inter prediction are further subdivided, the amount of residual signal data is reduced, but the amount of computation for video processing is greatly increased.

In particular, there are difficulties in pipeline implementation due to the increased complexity in the process of determining the division structure within a picture for video encoding and decoding, and the existing block division method and the size of the blocks divided accordingly It may not be suitable for coding.

The present invention is to solve the above problems, and an object of the present invention is to provide an image processing method suitable for encoding and decoding of a high-resolution image, and an image decoding and encoding method using the same.

In particular, in the configuration of reference samples for intra prediction, padding of the nearest sample to an area where there is no pre-decoded reference sample is an element that hinders directional prediction, which can prevent a decrease in coding performance Its purpose is to provide an image processing method and an image decoding and encoding method using the same.

An image processing method according to an embodiment of the present invention for solving the above problems is a basic unit in which inter prediction or intra prediction is performed on a picture of an image in the image processing method. Dividing into a plurality of coding units; identifying a reference sample used for intra prediction for a unit of the current block that is intra-predicted among the divided coding units; performing padding processing in which a variation amount of a neighboring block of the current block is reflected on a reference sample for which no previously decoded information is present among the reference samples; and performing intra prediction decoding based on the padded reference sample.

An image decoding method according to an embodiment of the present invention includes receiving an encoded bitstream; obtaining a residual block by performing inverse quantization and inverse transformation on the input bitstream; obtaining a prediction block by performing inter prediction or intra prediction; and reconstructing an image by adding the obtained residual block and the prediction block, wherein a coding unit, which is a basic unit in which the inter prediction or intra prediction is performed, is a block divided from a coding tree unit using a binary tree structure, wherein the The obtaining of the prediction block may include: identifying a reference sample used for intra prediction for a unit of the current block that is intra-predicted among the divided coding units; and performing a padding process in which a variation amount of a neighboring block of the current block is reflected, with respect to a reference sample having no pre-decoded information among the reference samples. and obtaining the prediction block by performing intra prediction decoding based on the padded reference sample.

On the other hand, the above methods can be implemented in a computer-readable recording medium on which a program to be executed by a computer is recorded.

According to an embodiment of the present invention, in the configuration of reference samples for intra prediction, more accurate prediction is achieved by constructing reference samples to which variations of the current block or neighboring blocks are applied to a region where previously decoded reference samples do not exist. and improve coding efficiency for high-resolution images.

1 is a block diagram showing the configuration of an image encoding apparatus according to an embodiment of the present invention.
2 to 5 are diagrams for explaining a first embodiment of a method of dividing and processing an image into blocks.
6 is a block diagram for explaining an embodiment of a method of performing inter prediction in an image encoding apparatus.
7 is a block diagram showing the configuration of a video decoding apparatus according to an embodiment of the present invention.
8 is a block diagram for explaining an embodiment of a method of performing inter prediction in a video decoding apparatus.
9 is a diagram for explaining a second embodiment of a method of dividing and processing an image into blocks.
10 is a diagram illustrating an embodiment of a syntax structure used to divide and process an image into blocks.
11 is a diagram for explaining a third embodiment of a method of dividing and processing an image into blocks.
12 is a diagram for explaining an embodiment of a method of constructing a transform unit by dividing a coding unit into a binary tree structure.
13 is a diagram for explaining a fourth embodiment of a method of dividing and processing an image into blocks.
14 to 16 are diagrams for explaining still other embodiments of a method of dividing and processing an image into blocks.
17 and 18 are diagrams for explaining embodiments of a method of determining a division structure of a conversion unit by performing rate distortion optimization (RDO).
19 is a diagram for explaining a configuration of a reference sample to which a variation amount is applied according to an embodiment of the present invention.
20 to 21 are flowcharts for describing an intra prediction process according to an embodiment of the present invention.
22 is a flowchart illustrating a process of generating a sample reflecting a single amount of change according to an embodiment of the present invention.
23 to 24 are exemplary diagrams of a single variation reflecting sample according to an embodiment of the present invention.
25 is a flowchart illustrating a sample padding process reflecting a single variation amount according to an embodiment of the present invention.
26 is a flowchart illustrating a process of generating samples reflecting multiple variations according to an embodiment of the present invention.
27 to 28 are exemplary diagrams of multiple change amount reflecting samples according to an embodiment of the present invention.
29 is a flowchart illustrating a selective single and multiple variation reflecting sample padding process according to an embodiment of the present invention.
30 and 31 are diagrams for explaining a reference sample non-existence case to which a variation-reflecting sample padding process according to an embodiment of the present invention is applicable.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described concretely with reference to drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description thereof will be omitted.

It is understood that when a component is referred to as being "connected" or "connected" to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. In addition, the description of "including" a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and means that additional configurations may be included in the practice of the present invention or the scope of the technical spirit of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

In addition, components appearing in the embodiments of the present invention are shown independently to represent different characteristic functions, and do not mean that each component is made of separate hardware or a single software component unit. That is, each component is listed and included as each component for convenience of explanation, and at least two components of each component can be combined to form a single component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separated embodiments of components are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

In addition, some of the components may be optional components for improving performance rather than essential components that perform essential functions in the present invention. The present invention can be implemented by including only components essential to implement the essence of the present invention, excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement. Also included in the scope of the present invention.

1 is a block diagram showing the configuration of an image encoding apparatus according to an embodiment of the present invention. The image encoding apparatus 10 includes a picture division unit 110, a transform unit 120, a quantization unit 130, a scanning unit 131, entropy encoding unit 140, intra prediction unit 150, inter prediction unit 160, inverse quantization unit 135, inverse transformation unit 125, post-processing unit 170, picture storage unit 180 ), a subtraction unit 190 and an addition unit 195 are included.

Referring to FIG. 1 , the picture divider 110 analyzes an input video signal, divides the picture into coding units, determines a prediction mode, and determines the size of the prediction unit for each coding unit.

Also, the picture division unit 110 sends a prediction unit to be encoded to the intra prediction unit 150 or the inter prediction unit 160 according to a prediction mode (or prediction method). In addition, the picture division unit 110 sends a prediction unit to be encoded to the subtraction unit 190.

Here, a picture of an image is composed of a plurality of slices, and a slice may be divided into a plurality of Coding Tree Units (CTUs), which are basic units for dividing a picture.

The coding tree unit may be divided into one or more coding units (CUs), which are basic units in which inter prediction or intra prediction is performed.

A coding unit (CU) may be divided into one or more prediction units (PUs), which are basic units in which prediction is performed.

In this case, the encoding apparatus 10 determines either inter prediction or intra prediction as a prediction method for each of the divided coding units (CUs), but predicts a prediction block differently for each prediction unit (PU). can create

Meanwhile, a coding unit (CU) may be divided into one or two or more transform units (TUs), which are basic units in which transformation of a residual block is performed.

In this case, the picture segmentation unit 110 may transmit image data to the subtraction unit 190 in units of blocks divided as described above (eg, a prediction unit (PU) or a transformation unit (TU)).

Referring to FIG. 2 , a coding tree unit (CTU) having a maximum size of 256x256 pixels may be divided into four coding units (CUs) having a square shape by dividing into a quad tree structure.

Each of the four coding units (CUs) having the shape of a square may be subdivided into a quad tree structure, and the depth (Depth) of the coding units (CUs) split into the quad tree structure as described above is any one of 0 to 3. It can have a single integer value.

A coding unit (CU) may be divided into one or two or more prediction units (PUs) according to a prediction mode.

In the case of the intra prediction mode, when the size of the coding unit (CU) is 2Nx2N, the prediction unit (PU) may have a size of 2Nx2N shown in (a) of FIG. 3 or NxN shown in (b) of FIG. 3 there is.

Meanwhile, in the case of the inter prediction mode, when the size of the coding unit (CU) is 2Nx2N, the prediction unit (PU) is 2Nx2N shown in (a) of FIG. 4, 2NxN shown in (b) of FIG. Nx2N shown in (c) of (c), NxN shown in (d) of FIG. 4, 2NxnU shown in (e) of FIG. 4, 2NxnD shown in (f) of FIG. 4, shown in (g) of FIG. nLx2N and nRx2N shown in (h) of FIG. 4.

Referring to FIG. 5 , a coding unit (CU) may be divided into four transform units (TUs) having a quad tree structure and having a square shape.

Each of the four transform units (TUs) having the shape of a square may be re-divided into a quad tree structure, and the depth of each transform unit (TU) divided into the quad tree structure as described above is any one of 0 to 3. It can have a single integer value.

Here, when the coding unit (CU) is in the inter prediction mode, the prediction unit (PU) and the transform unit (TU) divided from the corresponding coding unit (CU) may have a partition structure independent of each other.

When a coding unit (CU) is in intra prediction mode, a transform unit (TU) divided from the corresponding coding unit (CU) cannot be larger than the size of the prediction unit (PU).

In addition, the transform unit (TU) divided as described above may have a maximum size of 64x64 pixels.

The transform unit 120 transforms a residual block, which is a residual signal between an input original block of the prediction unit (PU) and a prediction block generated by the intra predictor 150 or the inter predictor 160, and the transform is It may be performed using a unit (TU) as a basic unit.

In the transformation process, different transformation matrices may be determined according to the prediction mode (intra or inter), and since the residual signal of intra prediction has a direction according to the intra prediction mode, the transformation matrix may be adaptively determined according to the intra prediction mode. there is.

The transformation unit may be transformed by two (horizontal and vertical) one-dimensional transformation matrices, and for example, in the case of inter prediction, one predetermined transformation matrix may be determined.

On the other hand, in the case of intra prediction, if the intra prediction mode is horizontal, since the probability that the residual block has a vertical direction increases, a DCT-based integer matrix is applied in the vertical direction, and a DST-based or DST-based integer matrix is applied in the horizontal direction. Apply KLT-based integer matrix. When the intra prediction mode is vertical, a DST-based or KLT-based integer matrix may be applied in the vertical direction, and a DCT-based integer matrix may be applied in the horizontal direction.

In addition, in the case of the DC mode, a DCT-based integer matrix can be applied to both directions.

And, in the case of intra prediction, a transform matrix may be adaptively determined based on the size of a transform unit (TU).

The quantization unit 130 determines a quantization step size for quantizing coefficients of the residual block transformed by the transformation matrix, and the quantization step size may be determined for each quantization unit having a predetermined size or larger.

The size of the quantization unit may be 8x8 or 16x16, and the quantization unit 130 quantizes the coefficients of the transform block using a quantization matrix determined according to a quantization step size and a prediction mode.

Also, the quantization unit 130 may use a quantization step size of a quantization unit adjacent to the current quantization unit as a quantization step size predictor of the current quantization unit.

The quantization unit 130 searches the left quantization unit, the upper quantization unit, and the upper left quantization unit of the current quantization unit in order, and generates a quantization step size predictor of the current quantization unit using one or two effective quantization step sizes. there is.

For example, the quantization unit 130 determines the first effective quantization step size retrieved in the above order as the quantization step size predictor, or determines the average value of two valid quantization step sizes retrieved in the above order as the quantization step size predictor, or If only one quantization step size is valid, this can be determined as a quantization step size predictor.

When the quantization step size predictor is determined, the quantization unit 130 transmits a difference value between the quantization step size predictor of the current quantization unit and the quantization step size predictor to the entropy encoding unit 140 .

Meanwhile, all of the left coding unit, the above coding unit, and the top left coding unit of the current coding unit do not exist. Alternatively, a coding unit previously present in the coding order within the largest coding unit may exist.

Therefore, in quantization units adjacent to the current coding unit and the maximum coding unit, a quantization step size of a quantization unit immediately preceding in coding order may be a candidate.

In this case, priorities are set in the order of 1) the left quantization unit of the current coding unit, 2) the upper quantization unit of the current coding unit, 3) the upper left quantization unit of the current coding unit, and 4) the immediately previous quantization unit in the coding order. It can be. The order may be reversed, and the upper left quantization unit may be omitted.

Meanwhile, the quantized transform block as described above is transmitted to the inverse quantization unit 135 and the scanning unit 131.

The scanning unit 131 scans the coefficients of the quantized transform block and converts them into 1-dimensional quantization coefficients. In this case, since the coefficient distribution of the transform block after quantization may be dependent on the intra prediction mode, the scanning method is applied to the intra prediction mode. can be determined according to

In addition, the coefficient scanning method may be determined differently according to the size of the transform unit, and the scan pattern may be changed according to the directional intra prediction mode. In this case, the scan order of quantization coefficients may be scanned in the reverse direction.

When the quantized coefficients are divided into a plurality of subsets, the same scan pattern may be applied to the quantization coefficients in each subset, and zigzag scan or diagonal scan may be applied to the scan pattern between the subsets.

Meanwhile, the scan pattern is preferably scanned from the main subset including DC to the remaining subsets in a forward direction, but the reverse direction is also possible.

Also, a scan pattern between subsets may be set identically to a scan pattern of quantized coefficients within the subset, and the scan pattern between subsets may be determined according to an intra prediction mode.

On the other hand, the encoding device 10 includes information indicating the position of the last non-zero quantization coefficient in the transform unit PU and the position of the last non-zero quantization coefficient in each subset in a bitstream so that the decoding device ( 20) can be sent.

The inverse quantization unit 135 inversely quantizes the quantized coefficients quantized as described above, and the inverse transformation unit 125 performs inverse transformation in units of transform units (TUs) to restore the inversely quantized transform coefficients into residual blocks in the spatial domain. can do.

The adder 195 may generate a reconstructed block by adding the residual block reconstructed by the inverse transform unit 125 and the prediction block received from the intra predictor 150 or the inter predictor 160.

In addition, the post-processing unit 170 performs a deblocking filtering process to remove a blocking effect occurring in the reconstructed picture, and a sample adaptive offset (Sample Adaptive Offset: A SAO) application process and an adaptive loop filtering (ALF) process for compensating for a difference value from an original image may be performed by a coding unit.

The deblocking filtering process may be applied to a boundary of a prediction unit (PU) or a transform unit (TU) having a size greater than or equal to a predetermined size.

For example, the deblocking filtering process may include determining a boundary to be filtered, determining a boundary filtering strength to be applied to the boundary, determining whether to apply a deblocking filter, When it is determined to apply the deblocking filter, selecting a filter to be applied to the boundary may be included.

On the other hand, whether or not the deblocking filter is applied depends on i) whether the boundary filtering strength is greater than 0 and ii) the degree of change in pixel values at the boundary of two blocks (P block, Q block) adjacent to the boundary to be filtered. It may be determined by whether the indicated value is smaller than the first reference value determined by the quantization parameter.

It is preferable that the said filter is at least two or more. When the absolute value of the difference between two pixels located at the block boundary is greater than or equal to the second reference value, a filter that performs relatively weak filtering is selected.

The second reference value is determined by the quantization parameter and the boundary filtering strength.

In addition, the sample adaptive offset (SAO) application process is to reduce the distortion between pixels in the image to which the deblocking filter is applied and original pixels, and the sample adaptive offset (SAO) application process is performed in units of pictures or slices. It can be decided whether or not to do so.

A picture or slice may be divided into a plurality of offset regions, and an offset type may be determined for each offset region, and the offset type includes a predetermined number (eg, 4) of edge offset types and two band offsets. Can contain types.

For example, if the offset type is an edge offset type, an edge type to which each pixel belongs is determined and a corresponding offset is applied, and the edge type may be determined based on a distribution of values of two pixels adjacent to the current pixel. there is.

The adaptive loop filtering (ALF) process may perform filtering based on a value obtained by comparing a reconstructed image that has undergone a deblocking filtering process or an adaptive offset application process with an original video.

The picture storage unit 180 receives post-processed image data from the post-processing unit 170 and restores and stores the image in units of pictures. The picture may be an image in units of frames or an image in units of fields.

The inter prediction unit 160 may perform motion estimation using at least one reference picture stored in the picture storage unit 180, and may determine a reference picture index and a motion vector representing the reference picture.

In this case, a prediction block corresponding to a prediction unit to be encoded may be extracted from a reference picture used for motion estimation among a plurality of reference pictures stored in the picture storage 180 according to the determined reference picture index and motion vector. there is.

The intra prediction unit 150 may perform intra prediction encoding using reconstructed pixel values inside a picture including a current prediction unit.

The intra prediction unit 150 may perform intra prediction by receiving a current prediction unit to be predictively encoded and selecting one of a preset number of intra prediction modes according to the size of a current block.

The intra predictor 150 may adaptively filter reference pixels to generate an intra prediction block, and generate reference pixels using available reference pixels when a reference pixel is not available.

The entropy encoding unit 140 may entropy encode the quantization coefficients quantized by the quantization unit 130, the intra prediction information received from the intra prediction unit 150, and the motion information received from the inter prediction unit 160. .

6 is a block diagram showing an embodiment of a configuration for performing inter prediction in the encoding device 10. The inter prediction encoder shown includes a motion information determining unit 161 and a motion information encoding mode determining unit 162. , a motion information encoder 163, a prediction block generator 164, a residual block generator 165, a residual block encoder 166, and a multiplexer 167.

Referring to FIG. 6 , the motion information determination unit 161 determines motion information of a current block, the motion information includes a reference picture index and a motion vector, and the reference picture index is one of previously encoded and reconstructed pictures. can represent

A reference picture indicating one of the reference pictures belonging to list 0 (L0) when the current block is unidirectional inter-prediction coded, and indicating one of the reference pictures of list 0 (L0) when the current block is bi-directional predictive coded. index and a reference picture index indicating one of the reference pictures of List 1 (L1).

In addition, when the current block is bi-directionally predicted-coded, it may include indices indicating one or two pictures among reference pictures of a composite list (LC) generated by combining list 0 and list 1.

A motion vector represents a position of a prediction block within a picture indicated by each reference picture index, and the motion vector may be in a pixel unit (integer unit) or a sub-pixel unit.

For example, the motion vector may have a resolution of 1/2, 1/4, 1/8, or 1/16 pixels, and if the motion vector is not an integer unit, a prediction block may be generated from pixels in integer units. can

The motion information encoding mode determination unit 162 may determine an encoding mode for the motion information of the current block as one of a skip mode, a merge mode, and an AMVP mode.

The skip mode is applied when there is a skip candidate having the same motion information as that of the current block and the residual signal is 0. can be applied when is equal to

The merge mode is applied when there is a merge candidate having the same motion information as the motion information of the current block. applies in case Meanwhile, the merge candidate and the skip candidate may be the same.

The AMVP mode is applied when the skip mode and merge mode are not applied, and an AMVP candidate having a motion vector most similar to that of the current block may be selected as an AMVP predictor.

The motion information encoding unit 163 may encode the motion information according to the method determined by the motion information encoding mode determination unit 162 .

For example, the motion information encoder 163 may perform a merge motion vector encoding process when the motion information encoding mode is a skip mode or a merge mode, and may perform an AMVP encoding process when the motion information encoding mode is the AMVP mode.

The prediction block generation unit 164 generates a prediction block using motion information of the current block, and if the motion vector is an integer unit, copies a block corresponding to a position indicated by the motion vector in the picture indicated by the reference picture index to copy the current block. Generates a prediction block of

Meanwhile, when the motion vector is not an integer unit, the prediction block generator 164 may generate pixels of a prediction block from pixels of an integer unit in a picture indicated by a reference picture index.

In this case, a predicted pixel may be generated using an 8-tap interpolation filter for a luminance pixel, and a predicted pixel may be generated using a 4-tap interpolation filter for a chrominance pixel.

The residual block generation unit 165 generates a residual block using a current block and a prediction block of the current block, and when the size of the current block is 2Nx2N, a residual block is generated using the current block and a prediction block having a size of 2Nx2N corresponding to the current block. blocks can be created.

Meanwhile, when the size of the current block used for prediction is 2NxN or Nx2N, after obtaining a prediction block for each of two 2NxN blocks constituting 2Nx2N, the final prediction block of size 2Nx2N is obtained by using the two 2NxN prediction blocks. can be created

In addition, a 2Nx2N-sized residual block may be generated using the 2Nx2N-sized prediction block, and overlap smoothing is applied to pixels of the boundary portion in order to resolve the discontinuity of the boundary portion of the two 2NxN-sized prediction blocks. can

The residual block encoder 166 divides the residual block into one or more transform units (TUs), and each transform unit (TU) can be transform-encoded, quantized, and entropy-encoded.

The residual block encoder 166 may transform the residual block generated by the inter prediction method using an integer-based transform matrix, and the transform matrix may be an integer-based DCT matrix.

Meanwhile, the residual block encoder 166 uses a quantization matrix to quantize coefficients of the residual block transformed by the transformation matrix, and the quantization matrix may be determined by a quantization parameter.

The quantization parameter is determined for each coding unit (CU) having a predetermined size or more, and when the current coding unit (CU) is smaller than the predetermined size, the first coding unit (CU) among coding units (CUs) within the predetermined size in coding order ( Since only the quantization parameters of the CU) are coded and the quantization parameters of the remaining coding units (CUs) are identical to the above parameters, coding may not be performed.

Also, the coefficients of the transform block may be quantized using a quantization matrix determined according to the quantization parameter and the prediction mode.

A quantization parameter determined for each coding unit (CU) having a predetermined size or larger may be predicted and coded using a quantization parameter of a coding unit (CU) adjacent to the current coding unit (CU).

A quantization parameter predictor of the current coding unit (CU) may be generated using one or two valid quantization parameters by searching in the order of the left coding unit (CU) and the upper coding unit (CU) of the current coding unit (CU). there is.

For example, the first valid quantization parameter searched in the above order may be determined as the quantization parameter predictor, and the first valid quantization parameter may be quantized by searching in the order of the left coding unit (CU) and the immediately previous coding unit (CU) in the coding order. It can be determined as a parameter predictor.

Coefficients of the quantized transform block are scanned and converted into one-dimensional quantization coefficients, and a scanning method may be set differently according to an entropy encoding mode.

For example, in the case of CABAC encoding, inter-prediction coded quantization coefficients may be scanned in one predetermined method (zigzag or raster scan in a diagonal direction), and in case of CAVLC encoding, scanning in a different method from the above method. It can be.

For example, the scanning method may be determined according to zigzag in the case of inter and the intra prediction mode in the case of intra, and the coefficient scanning method may be determined differently according to the size of the transformation unit.

Meanwhile, the scan pattern may vary according to the directional intra prediction mode, and the scan order of quantization coefficients may be reversed.

The multiplexer 167 multiplexes the motion information encoded by the motion information encoder 163 and the residual signals encoded by the residual block encoder 166.

The motion information may vary according to the coding mode. For example, in case of skip or merge, it may include only an index indicating a predictor, and in case of AMVP, it may include a reference picture index of the current block, a differential motion vector, and an AMVP index. .

Hereinafter, an embodiment of the operation of the intra prediction unit 150 shown in FIG. 1 will be described in detail.

First, the intra prediction unit 150 receives prediction mode information and the size of the prediction unit PU from the picture division unit 110, and sets reference pixels to the picture storage unit to determine the intra prediction mode of the prediction unit PU. It can be read from (180).

The intra prediction unit 150 determines whether a reference pixel is generated by examining whether unavailable reference pixels exist, and the reference pixels can be used to determine an intra prediction mode of a current block.

If the current block is located on the upper boundary of the current picture, pixels adjacent to the upper side of the current block are not defined, and if the current block is located on the left boundary of the current picture, pixels adjacent to the left of the current block are not defined. The pixels may be determined to be not usable pixels.

Also, when the current block is located on a slice boundary and pixels adjacent to the upper or left side of the slice are not pixels to be encoded and reconstructed first, it may be determined that they are not usable pixels.

As described above, when pixels adjacent to the left side or above the current block do not exist or pixels that have been previously encoded and reconstructed do not exist, the intra prediction mode of the current block may be determined using only available pixels.

Meanwhile, a reference pixel at an unavailable location may be generated using available reference pixels of the current block. For example, when pixels of the upper block are not available, some or all of the left pixels may be used to generate the upper pixel. can be created, and vice versa.

That is, a reference pixel is generated by copying an available reference pixel at a location closest to a predetermined direction from a reference pixel at an unavailable location, or when there is no reference pixel available in a predetermined direction, the closest available reference pixel in the opposite direction exists. A reference pixel may be created by copying an available reference pixel of a location.

Meanwhile, even when there are pixels above or to the left of the current block, they may be determined as unavailable reference pixels according to the coding mode of the block to which the pixels belong.

For example, if a block to which a reference pixel adjacent to the upper side of the current block belongs is a block reconstructed by inter-encoding, the pixels may be determined as unavailable pixels.

In this case, a block adjacent to the current block is intra-coded, and usable reference pixels may be generated using pixels belonging to the reconstructed block, and the information that the encoding apparatus 10 determines available reference pixels according to the encoding mode is provided. It is transmitted to the decryption device 20.

The intra predictor 150 determines an intra prediction mode of a current block using the reference pixels, and the number of intra prediction modes allowable for the current block may vary depending on the size of the block.

For example, when the size of the current block is 8x8, 16x16, or 32x32, 34 intra prediction modes may exist, and when the size of the current block is 4x4, 17 intra prediction modes may exist.

The 34 or 17 intra prediction modes may include at least one non-directional mode (non-directional mode) and a plurality of directional modes (directional modes).

One or more non-directional modes may be DC modes and/or planar modes. When the DC mode and the planar mode are included as non-directional modes, 35 intra prediction modes may exist regardless of the size of the current block.

In this case, two non-directional modes (DC mode and planar mode) and 33 directional modes may be included.

In the planar mode, a prediction block of the current block is formed using at least one pixel value (or a predicted value of the pixel value, hereinafter referred to as a first reference value) located at the bottom-right of the current block and reference pixels. is created

The configuration of the video decoding apparatus according to an embodiment of the present invention may be derived from the configuration of the video encoding apparatus 10 described with reference to FIGS. 1 to 6, and for example, as described with reference to FIGS. 1 to 6 An image may be decoded by performing reversely the processes of the same image encoding method.

7 is a block diagram showing the configuration of a video decoding apparatus according to an embodiment of the present invention. The decoding apparatus 20 includes an entropy decoding unit 210, an inverse quantization/inverse transform unit 220, an adder 270, A deblocking filter 250, a picture storage unit 260, an intra prediction unit 230, a motion compensated prediction unit 240, and an intra/inter conversion switch 280 are provided.

The entropy decoding unit 210 receives and decodes the bit stream encoded by the video encoding apparatus 10, separates it into an intra prediction mode index, motion information, a quantization coefficient sequence, etc., and converts the decoded motion information into a motion compensation prediction unit ( 240).

The entropy decoding unit 210 transfers the intra prediction mode index to the intra prediction unit 230 and the inverse quantization/inverse transformation unit 220, and transfers the inverse quantization coefficient sequence to the inverse quantization/inverse transformation unit 220.

The inverse quantization/inverse transformation unit 220 transforms the quantization coefficient sequence into a 2-dimensional array of inverse quantization coefficients, and may select one of a plurality of scanning patterns for the conversion. For example, the prediction mode of the current block (i.e., , intra prediction or inter prediction) and an intra prediction mode, a scanning pattern may be selected.

The inverse quantization/inverse transform unit 220 restores the quantization coefficient by applying a quantization matrix selected from among a plurality of quantization matrices to the inverse quantization coefficient of the two-dimensional array.

Meanwhile, different quantization matrices are applied according to the size of a current block to be reconstructed, and a quantization matrix may be selected for blocks of the same size based on at least one of a prediction mode and an intra prediction mode of the current block.

The inverse quantization/inverse transformation unit 220 restores a residual block by inversely transforming the reconstructed quantization coefficient, and the inverse transformation process may be performed using a transform unit (TU) as a basic unit.

The adder 270 restores an image block by adding the residual block reconstructed by the inverse quantization/inverse transformation unit 220 and the prediction block generated by the intra prediction unit 230 or the motion compensation prediction unit 240.

The deblocking filter 250 may reduce deblocking artifacts caused by image loss due to the quantization process by performing a deblocking filter process on the reconstructed image generated by the adder 270.

The picture storage unit 260 is a frame memory for storing a locally decoded image on which deblocking filter processing is performed by the deblocking filter 250 .

The intra prediction unit 230 restores the intra prediction mode of the current block based on the intra prediction mode index received from the entropy decoding unit 210 and generates a prediction block according to the restored intra prediction mode.

The motion compensation prediction unit 240 generates a prediction block for a current block from a picture stored in the picture storage unit 260 based on the motion vector information, and when fractional-precision motion compensation is applied, a selected interpolation filter is applied to the prediction block. can create

The intra/inter conversion switch 280 may provide the prediction block generated by either the intra prediction unit 230 or the motion compensation prediction unit 240 to the adder 270 based on the encoding mode.

8 is a block diagram showing an embodiment of a configuration for performing inter prediction in the video decoding apparatus 20. The inter prediction decoder includes a demultiplexer 241, a motion information encoding mode determination unit 242, and a merge mode motion It includes an information decoding unit 243, an AMVP mode motion information decoding unit 244, a prediction block generator 245, a residual block decoding unit 246, and a reconstruction block generator 247.

Referring to FIG. 8 , the demultiplexer 241 demultiplexes currently encoded motion information and encoded residual signals from the received bitstream, and transmits the demultiplexed motion information to the motion information encoding mode determination unit 242. and the demultiplexed residual signal may be transmitted to the residual block decoder 246.

The motion information encoding mode determination unit 242 determines the motion information encoding mode of the current block, and if skip_flag of the received bitstream has a value of 1, it is determined that the motion information encoding mode of the current block is encoded in the skip encoding mode. can do.

The motion information encoding mode determining unit 242 determines the motion information encoding mode of the current block when skip_flag of the received bitstream has a value of 0 and the motion information received from the demultiplexer 241 has only a merge index. It can be determined that is encoded in the merge mode.

In addition, the motion information coding mode determination unit 242 determines that the skip_flag of the received bitstream has a value of 0 and the motion information received from the demultiplexer 241 has a reference picture index, a differential motion vector, and an AMVP index. In this case, it can be determined that the motion information encoding mode of the current block is encoded in the AMVP mode.

The merge mode motion information decoding unit 243 is activated when the motion information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is skip or merge mode. It can be activated when the information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is the AMVP mode.

The prediction block generation unit 245 generates a prediction block of a current block using motion information reconstructed by the merge mode motion information decoding unit 243 or the AMVP mode motion information decoding unit 244.

When the motion vector is an integer unit, a prediction block of the current block may be generated by copying a block corresponding to a position indicated by the motion vector in a picture indicated by a reference picture index.

On the other hand, if the motion vector is not an integer unit, pixels of a prediction block are generated from integer unit pixels in the picture indicated by the reference picture index. In this case, an 8-tap interpolation filter is used for luminance pixels and for chrominance pixels A prediction pixel may be generated using a 4-tap interpolation filter.

The residual block decoder 246 entropy-decodes the residual signal and inverse-scans the entropy-decoded coefficients to generate a 2-dimensional quantized coefficient block. The inverse scanning method may vary depending on the entropy decoding method.

For example, the inverse scanning method may be applied as a raster inverse scan method in a diagonal direction in case of CABAC-based decoding, and as a zigzag inverse scan method in case of CAVLC-based decoding. Also, the inverse scanning method may be determined differently according to the size of the prediction block.

The residual block decoder 246 inversely quantizes the generated coefficient block using an inverse quantization matrix, and may restore quantization parameters to derive the quantization matrix. Here, the quantization step size may be reconstructed for each coding unit equal to or greater than a predetermined size.

The residual block decoder 260 inversely transforms the inverse quantized coefficient block to restore a residual block.

The reconstruction block generator 270 generates a reconstruction block by adding the prediction block generated by the prediction block generator 250 and the residual block generated by the residual block decoder 260.

Hereinafter, an embodiment of a process of reconstructing a current block through intra prediction will be described with reference to FIG. 7 again.

First, the intra prediction mode of the current block is decoded from the received bitstream, and for this, the entropy decoding unit 210 restores the first intra prediction mode index of the current block by referring to one of a plurality of intra prediction mode tables. can

As a table shared by the encoding apparatus 10 and the decoding apparatus 20 of the plurality of intra prediction mode tables, any one table selected according to the distribution of intra prediction modes for a plurality of blocks adjacent to the current block may be applied. .

For example, if the intra prediction mode of the left block of the current block and the intra prediction mode of the upper block of the current block are the same, the first intra prediction mode index of the current block is restored by applying the first intra prediction mode table, and the first intra prediction mode index of the current block is restored. Otherwise, the first intra prediction mode index of the current block may be restored by applying the second intra prediction mode table.

As another example, when the intra prediction modes of the upper block and the left block of the current block are both directional intra prediction modes, the direction of the intra prediction mode of the upper block and the direction of the intra prediction mode of the left block If the angle is within the predetermined angle, the first intra prediction mode index of the current block is restored by applying the first intra prediction mode table, and if the angle is out of the predetermined angle, the first intra prediction mode index of the current block is restored by applying the second intra prediction mode table. can also be restored.

The entropy decoding unit 210 transmits the first intra prediction mode index of the reconstructed current block to the intra prediction unit 230 .

Upon receiving the index of the first intra prediction mode, the intra predictor 230 may determine the maximum possible mode of the current block as the intra prediction mode of the current block when the index has a minimum value (ie, 0). .

Meanwhile, when the index has a value other than 0, the intra predictor 230 compares the index indicated by the maximum possible mode of the current block with the first intra prediction mode index, and as a result of the comparison, the first intra prediction mode index. If the index is not smaller than the index indicated by the maximum possible mode of the current block, an intra-prediction mode corresponding to a second intra-prediction mode index obtained by adding 1 to the first intra-prediction mode index is determined as the intra-prediction mode of the current block; Otherwise, the intra prediction mode corresponding to the first intra prediction mode index may be determined as the intra prediction mode of the current block.

The intra prediction modes allowable for the current block may include at least one non-directional mode (non-directional mode) and a plurality of directional modes (directional modes).

One or more non-directional modes may be DC modes and/or planar modes. Also, any one of a DC mode and a planner mode may be adaptively included in the set of allowable intra prediction modes.

To this end, information specifying a non-directional mode included in the set of allowable intra prediction modes may be included in a picture header or slice header.

Next, the intra prediction unit 230 reads reference pixels from the picture storage unit 260 to generate an intra prediction block, and determines whether unavailable reference pixels exist.

The determination may be made according to whether reference pixels used to generate an intra-prediction block by applying the decoded intra-prediction mode of the current block exist.

Next, when it is necessary to generate a reference pixel, the intra prediction unit 230 may generate reference pixels at unavailable locations using previously reconstructed available reference pixels.

The definition of an unavailable reference pixel and the method of generating the reference pixel may be the same as the operation of the intra prediction unit 150 according to FIG. 1, but an intra prediction block is generated according to the decoded intra prediction mode of the current block. Reference pixels used to do this may be selectively reconstructed.

In addition, the intra prediction unit 230 determines whether or not to apply a filter to reference pixels to generate a prediction block, that is, whether to apply filtering to reference pixels to generate an intra prediction block of the current block. It may be determined based on the decoded intra prediction mode and the size of the current prediction block.

Since the problem of blocking artifacts increases as the size of the block increases, the number of prediction modes for filtering reference pixels can be increased as the size of the block increases. For this, the reference pixel may not be filtered.

When it is determined that a filter needs to be applied to the reference pixel, the intra prediction unit 230 filters the reference pixels using a filter.

At least two or more filters may be adaptively applied according to the level of difference between the reference pixels. The filter coefficient of the filter is preferably symmetrical.

In addition, the above two or more filters may be adaptively applied according to the size of the current block. When applying the filter, a filter with a narrow bandwidth is used for small blocks and a filter with a wide bandwidth is used for large blocks. may be applied.

In the case of DC mode, a prediction block is generated with the average value of reference pixels, so there is no need to apply a filter. It may not be necessary to apply a filter to a reference pixel even in a horizontal mode in which direction is correlated.

In this way, since whether filtering is applied is also related to the intra prediction mode of the current block, the reference pixel may be adaptively filtered based on the intra prediction mode of the current block and the size of the prediction block.

Next, the intra prediction unit 230 generates a prediction block using a reference pixel or filtered reference pixels according to the reconstructed intra prediction mode, and generation of the prediction block is the same as the operation in the encoding apparatus 10. Therefore, a detailed description thereof will be omitted.

The intra predictor 230 determines whether to filter the generated prediction block, and the filtering can be determined using information included in a slice header or a coding unit header or according to an intra prediction mode of a current block.

When it is determined that the generated prediction block is to be filtered, the intra predictor 230 may generate a new pixel by filtering a pixel at a specific position of the generated prediction block using available reference pixels adjacent to the current block. .

For example, in the DC mode, among prediction pixels, prediction pixels that come into contact with reference pixels may be filtered using the reference pixels that come into contact with the prediction pixels.

Therefore, prediction pixels are filtered using one or two reference pixels according to the position of the prediction pixels, and prediction pixel filtering in the DC mode can be applied to prediction blocks of all sizes.

Meanwhile, in the vertical mode, among prediction pixels of a prediction block, prediction pixels adjacent to a left reference pixel may be changed using reference pixels other than an upper pixel used to generate the prediction block.

Similarly, in the horizontal mode, among generated prediction pixels, prediction pixels contacting an upper reference pixel may be changed using reference pixels other than the left pixel used to generate the prediction block.

The current block may be reconstructed using the predicted block of the current block reconstructed in this way and the residual block of the current block decoded.

9 is a diagram for explaining a second embodiment of a method of dividing and processing an image into blocks.

Referring to FIG. 9 , a coding tree unit (CTU) having a maximum size of 256x256 pixels may be first divided into a quad tree structure and then divided into four coding units (CUs) having a square shape.

Here, at least one of the coding units divided into the quad tree structure may be divided into a binary tree structure and re-divided into two coding units (CUs) having a rectangular shape.

Meanwhile, at least one of the coding units divided into the quad tree structure may be divided into the quad tree structure and re-divided into four coding units (CUs) having a square shape.

In addition, at least one of the coding units re-divided into the binary tree structure may be divided into two coding units (CUs) having a square or rectangular shape.

Meanwhile, at least one of the coding units re-divided into the quad-tree structure may be further divided into a quad-tree structure or a binary tree structure and divided into coding units (CUs) having a square or rectangular shape.

Coding blocks (CBs) configured by being divided into a binary tree structure as described above can be used for prediction and transformation without being further divided. That is, the size of the prediction unit (PU) and the transform unit (TU) belonging to the coding block (CB) shown in FIG. 9 may be the same as the size of the corresponding coding block (CB).

Coding units divided into a quad tree structure as described above may be divided into one or two or more prediction units (PUs) using the method described with reference to FIGS. 3 and 4 .

In addition, the coding unit divided into the quad tree structure as described above may be divided into one or two or more transform units (TUs) using the method described with reference to FIG. 5, and the divided transform units (TUs) may have a maximum size of 64x64 pixels.

10 illustrates an embodiment of a syntax structure used to divide and process an image into blocks.

Referring to FIG. 10, whether to split a coding unit (CU) as described with reference to FIG. 9 is indicated using split_cu_flag, and the depth of a coding unit (CU) split using a binary tree is indicated using binary_depth. can be

In addition, whether the coding unit (CU) is split into a binary tree structure may be indicated as a separate binary_split_flag.

1 to 8 for blocks (eg, coding unit (CU), prediction unit (PU), and transform unit (TU)) partitioned by the method as described with reference to FIGS. 9 and 10 By applying the methods as described above, encoding and decoding of an image can be performed.

Hereinafter, other embodiments of a method of dividing a coding unit (CU) into one or more transform units (TUs) will be described with reference to FIGS. 11 to 16 .

According to an embodiment of the present invention, a coding unit (CU) may be divided into a binary tree structure and divided into transform units (TUs), which are basic units in which transform on a residual block is performed.

Referring to FIG. 11, at least one of the rectangular coding blocks (CB ₀ and CB ₁ ) having a size of Nx2N or 2NxN divided into a binary tree structure is divided into a binary tree structure and formed into a square having a size of NxN. It may be divided into transform units TU ₀ and TU ₁ .

As described above, the block-based video encoding method may perform prediction, transformation, quantization, and entropy encoding steps.

In the prediction step, a prediction signal is generated by referring to a block to be currently encoded and an existing encoded image or neighboring images, and a difference signal between the current block and the current block can be calculated through this.

On the other hand, in the conversion step, conversion is performed using various conversion functions by taking the differential signal as an input, and the converted signal is classified into DC coefficients and AC coefficients, and energy compaction is performed to improve coding efficiency. can

Also, in the quantization step, quantization is performed using transform coefficients as inputs, and then entropy encoding is performed on the quantized signal, so that an image can be encoded.

Meanwhile, the video decoding method proceeds in the reverse order of the above encoding process, and image quality distortion may occur in the quantization step.

As a method for reducing image quality distortion while improving coding efficiency, the size or shape of the transform unit (TU) and the type of applied transform function can be varied according to the distribution of the difference signal coming into the input in the transform step and the characteristics of the image. there is.

For example, in the case of finding a block similar to the current block through a block-based motion estimation process in the prediction step, using a cost measurement method such as SAD (Sum of Absolute Difference) or MSE (Mean Square Error), the difference Distribution of signals may occur in various forms depending on characteristics of an image.

Accordingly, effective encoding can be performed by performing transformation by selectively determining the size or shape of the transform unit (CU) based on the distribution of various differential signals.

Referring to FIG. 12, when a differential signal is generated as shown in (a) in an arbitrary coding block (CBx), as shown in (b), the corresponding coding block (CBx) is divided into a binary tree structure to obtain 2 By dividing into two transform units (TUs), efficient transformation can be performed.

For example, since a DC value can generally be said to represent an average value of an input signal, when a differential signal as shown in FIG. Dividing into transform units (TUs) effectively represents a DC value.

Referring to FIG. 13, a square coding unit (CU ₀ ) having a size of 2Nx2N may be divided into a binary tree structure and divided into rectangular transform units (TU ₀ and TU ₁ ) having a size of Nx2N or 2NxN. .

According to another embodiment of the present invention, the step of dividing the coding unit (CU) into a binary tree structure as described above may be repeatedly performed twice or more to divide the coding unit (CU) into a plurality of transform units (TUs).

Referring to FIG. 14, a rectangular coding block (CB ₁ ) having a size of Nx2N is divided into a binary tree structure, and the block having a size of NxN is divided into a binary tree structure again to N/2xN or NxN/ After constructing a rectangular block having a size of 2, the block having a size of N/2xN or NxN/2 is divided into a binary tree structure to form square transformation units having a size of N/2xN/2 (TU ₁ , TU ₂ , TU ₄ , TU ₅ ).

Referring to FIG. 15, a square coding block (CB ₀ ) having a size of 2Nx2N is divided into a binary tree structure, and the divided block having a size of Nx2N is again divided into a binary tree structure to form a square having a size of NxN. After constructing a block of NxN, the block having a size of NxN may be divided again into a binary tree structure and divided into rectangular transform units TU ₁ and TU ₂ having a size of N/2xN.

Referring to FIG. 16, a rectangular coding block (CB ₀ ) having a size of 2NxN is divided into a binary tree structure, and the divided block having a size of NxN is again divided into a quad tree structure so that N/2xN/2 It can be divided into square transform units (TU ₁ , TU ₂ , TU ₃ , and TU ₄ ) having sizes.

1 to 8 for blocks (eg, coding unit (CU), prediction unit (PU), and transform unit (TU)) partitioned by the method as described with reference to FIGS. 11 to 16 By applying the methods as described above, encoding and decoding of an image can be performed.

Hereinafter, embodiments of a method for determining a block division structure by the encoding apparatus 10 according to the present invention will be described.

The picture divider 110 provided in the image encoding apparatus 10 performs Rate Distortion Optimization (RDO) according to a preset order, and as described above, divideable Coding Units (CUs), Prediction Units (PUs), and Transformation A division structure of the unit (TU) may be determined.

For example, in order to determine the block division structure, the picture division unit 110 performs RDO-Q (Rate Distortion Optimization-Quantization) while determining an optimal block division structure in terms of bitrate and distortion. can decide

Referring to FIG. 17, when the coding unit (CU) has a shape of 2Nx2N pixels, the 2Nx2N pixel size shown in (a), the NxN pixel size shown in (b), and the Nx2N pixel size shown in (c) , it is possible to determine an optimal partition structure of the transform unit (PU) by performing RDO in the order of the partition structure of the transform unit (PU) of 2NxN pixel size shown in (d).

Referring to FIG. 18, when the coding unit (CU) has a pixel size of Nx2N or 2NxN, the pixel size of Nx2N (or 2NxN) shown in (a), the pixel size of NxN shown in (b), Pixel sizes of N/2xN (or NxN/2) and NxN shown in (c), pixel sizes of N/2xN/2, N/2xN and NxN shown in (d), N shown in (e) It is possible to determine an optimal division structure of the conversion unit (PU) by performing RDO in the order of the division structure of the conversion unit (PU) having a pixel size of /2xN.

In the above, the block division method of the present invention has been described by taking as an example that the block division structure is determined by performing rate distortion optimization (RDO), but the picture division unit 110 is SAD (Sum of Absolute Difference) or MSE (Mean Square Error) ) to determine the block division structure, it is possible to maintain appropriate efficiency while reducing complexity.

19 is a diagram for explaining a configuration of a reference sample to which a variation amount is applied according to an embodiment of the present invention.

The encoding apparatus 10 or the decoding apparatus 20 performing intra prediction according to an embodiment of the present invention may configure a reference sample for intra prediction decoding, and the reference sample is twice the width and height of the current block. It may be composed of pre-decoded samples corresponding to . Directional prediction may be performed using these reference samples.

However, when there is no pre-decoded reference sample, conventionally, as shown in FIG. 19(A), a reference sample is constructed by padding the value of the outermost pixel A4 of the current block, and directional prediction is performed using this. is processing However, this hinders direction formation and reduces the accuracy of a prediction block, resulting in a decrease in encoding/decoding performance.

Accordingly, the encoding apparatus 10 and the decoding apparatus 20 according to an embodiment of the present invention, in the configuration of the reference sample, correspond to a region in which the previously encoded/decoded reference sample does not exist, and the amount of change of the current or neighboring blocks. By constructing the applied reference sample by padding, more accurate prediction and encoding efficiency can be increased. Accordingly, as shown in FIG. 19(B), it is possible to construct a reference sample in a form in which directionality is maintained by padding with variation applied, and accordingly, encoding/decoding efficiency can be increased.

Hereinafter, such an encoding/decoding method will be mainly described in terms of the operation process of the decoding device 20, and the operation of the decoding device 20 described below can be equally applied to the encoding device 10.

20 to 21 are flowcharts for describing an intra prediction process according to an embodiment of the present invention.

Referring to FIG. 20, when a bitstream is received, the decoding apparatus 20 first processes entropy decoding (S101), performs inverse quantization/inverse transformation (S103), and performs intra-prediction decoding from inverse-transformed block information. A predictive information generation process is performed (S105).

Then, the decoding apparatus 20 performs correction processing of the prediction information in response to generation of the prediction information (S107), and performs intra-prediction encoding using the corrected prediction information (S109).

Here, the operating method of the decoding apparatus 20 according to the embodiment of the present invention for generating and correcting prediction information may be processed by the intra prediction unit 230, and may be described in more detail with reference to FIG. 21 .

Referring to FIG. 21, the intra prediction unit 230 enters a prediction information generation process (S201) and identifies existence information of a previously decoded reference sample among neighboring blocks of a current block to be subjected to intra prediction (S203). .

Then, when the region where the reference sample does not exist is identified, the intra predictor 230 performs padding on the reference sample based on the variation amount of the neighboring sample in response to the region where the reference sample does not exist (S205).

Then, the intra predictor 230 generates intra-prediction information using the padded reference sample (S207).

Here, the padding of the reference sample based on the variation of the neighboring sample according to an embodiment of the present invention may be processed by padding a single variation reflecting sample or multiple variation reflecting sample padding, or may be selectively processed in some cases. This will be described later in more detail.

22 is a flowchart illustrating a process of generating a single change amount reflecting sample according to an embodiment of the present invention, and FIGS. 23 to 24 are exemplary diagrams of a single change amount reflecting sample according to an embodiment of the present invention.

Referring to FIG. 22, when the intra predictor 230 according to an embodiment of the present invention identifies a reference sample for which no previously decoded information exists (S301), a single axial variation of the previously decoded block is applied, A change amount reflecting sample generation is generated (S303).

The process of padding a single change amount sample according to an embodiment of the present invention may include a step of processing padding by reflecting a single change amount in the x-axis or a single change amount in the y-axis with respect to a reference sample region where previously decoded information does not exist. .

To this end, the intra prediction unit 230 may first determine whether to pad a single variation reference sample.

For example, as shown in FIG. 23(A), when only some of the previously decoded reference samples exist on both the top and left sides of the current block, the non-existent reference samples may be padded using already-existing samples.

On the other hand, as shown in FIG. 23(B), when all of the previously decoded reference samples do not exist on the top or left side of the current block (only the top or left sample exists), since there are no existing samples, The reference sample may be padded using the top sample.

As such, FIG. 23 shows that samples in an area where there are no pre-decoded reference samples are padded and used for intra-prediction decoding, NA# means a sample that does not exist, and A# is a sample used for padding. can mean

23 illustrates an upper reference sample, and the same method may be applied to a non-existent sample (NL#) on the left side.

According to an embodiment of the present invention, generating a reference sample corresponding to a sample (NA _d , NL _d ) for which no previously decoded information exists can be exemplified as follows.

Here, k denotes the number of existing samples, and n may denote the number of samples to reflect the amount of change, where n may be a constant that can be appropriately adjusted according to the size of the current block.

For example, when the current block has a rectangular shape, as shown in FIG. 23 , in order to use a sample without pre-decoded information as a reference block, padding of the reference sample as much as the height of the current block may be required.

Accordingly, when n is large, the intra predictor 230 needs to pad a relatively small number of reference samples, but the number of samples reflecting the amount of change increases, and thus the correlation may deteriorate. Therefore, as shown in FIG. 24, the size of n is preferably adjusted to the height of the current block.

In the case of sample padding reflecting a single variation amount according to an embodiment of the present invention, padding reflecting a variation amount may be disadvantageous if the variation is irregular without increasing or decreasing because it is affected by the increase or decrease of existing samples. Accordingly, the intra predictor 230 may determine the linearity of existing samples and selectively determine whether to increase or decrease non-existent samples.

In order to determine linearity, the intra prediction unit 230 according to an embodiment of the present invention may use a condition function such as Equation 2 below.

Equation 2 above may mean that it is judged to be linear when the sum of the first pixel and the last pixel of the existing sample that determines whether a strong filter is applied is greater than twice the middle pixel.

In addition, the Pearson Coreelation Coefficent function can be used, and when calculated as ρ _X,Y = cov(X,Y)/(σ _X σ _Y ), -TH>ρ _X,Y || When the condition of TH<ρ _X,Y is satisfied, it can be determined to be linear. In general, TH is a threshold value, and 0.9 may be exemplified.

Meanwhile, the sample padding reflecting a single variation according to an embodiment of the present invention may be used as a candidate for finding an optimal prediction block in the encoding device 10. That is, an optimal prediction block can be found by performing the existing technology and intra prediction according to an embodiment of the present invention in parallel.

Accordingly, in the case of the present invention, in consideration of the purpose of the technology and the complexity of the encoder/decoder, for some directional modes (modes after the vertical mode, modes before the horizontal mode), simple padding and variation reflecting padding are compared, thereby providing optimal mode can be determined. In this case, the encoding device 10 may signal (1 bit) a reference sample padding mode flag corresponding to the selected reference sample padding method to the decoding device 20.

Accordingly, linearity determination and optional single variation sample padding based on flags can be processed.

25 is a flowchart illustrating a sample padding process reflecting a single variation amount according to an embodiment of the present invention.

Referring to FIG. 25 , the intra predictor 230 first identifies a reference sample for which no previously decoded information exists (S401), and determines linearity of neighboring blocks corresponding to the identified block (S403).

First, if it is determined to be linear, the intra prediction unit 230 checks the padding flag signaled from the encoding device 10 (S405), and if the padding flag is 1, the intra prediction unit 230 processes padding with the above-described single variation reflecting sample. (S407).

On the other hand, if it is determined non-linearly or if the padding flag is not 1, the intra predictor 230 performs padding in the same manner as the conventional method with the same value as the nearest sample (S409).

Thereafter, the intra prediction unit 230 performs intra prediction decoding processing using the padded reference sample (S411).

Such selective padding processing may be processed in parallel with conventional padding, and in some cases, processing of the padding flag may be omitted.

That is, the encoding device 10 performs intra-prediction by performing both the conventional padding method and padding reflecting the variation amount, selects an efficient method through RD-Cost comparison, and performs 1-bit signaling only when a padding flag is required. there is.

In addition, the decoding apparatus 20 also performs a linear determination first before determining the padding flag, and in the case of non-linearity, decoding can be performed without additional bits, and thus the selective padding process can be efficiently processed.

26 is a flowchart illustrating a process of generating multiple variation-reflecting samples according to an embodiment of the present invention, and FIGS. 27 to 28 are exemplary diagrams of multiple variation-reflecting samples according to an embodiment of the present invention.

In the sample generation process reflecting the amount of change according to an embodiment of the present invention, sample padding is a method of reflecting the amount of change in the x-axis and the amount of change in the y-axis with respect to a reference sample for which there is no pre-decoded information. Right Bottom) generating a sample (S501), and generating multiple variation reflecting samples by applying the X-axis and Y-axis variation between the lower right sample and the upper right (RA, Right Above) sample (S501). S503).

This is a process of designating a reference sample for which there is no pre-decoded information as a sample (NA, NL) adjacent to the current block, and generating padding for the reference sample by reflecting the change amount of the x-axis and y-axis corresponding thereto. Samples (NA, NL, gray) for which no previously decoded information exists are displayed adjacent to the current block for convenience. However, in an actual case, NA and NL may be included in the reference sample array (one-dimensional array).

In FIG. 27, NA# may mean a sample that does not exist, and A# may mean a sample used for padding. In addition, FIG. 27 shows an example of the top reference sample, and the left nonexistent sample (NL#) can also be applied in the same way.

And, referring to FIG. 27(A), multiple sample variation sample padding may be applied when some of the top and left samples exist in pre-decoded samples. In this case, the intra prediction unit 230 may generate a right bottom (RB) sample using right above (RA) and left bottom (LB) to construct a reference sample in which no pre-decoded information exists.

On the other hand, referring to FIG. 27(B) , multiple sample variation sample padding may be applied when neither the top nor the left sample exists (only the top or left sample exists). In this case, the intra predictor 230 may copy RA and LB to samples (adjacent samples such as A4 and L2) where previously decoded information already exists, or may process the above-described single variation sample padding. For example, in this case, since there is no sample with pre-decoded information, the top sample of the already encoded/decoded block is obtained by performing the above-described single-variation sample padding, and then an RB sample is generated therefrom to obtain the previously decoded sample. It may be formed by padding a reference sample for which information does not exist.

In this configuration, as shown in FIG. 28 , the intra predictor 230 may first generate an arbitrary RB sample for padding multiple variation samples.

Here, the RB sample may be obtained as a distance-proportional sum of the top right sample (RA) and the right and left sample (LA), and the top sample (RA) and the left sample (LA) are obtained by copying from adjacent samples, or the above-mentioned single It can be created through a padding method that reflects the amount of change. In the case of a distance proportional sum, it may be calculated as shown in Equation 3 below.

In Equation 3, I(x,y) may represent a pixel value at a position of (x,y).

After generating the RB sample, the intra prediction unit 230 may generate a sample reflecting the reference axis variation between the RB sample and the upper sample RA. NA and NL can be generated as a distance-proportionate weight sum, which is shown in Equation 4 below.

In the case of such multi-variation-reflecting sample padding, since it is affected by the increase/decrease of existing samples, if the change is irregular without increasing/decreasing, the variation-reflecting padding may be disadvantageous. Therefore, it is possible to determine whether to increase or decrease non-existent samples by determining the linearity of the existing samples. Linearity determination may be determined according to a strong filter determination condition or a threshold value comparison of Pearson correlation coefficients, as in the case of the above-described single change amount reflecting sample.

Meanwhile, sample padding reflecting multiple variations according to an embodiment of the present invention may also be used as a candidate for finding an optimal prediction block in the encoding apparatus 10. That is, an optimal prediction block can be found by performing the existing technology and intra prediction according to an embodiment of the present invention in parallel.

Accordingly, in the case of the present invention, for some directional modes (modes after vertical mode, modes before horizontal) in consideration of the purpose of the technology and the complexity of the encoder/decoder, simple padding and multiple variation reflection padding and the aforementioned single variation reflection By comparing and performing padding, an optimal mode can be determined. In this case, the encoding device 10 may signal a reference sample padding mode flag corresponding to the selected reference sample padding method to the decoding device 20. If both the multiple variation padding and the multi-variation amount padding are used, a padding mode flag of at least 1 bit and at most 2 bits may be used.

29 is a flowchart illustrating a selective single and multiple variation reflecting sample padding process according to an embodiment of the present invention.

Referring to FIG. 29 , the intra predictor 230 first identifies a reference sample for which no previously decoded information exists (S601), and determines linearity of neighboring blocks corresponding to the identified block (S603).

First, if it is determined to be linear, the intra predictor 230 checks the padding flag signaled from the encoding device 10 (S605), and if the padding flag is 1, the intra prediction unit 230 processes padding with the above-described single variation reflecting sample. (S608).

Meanwhile, when the padding flag is not 1, the intra predictor 230 determines whether the padding mode is 2 (S606).

On the other hand, if it is determined non-linearly or if the padding flag is not 1 or 2, the intra prediction unit 230 performs padding in the same manner as the conventional method with the same value as the nearest sample (S609).

Then, the intra prediction unit 230 performs intra prediction decoding processing using the padded reference sample (S211).

Such selective padding processing may be processed in parallel with conventional padding, and in some cases, processing of the padding flag may be omitted.

That is, the encoding device 10 performs intra-prediction by performing both the conventional padding method and padding reflecting the variation amount, and selects an efficient method through RD-Cost comparison, and performs 1-bit or 2-bit signaling only when a padding flag is required can do.

In addition, the decoding apparatus 20 also performs a linear determination first before determining the padding flag, and in the case of non-linearity, decoding can be performed without additional bits, and thus the selective padding process can be efficiently processed.

Accordingly, information signaled from the encoding device 10 to the decoding device 20 may be exemplified as follows.

A. Signaling information of 0 to 1 bit (Padding flag) when only one of the single variation padding or multiple variation padding is compared with the existing padding

B. 1~2 bit signaling information (Padding flag / Padding mode) when comparing single variation and multiple variation methods with existing padding

C. When padding (single or multiple) reflecting variation is performed only when it is linear by first making a linear decision, and excluding existing padding methods from comparison targets, 1-bit signaling is information (Padding mode)

30 and 31 are diagrams for explaining a reference sample non-existence case to which a variation-reflecting sample padding process according to an embodiment of the present invention is applicable.

Referring to FIGS. 30 and 31, the case in which there is no reference sample to which the variation-reflecting sample padding process according to an embodiment of the present invention is applicable is the case where all previously decoded reference samples exist in generating intra-prediction information. and may include all cases in which at least one sample does not exist. Accordingly, the sample padding process reflecting the amount of variation according to an embodiment of the present invention may be applied from a case in which all previously decoded reference samples do not exist to a case in which they partially exist, and the intra predictor 230 determines the condition for this. Processing in advance enables efficient processing.

The method according to the present invention described above may be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, and magnetic tape. , floppy disks, and optical data storage devices.

The computer-readable recording medium is distributed to computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the technical field to which the present invention belongs.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (10)

In the image processing method,
Dividing a picture of an image into a plurality of coding units, which are basic units in which inter prediction or intra prediction is performed;
identifying a reference sample used for intra prediction for a unit of the current block that is intra-predicted among the divided coding units;
performing padding processing in which a variation amount of a neighboring block of the current block is reflected on a reference sample for which no previously decoded information is present among the reference samples; and
Including; performing intra prediction decoding based on the padded reference sample;
The step of performing the padding process,
Further comprising determining linearity of neighboring blocks to determine whether to perform padding reflecting the amount of change
Image processing method. According to claim 1,
The step of performing the padding process,
Performing a single change amount padding process corresponding to the reference sample
Image processing method. According to claim 1,
The step of performing the padding process,
Comprising the step of performing multiple variation padding processing corresponding to the reference sample
Image processing method. delete According to claim 1,
The step of performing the padding process,
receiving signaling information corresponding to the padding in which the amount of change is reflected; and
Determining whether to perform padding reflecting the amount of change from the signaling information
Image processing method. According to claim 5,
The signaling information includes padding flag information indicating whether padding reflecting a variation amount is applied.
Image processing method. According to claim 5,
The signaling information includes padding mode information indicating either a single change amount reflecting padding or multiple change amount reflecting padding.
Image processing method. According to claim 1,
Whether or not to apply the variation reflecting padding is determined according to a cost operation performed during encoding.
Image processing method. In the method of decoding an image,
Receiving an encoded bitstream;
obtaining a residual block by performing inverse quantization and inverse transformation on the input bitstream;
obtaining a prediction block by performing inter prediction or intra prediction;
Reconstructing an image by adding the obtained residual block and the prediction block;
A coding unit, which is a basic unit in which the inter prediction or intra prediction is performed, is a block divided from a coding tree unit using a binary tree structure,
Obtaining the prediction block,
identifying a reference sample used for intra prediction for a unit of the current block that is intra-predicted among the divided coding units; and
performing padding processing in which a variation amount of a neighboring block of the current block is reflected on a reference sample for which no previously decoded information is present among the reference samples; and
Obtaining the prediction block by performing intra prediction decoding based on the padded reference sample;
The step of performing the padding process,
Further comprising determining linearity of neighboring blocks to determine whether to perform padding reflecting the amount of change
Video decoding method. In the video encoding method,
Dividing a picture of an image into a plurality of coding units, which are basic units in which inter prediction or intra prediction is performed;
identifying a reference sample used for intra prediction for a unit of the current block that is intra-predicted among the divided coding units;
performing padding processing in which a variation amount of a neighboring block of the current block is reflected on a reference sample for which no previously decoded information is present among the reference samples; and
Including; performing intra prediction encoding based on the padded reference sample;
The step of performing the padding process,
Further comprising determining linearity of neighboring blocks to determine whether to perform padding reflecting the amount of change
Video encoding method.

Images