KR20240052921A - A method of video processing providing independent properties between coding tree units and coding units, a method and appratus for decoding and encoding video using the processing - Google Patents

Abstract

An image decoding method according to an embodiment of the present invention includes: identifying a parallel motion information prediction unit index of a current block from which motion information is to be decoded; Obtaining motion information of at least one neighboring block among blocks remaining after excluding blocks belonging to a parallel motion information prediction unit of the previous index from neighboring blocks of the current block; and processing motion information prediction decoding for the current block based on the obtained motion information.

Description

Image processing method for processing motion information for parallel processing, image decoding and encoding method using the same, and device therefor THE PROCESSING}

The present invention relates to an image processing method, an image decoding and encoding method using the same, and a device therefor. More specifically, it relates to an image processing method for processing motion information for parallel processing, an image decoding and encoding method using the same, and a device therefor. will be.

Digital video technologies include digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, video gaming devices, video game consoles, cellular or satellite radio telephones, etc. It can be integrated and applied to a wide range of digital video devices, including. Digital video devices support video compression technologies such as MPEG-2, MPEG-4, or ITU-T H.264/MPEG-4, Part 10, AVC (Advanced Video Coding), and H.265/HEVC (High Efficiency Video Coding). is implemented to transmit and receive digital video information more efficiently. Video compression techniques perform spatial prediction and temporal prediction to remove or reduce redundancy inherent in a video sequence.

As such video compression technology, inter-screen prediction technology predicts pixel values included in the current picture from pictures before or after the current picture, and intra-screen prediction technology predicts pixel values included in the current picture using pixel information in the current picture. There are various technologies, such as entropy encoding technology, which assigns short codes to values with a high frequency of occurrence and long codes to values with a low frequency of occurrence. Using these video compression technologies, video data can be effectively compressed and transmitted or stored. .

In order to cost-effectively respond to various resolutions, frame rates, etc. according to such applications, it is necessary to have a video decoding device that can easily process according to the performance and functions required by the application.

For example, in a video compression method, one picture is divided into a plurality of blocks of a certain size and encoding is performed. Additionally, to increase compression efficiency, inter prediction and intra prediction techniques are used to remove redundancy between pictures.

In this case, intra prediction and inter prediction are used to create a residual signal, and the reason for obtaining the residual signal is that when coding with the residual signal, the amount of data is small, so the data compression rate increases, and the better the prediction, the higher the residual signal. This is because the value of becomes small.

The intra prediction method predicts data of the current block using pixels surrounding the current block. The difference between the actual value and the predicted value is called a residual signal block. In the case of HEVC, the intra prediction method increases from the 9 prediction modes used in existing H.264/AVC to 35 prediction modes, making predictions more granular.

In the case of the inter prediction method, the current block is compared with blocks in surrounding pictures to find the most similar block. At this time, the location information (Vx, Vy) about the found block is called a motion vector. The difference in pixel values within the block between the current block and the prediction block predicted by the motion vector is called a residual signal block (motion-compensated residual block).

As intra-prediction and inter-prediction have become more subdivided, the amount of data in the residual signal is decreasing, but the amount of computation for video processing has increased significantly.

In particular, there are difficulties in pipeline implementation due to the increase in complexity in the process of determining the division structure within the picture for video encoding and decoding, and the existing block division method and the size of the block divided accordingly are difficult to obtain in high-resolution images. It may not be suitable for encoding.

For example, in parallel processing of motion information for inter prediction, the current block can be processed only after the encoding or decoding of the left, upper, and upper-left neighboring blocks has been completed, so even when implementing a parallel pipeline, the current block can be processed. There is a problem in that the block has to wait for the split size and block mode decision of the neighboring block to be processed in another pipeline, which causes a pipeline stall.

To solve this problem, in HEVC, a merge method was proposed to merge motion information for some prediction units within a random block size. However, when calculating motion information for the first prediction unit within the random block, motion information for neighboring blocks is required for the same block size. A problem arises, and as a result, the coding efficiency is greatly reduced.

The present invention is intended to solve the above problems, and its purpose is to provide an image decoding and encoding method and apparatus for improving encoding efficiency by parallel processing of motion information prediction decoding and encoding of high-resolution images.

A motion information decoding method according to an embodiment of the present invention to solve the above problems includes identifying a parallel motion information prediction unit index of a current block to decode motion information; Obtaining motion information of at least one neighboring block among blocks remaining after excluding blocks belonging to a parallel motion information prediction unit of the previous index from neighboring blocks of the current block; and processing motion information prediction decoding for the current block based on the obtained motion information.

A motion information encoding method according to an embodiment of the present invention to solve the above problems includes identifying a parallel motion information prediction unit index of a current block for encoding motion information; Obtaining motion information of at least one neighboring block among blocks remaining after excluding blocks belonging to a parallel motion information prediction unit of the previous index from neighboring blocks of the current block; and processing motion information prediction encoding for the current block based on the obtained motion information.

Meanwhile, the method according to an embodiment of the present invention for solving the above problems can be implemented with a program for executing the method on a computer and a non-volatile recording medium in which the program is stored and readable by the computer.

According to an embodiment of the present invention, decoding blocks can be sequentially grouped into predetermined parallel motion information prediction units.

Additionally, according to an embodiment of the present invention, in motion information decoding, motion information decoding can be performed using motion information of blocks remaining among blocks surrounding the current block, excluding blocks belonging to the previous parallel motion information prediction unit.

Accordingly, independent pipeline processing for each parallel motion information prediction unit is possible, pipeline stalls can be blocked in advance, and encoding and decoding efficiency can be improved.

1 is a block diagram showing the configuration of a video encoding device according to an embodiment of the present invention.
2 to 5 are diagrams for explaining a first embodiment of a method of dividing and processing an image into blocks.
FIG. 6 is a block diagram illustrating an embodiment of a method for performing inter prediction in a video encoding device.
Figure 7 is a block diagram showing the configuration of a video decoding device according to an embodiment of the present invention.
FIG. 8 is a block diagram illustrating an embodiment of a method for performing inter prediction in a video decoding apparatus.
Figure 9 is a diagram for explaining a second embodiment of a method of dividing and processing an image in blocks.
Figure 10 is a diagram for explaining a third embodiment of a method of dividing and processing an image in blocks.
FIG. 11 is a diagram illustrating an embodiment of a method of configuring a transformation unit by dividing a coding unit into a binary tree structure.
Figure 12 is a diagram for explaining a fourth embodiment of a method of dividing and processing an image in blocks.
Figures 13 and 14 are diagrams for explaining another embodiment of a method of dividing and processing an image in blocks.
FIGS. 15 and 16 are diagrams for explaining embodiments of a method for determining the division structure of a conversion unit by performing RDO (Rate Distortion Optimization).
Figures 17 and 18 are flowcharts to explain an image processing method of processing motion information for parallel processing according to an embodiment of the present invention.
19 to 22 are diagrams illustrating the motion information processing method for each case according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description will be omitted.

When a component is said to be “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but that other components may exist in between. It should be. In addition, the description of “including” a specific configuration in the present invention does not exclude configurations other than the configuration, and means that additional configurations may be included in the practice of the present invention or the scope of the technical idea of the present invention.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention.

Additionally, the components appearing in the embodiments of the present invention are shown independently to show different characteristic functions, and this does not mean that each component is comprised of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two of each component can be combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can be divided into a plurality of components. Integrated embodiments and separate embodiments of the constituent parts are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

Additionally, some components may not be essential components that perform essential functions in the present invention, but may simply be optional components to improve performance. The present invention can be implemented by including only essential components for implementing the essence of the present invention excluding components used only to improve performance, and a structure including only essential components excluding optional components used only to improve performance. is also included in the scope of rights of the present invention.

Figure 1 is a block diagram showing the configuration of an image encoding device according to an embodiment of the present invention. The image encoding device 10 includes a picture segmentation unit 110, a transformation unit 120, a quantization unit 130, and a scanning unit. Unit 131, entropy encoding unit 140, intra prediction unit 150, inter prediction unit 160, inverse quantization unit 135, inverse transform unit 125, post-processing unit 170, picture storage unit 180 ), a subtraction unit 190, and an addition unit 195.

Referring to FIG. 1, the picture division unit 110 analyzes an input video signal, divides the picture into coding units, determines a prediction mode, and determines the size of the prediction unit for each coding unit.

Additionally, the picture division unit 110 sends the prediction unit to be encoded to the intra prediction unit 150 or the inter prediction unit 160 according to the prediction mode (or prediction method). Additionally, the picture division unit 110 sends the prediction unit to be encoded to the subtraction unit 190.

Here, the picture of the video is composed of a plurality of slices, and the slices can be divided into a plurality of Coding Tree Units (CTUs), which are the basic units for dividing the picture.

The coding tree unit may be divided into one or two or more coding units (CUs), which are basic units in which inter prediction or intra prediction is performed.

Here, the maximum size of the coding tree unit and the coding unit may be different, and signaling information regarding this may be transmitted to the decoding device 20. This will be described in more detail later in Figure 17.

A coding unit (CU) can be divided into one or more prediction units (PUs), which are the basic units in which prediction is performed.

In this case, the encoding device 10 determines one of inter prediction and intra prediction as the prediction method for each of the divided coding units (CUs), but generates prediction blocks differently for each prediction unit (PU). can be created.

Meanwhile, a coding unit (CU) may be divided into one or two or more transform units (TUs), which are basic units in which transformation of a residual block is performed.

In this case, the picture division unit 110 may transmit image data to the subtraction unit 190 in units of blocks divided as described above (eg, prediction units (PUs) or transform units (TUs)).

Referring to FIG. 2, a coding tree unit (CTU) with a maximum size of 256x256 pixels can be divided into a quad tree structure and divided into four coding units (CUs) having a square shape.

The four coding units (CUs) having the square shape can each be re-divided into a quad tree structure, and the depth of the coding units (CUs) divided into the quad tree structure as described above ranges from 0 to 3. It can have one integer value.

A coding unit (CU) may be divided into one or two or more prediction units (PUs) depending on the prediction mode.

In the case of intra prediction mode, when the size of the coding unit (CU) is 2Nx2N, the prediction unit (PU) may have a size of 2Nx2N shown in (a) of FIG. 3 or NxN shown in (b) of FIG. 3. there is.

Meanwhile, in the case of inter prediction mode, when the size of the coding unit (CU) is 2Nx2N, the prediction unit (PU) is 2Nx2N shown in (a) of FIG. 4, 2NxN shown in (b) of FIG. 4, and FIG. 4. Nx2N shown in (c), NxN shown in (d) of Figure 4, 2NxnU shown in (e) of Figure 4, 2NxnD shown in (f) of Figure 4, shown in (g) of Figure 4 It may have one of the sizes of nLx2N and nRx2N shown in (h) of FIG. 4.

Referring to FIG. 5, a coding unit (CU) may be divided into a quad tree structure and divided into four transformation units (TUs) having a square shape.

The four transformation units (TUs) having the shape of a square can each be re-divided into a quad tree structure, and the depth of the transformation units (TUs) divided into the quad tree structure as described above ranges from 0 to 3. It can have one integer value.

Here, when the coding unit (CU) is in the inter prediction mode, the prediction unit (PU) and transform unit (TU) divided from the corresponding coding unit (CU) may have a split structure that is independent of each other.

When a coding unit (CU) is in intra prediction mode, a transform unit (TU) divided from the corresponding coding unit (CU) cannot be larger than the size of the prediction unit (PU).

Additionally, the transform unit (TU) divided as described above may have a maximum size of 64x64 pixels.

The transform unit 120 transforms a residual block, which is a residual signal between the original block of the input prediction unit (PU) and the prediction block generated by the intra prediction unit 150 or the inter prediction unit 160. It can be performed using a unit (TU) as the basic unit.

In the transformation process, different transformation matrices may be determined depending on the prediction mode (intra or inter), and since the residual signal of intra prediction has directionality according to the intra prediction mode, the transformation matrix may be adaptively determined according to the intra prediction mode. there is.

A transformation unit can be transformed by two (horizontal, vertical) one-dimensional transformation matrices. For example, in the case of inter prediction, one predetermined transformation matrix can be determined.

Meanwhile, in the case of intra prediction, if the intra prediction mode is horizontal, the probability that the residual block has vertical direction increases, so a DCT-based integer matrix is applied in the vertical direction, and a DST-based or DST-based matrix is applied in the horizontal direction. A KLT-based integer matrix is applied. When the intra prediction mode is vertical, a DST-based or KLT-based integer matrix can be applied in the vertical direction, and a DCT-based integer matrix can be applied in the horizontal direction.

Additionally, in the case of DC mode, a DCT-based integer matrix can be applied in both directions.

And, in the case of intra prediction, the transform matrix may be adaptively determined based on the size of the transform unit (TU).

The quantization unit 130 determines a quantization step size for quantizing the coefficients of the residual block transformed by the transformation matrix, and the quantization step size may be determined for each quantization unit of a predetermined size or more.

The size of the quantization unit may be 8x8 or 16x16, and the quantization unit 130 quantizes the coefficients of the transform block using a quantization matrix determined according to the quantization step size and prediction mode.

Additionally, the quantization unit 130 may use the quantization step size of a quantization unit adjacent to the current quantization unit as a quantization step size predictor of the current quantization unit.

The quantization unit 130 searches in the order of the left quantization unit, the upper quantization unit, and the upper left quantization unit of the current quantization unit and generates a quantization step size predictor of the current quantization unit using one or two valid quantization step sizes. there is.

For example, the quantization unit 130 determines the first valid quantization step size found in the above order as the quantization step size predictor, or determines the average value of the two valid quantization step sizes found in the above order as the quantization step size predictor, or If only one quantization step size is valid, this can be determined as the quantization step size predictor.

Once the quantization step size predictor is determined, the quantization unit 130 transmits the difference value between the quantization step size of the current quantization unit and the quantization step size predictor to the entropy encoding unit 140.

Meanwhile, all of the left coding unit, upper coding unit, and upper left coding unit of the current coding unit do not exist. Alternatively, a previously existing coding unit may exist in coding order within the largest coding unit.

Accordingly, within the quantization units adjacent to the current coding unit and the maximum coding unit, the quantization step size of the quantization unit immediately preceding in coding order may be candidates.

In this case, the priority is set in the following order: 1) the left quantization unit of the current coding unit, 2) the upper left quantization unit of the current coding unit, 3) the upper left quantization unit of the current coding unit, and 4) the immediately previous quantization unit in coding order. It can be. The order may be changed, and the upper left quantization unit may be omitted.

Meanwhile, the quantized transform block as described above is transmitted to the inverse quantization unit 135 and the scanning unit 131.

The scanning unit 131 scans the coefficients of the quantized transform block and converts them into one-dimensional quantized coefficients. In this case, the coefficient distribution of the transform block after quantization may depend on the intra prediction mode, so the scanning method is used in the intra prediction mode. It can be decided accordingly.

Additionally, the coefficient scanning method may be determined differently depending on the size of the transformation unit, and the scan pattern may vary depending on the directional intra prediction mode. In this case, the scanning order of the quantization coefficients may be scanned in the reverse direction.

When the quantized coefficients are divided into a plurality of subsets, the same scan pattern may be applied to the quantized coefficients within each subset, and a zigzag scan or diagonal scan may be applied as the scan pattern between subsets.

Meanwhile, it is preferable that the scan pattern scans the remaining subsets in a forward direction from the main subset including DC, but the reverse direction is also possible.

Additionally, the scan pattern between subsets may be set to be the same as the scan pattern of quantized coefficients within the subset, and the scan pattern between subsets may be determined according to the intra prediction mode.

Meanwhile, the encoding device 10 includes information that can indicate the location of the last non-zero quantization coefficient in the transform unit (PU) and the location of the last non-zero quantization coefficient in each subset in the bitstream, and the decoding device ( 20).

The inverse quantization unit 135 inversely quantizes the quantized coefficient as described above, and the inverse transformation unit 125 performs inverse transformation in units of transform units (TUs) to restore the inverse quantized transform coefficient to a residual block in the spatial domain. can do.

The adder 195 may generate a restored block by adding the residual block restored by the inverse transform unit 125 and the prediction block received from the intra prediction unit 150 or the inter prediction unit 160.

In addition, the post-processing unit 170 performs a deblocking filtering process to remove the blocking effect that occurs in the restored picture, and a sample adaptive offset (Sample Adaptive Offset) to compensate for the difference value from the original image on a pixel basis. With the SAO) application process and coding unit, an adaptive loop filtering (ALF) process can be performed to compensate for the difference values from the original image.

The deblocking filtering process can be applied to the boundaries of prediction units (PUs) or transform units (TUs) that have a size greater than or equal to a predetermined size.

For example, the deblocking filtering process includes determining a boundary to be filtered, determining a boundary filtering strength to be applied to the boundary, determining whether to apply a deblocking filter, When it is determined to apply the deblocking filter, the method may include selecting a filter to be applied to the boundary.

Meanwhile, whether to apply the deblocking filter depends on i) whether the boundary filtering strength is greater than 0, and ii) the degree of change in pixel values at the boundaries of two blocks (P block, Q block) adjacent to the boundary to be filtered. It may be determined by whether the indicated value is smaller than the first reference value determined by the quantization parameter.

It is preferable that there are at least two filters. When the absolute value of the difference between two pixels located at the block boundary is greater than or equal to the second reference value, a filter that performs relatively weak filtering is selected.

The second reference value is determined by the quantization parameter and the boundary filtering strength.

In addition, the sample adaptive offset (SAO) application process is intended to reduce the distortion between the pixels in the image to which the deblocking filter is applied and the original pixel. The sample adaptive offset (SAO) application process is performed on a picture or slice basis. A decision can be made whether to perform it or not.

A picture or slice may be divided into a plurality of offset areas, and an offset type may be determined for each offset area, and the offset type may include a predetermined number (e.g., 4) of edge offset types and 2 band offsets. Can include types.

For example, when the offset type is an edge offset type, the edge type to which each pixel belongs is determined and the corresponding offset is applied, and the edge type can be determined based on the distribution of the two pixel values adjacent to the current pixel. there is.

The adaptive loop filtering (ALF) process can perform filtering based on the value of comparing the original image with the restored image that has gone through the deblocking filtering process or the adaptive offset application process.

The picture storage unit 180 receives post-processed image data from the post-processing unit 170 and restores and stores the image in units of pictures. The pictures may be images in frames or fields.

The inter prediction unit 160 may perform motion estimation using at least one reference picture stored in the picture storage unit 180 and determine a reference picture index and motion vector representing the reference picture.

In this case, according to the determined reference picture index and motion vector, a prediction block corresponding to the prediction unit to be encoded can be extracted from the reference picture used for motion estimation among the plurality of reference pictures stored in the picture storage unit 180. there is.

The intra prediction unit 150 may perform intra prediction coding using reconstructed pixel values within a picture including the current prediction unit.

The intra prediction unit 150 may receive a current prediction unit to be predicted and encoded and perform intra prediction by selecting one of a preset number of intra prediction modes according to the size of the current block.

The intra prediction unit 150 adaptively filters reference pixels to generate an intra prediction block, and when a reference pixel is not available, it can generate reference pixels using available reference pixels.

In addition, the entropy encoding unit 140 entropy encodes the quantization coefficient quantized by the quantization unit 130, intra prediction information received from the intra prediction unit 150, and motion information received from the inter prediction unit 160. You can.

FIG. 6 is a block diagram showing an embodiment of a configuration for performing inter prediction in the encoding device 10. The inter prediction encoder shown includes a motion information determination unit 161 and a motion information encoding mode determination unit 162. , It may be configured to include a motion information encoder 163, a prediction block generator 164, a residual block generator 165, a residual block encoder 166, and a multiplexer 167.

Referring to FIG. 6, the motion information determination unit 161 determines motion information of the current block, the motion information includes a reference picture index and a motion vector, and the reference picture index is one of the previously encoded and restored pictures. can represent.

When the current block is subjected to unidirectional inter prediction encoding, it represents one of the reference pictures belonging to list 0 (L0), and when the current block is bidirectionally predicted encoding, it represents one of the reference pictures belonging to list 0 (L0). It may include an index and a reference picture index indicating one of the reference pictures in List 1 (L1).

Additionally, when the current block is bidirectionally predicted encoded, it may include an index indicating one or two pictures among the reference pictures of the composite list (LC) generated by combining list 0 and list 1.

The motion vector indicates the position of the prediction block within the picture indicated by each reference picture index, and the motion vector may be in pixel units (integer units) or sub-pixel units.

For example, the motion vector may have a resolution of 1/2, 1/4, 1/8 or 1/16 pixels, and if the motion vector is not in integer units, the prediction block may be generated from pixels in integer units. You can.

The motion information encoding mode determination unit 162 may determine the encoding mode for the motion information of the current block to be one of skip mode, merge mode, and AMVP mode.

Skip mode is applied when there is a skip candidate with the same motion information as the motion information of the current block and the residual signal is 0, and the skip mode is applied when the current block, which is a prediction unit (PU), has the size and size of the coding unit (CU). Can be applied when are the same.

The merge mode is applied when there is a merge candidate with the same motion information as the motion information of the current block. The merge mode is applied when the current block has a different size or the same size as the coding unit (CU), and a residual signal exists. Applies to cases. Meanwhile, the merge candidate and the skip candidate may be the same.

AMVP mode is applied when skip mode and merge mode are not applied, and the AMVP candidate with the motion vector most similar to the motion vector of the current block can be selected as the AMVP predictor.

The motion information encoding unit 163 may encode the motion information according to the method determined by the motion information encoding mode determination unit 162.

For example, the motion information encoder 163 may perform a merge motion vector encoding process when the motion information encoding mode is skip mode or merge mode, and may perform an AMVP encoding process when the motion information encoding mode is AMVP mode.

The prediction block generator 164 generates a prediction block using the motion information of the current block. If the motion vector is in integer units, the prediction block generator 164 copies the block corresponding to the position indicated by the motion vector in the picture indicated by the reference picture index to create the current block. Generate a prediction block of

Meanwhile, when the motion vector is not in integer units, the prediction block generator 164 may generate pixels of the prediction block from integer unit pixels in the picture indicated by the reference picture index.

In this case, a prediction pixel may be generated using an 8-tap interpolation filter for the luminance pixel, and a prediction pixel may be generated using a 4-tap interpolation filter for the chrominance pixel.

The residual block generator 165 generates a residual block using the current block and a prediction block of the current block. If the size of the current block is 2Nx2N, the residual block is generated using the current block and a prediction block of size 2Nx2N corresponding to the current block. Blocks can be created.

Meanwhile, if the size of the current block used for prediction is 2NxN or Nx2N, the prediction block for each of the two 2NxN blocks constituting 2Nx2N is obtained, and then the final prediction block of size 2Nx2N is created using the two 2NxN prediction blocks. can be created.

In addition, a residual block of 2Nx2N size may be generated using the prediction block of 2Nx2N size, and overlap smoothing may be applied to the pixels of the boundary portion to resolve the discontinuity of the boundary portion of two prediction blocks of 2NxN size. You can.

The residual block encoder 166 divides the residual block into one or more transform units (TUs), and each transform unit (TU) can be transform-encoded, quantized, and entropy-encoded.

The residual block encoder 166 may transform the residual block generated by the inter prediction method using an integer-based transformation matrix, and the transformation matrix may be an integer-based DCT matrix.

Meanwhile, the residual block encoder 166 uses a quantization matrix to quantize the coefficients of the residual block transformed by the transformation matrix, and the quantization matrix may be determined by a quantization parameter.

The quantization parameter is determined for each coding unit (CU) of a predetermined size or larger, and if the current coding unit (CU) is smaller than the predetermined size, the first coding unit (CU) in coding order among the coding units (CUs) within the predetermined size ( Only the quantization parameters of the coding unit (CU) may be encoded and the quantization parameters of the remaining coding units (CUs) may not be encoded because they are the same as the above parameters.

Additionally, the coefficients of the transform block may be quantized using a quantization matrix determined according to the quantization parameter and prediction mode.

The quantization parameter determined for each coding unit (CU) of a predetermined size or larger may be predictively encoded using the quantization parameter of a coding unit (CU) adjacent to the current coding unit (CU).

A quantization parameter predictor of the current coding unit (CU) can be generated using one or two valid quantization parameters by searching in the order of the left coding unit (CU) and the upper coding unit (CU) of the current coding unit (CU). there is.

For example, the first valid quantization parameter searched in the above order can be determined as a quantization parameter predictor, and the first valid quantization parameter can be quantized by searching in the order of the left coding unit (CU) and the immediately previous coding unit (CU) in the coding order. It can be determined with a parameter predictor.

The coefficients of the quantized transform block are scanned and converted into one-dimensional quantization coefficients, and the scanning method can be set differently depending on the entropy encoding mode.

For example, when encoded with CABAC, the inter prediction encoded quantization coefficients may be scanned in a predetermined manner (zigzag, or raster scan in the diagonal direction), and when encoded with CAVLC, they may be scanned in a different manner from the above scheme. It can be.

For example, the scanning method may be determined according to zigzag in the case of inter, the intra prediction mode in the case of intra, and the coefficient scanning method may be determined differently depending on the size of the transformation unit.

Meanwhile, the scan pattern may vary depending on the directional intra prediction mode, and the scan order of quantization coefficients may be scanned in the reverse direction.

The multiplexer 167 multiplexes the motion information encoded by the motion information encoder 163 and the residual signals encoded by the residual block encoder 166.

The motion information may vary depending on the encoding mode. For example, in the case of skip or merge, it includes only an index indicating the predictor, and in the case of AMVP, it may include the reference picture index of the current block, differential motion vector, and AMVP index. .

Hereinafter, an embodiment of the operation of the intra prediction unit 150 shown in FIG. 1 will be described in detail.

First, the intra prediction unit 150 receives prediction mode information and the size of the prediction unit (PU) from the picture division unit 110, and stores reference pixels in order to determine the intra prediction mode of the prediction unit (PU). It can be read from (180).

The intra prediction unit 150 determines whether to generate a reference pixel by examining whether unavailable reference pixels exist, and the reference pixels can be used to determine the intra prediction mode of the current block.

If the current block is located at the upper border of the current picture, pixels adjacent to the upper side of the current block are not defined. If the current block is located at the left border of the current picture, pixels adjacent to the left side of the current block are not defined. The pixels may be determined to be not available pixels.

Additionally, even if the current block is located at a slice boundary and pixels adjacent to the upper or left side of the slice are not pixels that are encoded and restored first, it may be determined that they are not usable pixels.

As described above, when there are no pixels adjacent to the left or top of the current block, or when pixels that have been pre-encoded and restored do not exist, the intra prediction mode of the current block may be determined using only available pixels.

Meanwhile, a reference pixel at an unavailable position may be created using the available reference pixels of the current block. For example, if the pixels of the upper block are not available, some or all of the left pixels may be used to create the upper pixel. You can create them, and vice versa.

That is, a reference pixel is created by copying the available reference pixel at the closest location in a predetermined direction from a reference pixel at an unavailable location, or if there is no available reference pixel in the predetermined direction, the closest pixel in the opposite direction is created. A reference pixel may be created by copying an available reference pixel at a location.

Meanwhile, even if there are pixels above or to the left of the current block, the pixels may be determined to be unavailable reference pixels depending on the coding mode of the block to which they belong.

For example, if the block to which the reference pixel adjacent to the upper side of the current block belongs is a block that has been inter-coded and restored, the pixels may be determined to be unusable pixels.

In this case, available reference pixels can be generated using pixels belonging to the block in which a block adjacent to the current block is intra-coded and restored, and information that the encoding device 10 determines the available reference pixels according to the encoding mode is provided. Transmitted to the decryption device 20.

The intra prediction unit 150 determines the intra prediction mode of the current block using the reference pixels, and the number of intra prediction modes allowable for the current block may vary depending on the size of the block.

For example, if the size of the current block is 8x8, 16x16, 32x32, there may be 34 intra prediction modes, and if the size of the current block is 4x4, there may be 17 intra prediction modes.

The 34 or 17 intra prediction modes may be composed of at least one non-directional mode (non-directional mode) and a plurality of directional modes (directional modes).

The one or more non-directional modes may be DC mode and/or planar mode. When DC mode and planner mode are included as non-directional modes, 35 intra prediction modes may exist regardless of the size of the current block.

In this case, two non-directional modes (DC mode and planar mode) and 33 directional modes can be included.

In the case of planner mode, the prediction block of the current block is created using at least one pixel value (or predicted value of the pixel value, hereinafter referred to as the first reference value) located at the bottom-right of the current block and reference pixels. is created.

The configuration of the video decoding device according to an embodiment of the present invention can be derived from the configuration of the video encoding device 10 described with reference to FIGS. 1 to 6, for example, as described with reference to FIGS. 1 to 6. The video can be decoded by reversing the steps of the same video encoding method.

Figure 7 is a block diagram showing the configuration of a video decoding device according to an embodiment of the present invention. The decoding device 20 includes an entropy decoding unit 210, an inverse quantization/inverse transform unit 220, an adder 270, It is provided with a deblocking filter 250, a picture storage unit 260, an intra prediction unit 230, a motion compensation prediction unit 240, and an intra/inter conversion switch 280.

The entropy decoder 210 receives the encoded bit stream from the video encoding device 10, decodes it, separates it into an intra prediction mode index, motion information, and a quantization coefficient sequence, and stores the decoded motion information in a motion compensation prediction unit ( 240).

In addition, the entropy decoder 210 may transmit the intra prediction mode index to the intra prediction unit 230 and the inverse quantization/inverse transformation unit 220, and transmit the inverse quantization coefficient sequence to the inverse quantization/inverse transformation unit 220. .

The inverse quantization/inverse transformation unit 220 converts the quantization coefficient sequence into a two-dimensional array of inverse quantization coefficients, and can select one of a plurality of scanning patterns for the conversion, for example, the prediction mode of the current block (i.e. , intra prediction or inter prediction) and the scanning pattern can be selected based on the intra prediction mode.

The inverse quantization/inverse transform unit 220 restores the quantization coefficients by applying a quantization matrix selected from a plurality of quantization matrices to the inverse quantization coefficients of the two-dimensional array.

Meanwhile, different quantization matrices are applied depending on the size of the current block to be restored, and even for blocks of the same size, a quantization matrix may be selected based on at least one of the prediction mode and intra prediction mode of the current block.

The inverse quantization/inverse transformation unit 220 restores the residual block by inversely transforming the restored quantization coefficient, and the inverse transformation process may be performed using a transformation unit (TU) as a basic unit.

The adder 270 restores an image block by adding the residual block restored by the inverse quantization/inverse transform unit 220 and the prediction block generated by the intra prediction unit 230 or the motion compensation prediction unit 240.

The deblocking filter 250 may perform deblocking filter processing on the restored image generated by the adder 270 to reduce deblocking artifacts caused by image loss due to the quantization process.

The picture storage unit 260 is a frame memory for storing a local decoded image on which deblocking filter processing has been performed by the deblocking filter 250.

The intra prediction unit 230 restores the intra prediction mode of the current block based on the intra prediction mode index received from the entropy decoder 210 and generates a prediction block according to the restored intra prediction mode.

The motion compensation prediction unit 240 generates a prediction block for the current block from the picture stored in the picture storage unit 260 based on the motion vector information, and when fractional precision motion compensation is applied, the selected interpolation filter is applied to the prediction block. can be created.

The intra/inter conversion switch 280 may provide a prediction block generated by either the intra prediction unit 230 or the motion compensation prediction unit 240 to the adder 270 based on the encoding mode.

FIG. 8 is a block diagram showing an embodiment of a configuration for performing inter prediction in the video decoding device 20. The inter prediction decoder includes a demultiplexer 241, a motion information encoding mode determination unit 242, and a merge mode motion. It includes an information decoding unit 243, an AMVP mode motion information decoding unit 244, a prediction block generating unit 245, a residual block decoding unit 246, and a restored block generating unit 247. Here, the merge mode motion information decoding unit 243 and the AMVP mode motion information decoding unit 244 may be included in the motion information decoding unit 248.

Referring to FIG. 8, the de-multiplexer 241 demultiplexes the currently encoded motion information and the encoded residual signals from the received bitstream, and transmits the demultiplexed motion information to the motion information encoding mode determination unit 242. And, the demultiplexed residual signal can be transmitted to the residual block decoding unit 246.

The motion information coding mode determination unit 242 determines the motion information coding mode of the current block, and when skip_flag of the received bitstream has a value of 1, it determines that the motion information coding mode of the current block is encoded in skip coding mode. can do.

When skip_flag of the received bitstream has a value of 0 and the motion information received from the de-multiplexer 241 has only a merge index, the motion information encoding mode determination unit 242 determines the motion information encoding mode of the current block. It can be determined that is encoded in merge mode.

In addition, the motion information encoding mode determination unit 242 determines that skip_flag of the received bitstream has a value of 0, and the motion information received from the demultiplexer 241 has a reference picture index, a differential motion vector, and an AMVP index. In this case, it may be determined that the motion information encoding mode of the current block is encoded in AMVP mode.

The merge mode motion information decoder 243 is activated when the motion information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is skip or merge mode, and the AMVP mode motion information decoder 244 is activated when the motion information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is skip or merge mode. It may be activated when the information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is AMVP mode.

The prediction block generator 245 generates a prediction block of the current block using the motion information restored by the merge mode motion information decoder 243 or the AMVP mode motion information decoder 244.

When the motion vector is an integer unit, a prediction block of the current block can be generated by copying the block corresponding to the position indicated by the motion vector within the picture indicated by the reference picture index.

Meanwhile, if the motion vector is not in integer units, pixels of the prediction block are generated from integer unit pixels in the picture indicated by the reference picture index. In this case, an 8-tap interpolation filter is used for luminance pixels, and for chrominance pixels, an 8-tap interpolation filter is used. Predicted pixels can be generated using a 4-tap interpolation filter.

The residual block decoder 246 entropy decodes the residual signal and inversely scans the entropy decoded coefficients to generate a two-dimensional quantized coefficient block. The inverse scanning method may vary depending on the entropy decoding method.

For example, when decoded based on CABAC, the diagonal raster reverse scan method may be applied, and when decoded based on CAVLC, the reverse scanning method may be applied using a zigzag reverse scan method. Additionally, the reverse scanning method may be determined differently depending on the size of the prediction block.

The residual block decoder 246 dequantizes the coefficient block generated as described above using an inverse quantization matrix, and can restore quantization parameters to derive the quantization matrix. Here, the quantization step size can be restored for each coding unit of a predetermined size or larger.

The residual block decoder 260 restores the residual block by inversely transforming the inverse quantized coefficient block.

The restored block generator 270 generates a restored block by adding the prediction block generated by the prediction block generator 250 and the residual block generated by the residual block decoding unit 260.

Hereinafter, an embodiment of the process of restoring the current block through intra prediction will be described with reference to FIG. 7 again.

First, the intra prediction mode of the current block is decoded from the received bitstream, and for this, the entropy decoder 210 refers to one of the plurality of intra prediction mode tables to restore the first intra prediction mode index of the current block. You can.

The plurality of intra prediction mode tables are tables shared by the encoding device 10 and the decoding device 20, and any one table selected according to the distribution of intra prediction modes for a plurality of blocks adjacent to the current block can be applied. .

For example, if the intra prediction mode of the block to the left of the current block and the intra prediction mode of the block above the current block are the same, apply the first intra prediction mode table to restore the first intra prediction mode index of the current block, and if they are not the same, Otherwise, the first intra prediction mode index of the current block can be restored by applying the second intra prediction mode table.

As another example, when the intra prediction modes of the upper block and the left block of the current block are both directional intra prediction modes, the direction of the intra prediction mode of the upper block and the direction of the intra prediction mode of the left block If it is within this predetermined angle, the first intra prediction mode table is applied to restore the first intra prediction mode index of the current block, and if it is outside the predetermined angle, the second intra prediction mode table is applied to restore the first intra prediction mode index of the current block. can also be restored.

The entropy decoder 210 transmits the first intra prediction mode index of the restored current block to the intra prediction unit 230.

The intra prediction unit 230, which has received the index of the first intra prediction mode, may determine the maximum possible mode of the current block as the intra prediction mode of the current block when the index has the minimum value (i.e., 0). .

Meanwhile, if the index has a value other than 0, the intra prediction unit 230 compares the index indicated by the maximum possible mode of the current block with the first intra prediction mode index, and as a result of the comparison, the first intra prediction mode If the index is not smaller than the index indicated by the maximum possible mode of the current block, the intra prediction mode corresponding to the second intra prediction mode index obtained by adding 1 to the first intra prediction mode index is determined as the intra prediction mode of the current block, otherwise, Otherwise, the intra prediction mode corresponding to the first intra prediction mode index may be determined as the intra prediction mode of the current block.

The intra prediction mode allowable for the current block may consist of at least one non-directional mode (non-directional mode) and a plurality of directional modes (directional modes).

The one or more non-directional modes may be DC mode and/or planar mode. Additionally, either DC mode or planner mode may be adaptively included in the allowable intra prediction mode set.

To this end, information specifying a non-directional mode included in the allowable intra prediction mode set may be included in the picture header or slice header.

Next, the intra prediction unit 230 reads reference pixels from the picture storage unit 260 to generate an intra prediction block, and determines whether there is an unavailable reference pixel.

The determination may be made based on the presence or absence of reference pixels used to generate an intra prediction block by applying the decoded intra prediction mode of the current block.

Next, when it is necessary to generate reference pixels, the intra prediction unit 230 can generate reference pixels at unavailable positions using previously restored available reference pixels.

The definition of an unavailable reference pixel and the method of generating the reference pixel may be the same as the operation in the intra prediction unit 150 according to FIG. 1, but an intra prediction block is generated according to the decoded intra prediction mode of the current block. Reference pixels used to do so may be selectively restored.

Additionally, the intra prediction unit 230 determines whether to apply filtering to reference pixels to generate a prediction block, that is, whether to apply filtering to reference pixels to generate an intra prediction block of the current block. It can be determined based on the decoded intra prediction mode and the size of the current prediction block.

Since the problem of blocking artifacts increases as the size of the block increases, the number of prediction modes that filter reference pixels can be increased as the size of the block increases. However, if the block becomes larger than a certain size, it can be viewed as a flat area, reducing complexity. Reference pixels may not be filtered for .

If it is determined that a filter needs to be applied to the reference pixel, the intra prediction unit 230 filters the reference pixels using a filter.

At least two or more filters may be adaptively applied depending on the degree of difference in level between the reference pixels. It is desirable that the filter coefficient of the filter be symmetrical.

In addition, the above two or more filters may be adaptively applied depending on the size of the current block. When applying the filter, a narrow bandwidth filter is applied to small-sized blocks, and a wide-bandwidth filter is applied to large-sized blocks. may apply.

In the case of DC mode, there is no need to apply a filter because the prediction block is created with the average value of the reference pixels. In the vertical mode, where the images are correlated in the vertical direction, there is no need to apply a filter to the reference pixel, and the image is horizontal. Even in horizontal mode, which is directionally relevant, there may be no need to apply filters to reference pixels.

As such, whether or not filtering is applied is also related to the intra prediction mode of the current block, so reference pixels can be adaptively filtered based on the intra prediction mode of the current block and the size of the prediction block.

Next, the intra prediction unit 230 generates a prediction block using reference pixels or filtered reference pixels according to the restored intra prediction mode, and the generation of the prediction block is the same as the operation in the encoding device 10. Since this can be done, a detailed description thereof will be omitted.

The intra prediction unit 230 determines whether to filter the generated prediction block, and whether or not to filter the generated prediction block may be determined using information included in a slice header or a coding unit header or according to the intra prediction mode of the current block.

When it is determined that the generated prediction block is to be filtered, the intra prediction unit 230 may generate a new pixel by filtering a pixel at a specific position of the generated prediction block using available reference pixels adjacent to the current block. .

For example, in DC mode, among prediction pixels, prediction pixels that are in contact with reference pixels may be filtered using reference pixels that are in contact with the prediction pixels.

Therefore, the prediction pixel is filtered using one or two reference pixels depending on the location of the prediction pixel, and filtering of the prediction pixel in DC mode can be applied to prediction blocks of any size.

Meanwhile, in the vertical mode, prediction pixels in contact with the left reference pixel among the prediction pixels of the prediction block can be changed using reference pixels other than the upper pixel used to generate the prediction block.

Similarly, in the horizontal mode, among the generated prediction pixels, prediction pixels that are in contact with the upper reference pixel may be changed using reference pixels other than the left pixel used to generate the prediction block.

In this way, the current block can be restored using the prediction block of the restored current block and the residual block of the decoded current block.

Figure 9 is a diagram to explain a second embodiment of a method of dividing and processing an image in blocks.

Referring to FIG. 9, a coding tree unit (CTU) with a maximum size of 256x256 pixels can be first divided into a quad tree structure and divided into four coding units (CUs) having a square shape.

Here, at least one of the coding units divided into the quad tree structure may be divided into a binary tree structure and re-divided into two coding units (CUs) having a rectangular shape.

Meanwhile, at least one of the coding units divided into the quad tree structure may be divided into the quad tree structure and re-divided into four coding units (CUs) having a square shape.

And at least one of the coding units re-divided into the binary tree structure may be further divided into the binary tree structure into two coding units (CUs) having a square or rectangular shape.

Meanwhile, at least one of the coding units re-divided into the quad tree structure may be further divided into a quad tree structure or a binary tree structure and divided into coding units (CUs) having a square or rectangular shape.

Coding blocks (CBs) configured by being divided into a binary tree structure as described above are no longer divided and can be used for prediction and transformation. That is, the size of the prediction unit (PU) and transform unit (TU) belonging to the coding block (CB) as shown in FIG. 9 may be the same as the size of the corresponding coding block (CB).

The coding unit divided into the quad tree structure as described above may be divided into one or two or more prediction units (PUs) using the method described with reference to FIGS. 3 and 4.

In addition, the coding unit divided into a quad tree structure as described above may be divided into one or two or more transform units (TUs) using the method described with reference to FIG. 5, and the divided transform units (TUs) can have a maximum size of 64x64 pixels.

Additionally, the syntax structure used to divide and process an image in block units can represent the division information using a flag. For example, whether a coding unit (CU) is split can be indicated using split_cu_flag, and the depth of a coding unit (CU) split using a binary tree can be indicated using binary_depth. Additionally, whether the coding unit (CU) is split into a binary tree structure may be indicated as a separate binary_split_flag.

Blocks (e.g., coding unit (CU), prediction unit (PU), and transform unit (TU)) divided by the method described with reference to FIG. 9 are as described with reference to FIGS. 1 to 8. By applying the same methods, encoding and decoding of the image can be performed.

Hereinafter, with reference to FIGS. 10 to 15, further embodiments of a method for dividing a coding unit (CU) into one or two or more transform units (TUs) will be described.

According to an embodiment of the present invention, a coding unit (CU) may be divided into a binary tree structure into transformation units (TUs), which are basic units in which transformation on the residual block is performed.

Referring to FIG. 10, at least one of the rectangular coding blocks (CB0, CB1) divided into a binary tree structure and having a size of Nx2N or 2NxN is divided into a binary tree structure again to form a square conversion unit having a size of NxN. It can be divided into (TU0, TU1).

As described above, the block-based image encoding method can perform prediction, transformation, quantization, and entropy coding steps.

In the prediction step, a prediction signal is generated by referring to the block currently performing encoding and the existing encoded image or surrounding images, and through this, a differential signal with the current block can be calculated.

Meanwhile, in the conversion step, conversion is performed using various conversion functions using the differential signal as an input, and the converted signal is classified into DC coefficients and AC coefficients and energy compaction is performed to improve coding efficiency. You can.

Additionally, in the quantization step, quantization is performed using transform coefficients as input, and then entropy coding is performed on the quantized signal, so that the image can be encoded.

Meanwhile, the image decoding method proceeds in the reverse order of the above encoding process, and image quality distortion may occur in the quantization step.

As a method to reduce image quality distortion while improving coding efficiency, the size or shape of the conversion unit (TU) and the type of conversion function applied can be varied depending on the distribution of the input differential signal and the characteristics of the image in the conversion step. there is.

For example, when finding a block similar to the current block through a block-based motion estimation process in the prediction stage, a cost measurement method such as SAD (Sum of Absolute Difference) or MSE (Mean Square error) is used to calculate the difference. The distribution of signals can take various forms depending on the characteristics of the image.

Accordingly, effective encoding can be performed by performing conversion by selectively determining the size or shape of the conversion unit (CU) based on the distribution of various differential signals.

For example, when a differential signal occurs in a random coding block (CBx), effective transformation can be performed by dividing the coding block (CBx) into two transform units (TUs) by dividing it into a binary tree structure. . The DC value can generally be said to represent the average value of the input signal, so when a differential signal is received as an input to a conversion process, the DC value can be effectively represented by dividing the coding block (CBx) into two transformation units (TU). there is.

Referring to FIG. 11, a square coding unit (CU0) with a size of 2Nx2N may be divided into a binary tree structure into rectangular transformation units (TU0 and TU1) with a size of Nx2N or 2NxN.

According to another embodiment of the present invention, the step of dividing a coding unit (CU) into a binary tree structure as described above may be repeated two or more times to divide the coding unit (CU) into a plurality of transformation units (TU).

Referring to FIG. 12, a rectangular coding block (CB1) having a size of Nx2N is divided into a binary tree structure, and the divided block having a size of NxN is further divided into a binary tree structure to form N/2xN or NxN/2. After constructing a rectangular block with a size of , TU4, TU5).

Referring to FIG. 13, a square coding block (CB0) with a size of 2Nx2N is divided into a binary tree structure, and the divided block with a size of Nx2N is again divided into a binary tree structure to form a square coding block with a size of NxN. After configuring the block, the block having the size of NxN can be divided into a binary tree structure again into rectangular transformation units (TU1, TU2) having the size of N/2xN.

Referring to FIG. 14, a rectangular coding block (CB0) with a size of 2NxN is divided into a binary tree structure, and the divided block with a size of NxN is again divided into a quad tree structure to have a size of N/2xN/2. It can be divided into square transformation units (TU1, TU2, TU3, TU4).

See FIGS. 1 to 8 for blocks (e.g., coding units (CUs), prediction units (PUs), and transform units (TUs)) divided by the method described with reference to FIGS. 10 to 14. By applying the methods described above, encoding and decoding of the image can be performed.

Below, embodiments of a method for determining a block division structure by the encoding device 10 according to the present invention will be described.

The picture division unit 110 provided in the video encoding device 10 performs RDO (Rate Distortion Optimization) according to a preset order, and divides into divisible coding units (CUs), prediction units (PUs), and transforms as described above. The division structure of the unit (TU) can be determined.

For example, to determine the block division structure, the picture division unit 110 performs RDO-Q (Rate Distortion Optimization-Quantization) and determines the optimal block division structure in terms of bitrate and distortion. You can decide.

Referring to FIG. 15, when the coding unit (CU) has a 2Nx2N pixel size, the 2Nx2N pixel size shown in (a), the NxN pixel size shown in (b), and the Nx2N pixel size shown in (c). , the optimal division structure of the conversion unit (PU) can be determined by performing RDO in the order of the 2NxN pixel size PU division structure shown in (d).

Referring to FIG. 16, when the coding unit (CU) has a Nx2N or 2NxN pixel size, the pixel size of Nx2N (or 2NxN) shown in (a), the pixel size of NxN shown in (b), Pixel sizes of N/2xN (or NxN/2) and NxN shown in (c), pixel sizes of N/2xN/2, N/2xN, and NxN shown in (d), N shown in (e) The optimal division structure of the transformation unit (PU) can be determined by performing RDO in the order of the division structure of the transformation unit (PU) with a pixel size of /2xN.

In the above, the block division method of the present invention was explained by taking as an example that the block division structure is determined by performing RDO (Rate Distortion Optimization), but the picture division unit 110 uses the Sum of Absolute Difference (SAD) or Mean Square Error (MSE) ) can be used to determine the block division structure, thereby reducing complexity and maintaining appropriate efficiency.

Hereinafter, the image processing method and the resulting encoding and decoding methods according to an embodiment of the present invention will be described in more detail.

17 and 18 are flowcharts illustrating an image processing method for processing motion information for parallel processing according to an embodiment of the present invention, and FIGS. 19 to 22 illustrate a motion information processing method according to an embodiment of the present invention. These are drawings to illustrate each case.

As described above, when the motion compensation unit 160 of the encoding device 10 and the motion information decoding unit 248 of the decoding device 20 process motion information encoding and decoding in merge and AMVP modes, respectively, Processing such as compensation, mode decision, and entropy encoding can each be implemented as a hardware module in a parallel pipeline system.

Accordingly, if the mode decision of the previous block ends later than the processing time of the current block, motion compensation processing of the current block cannot proceed, which may cause a pipeline stall.

To solve this problem, according to an embodiment of the present invention, the encoding device 10 or the decoding device 20 sequentially groups a plurality of blocks into parallel motion information prediction units according to an arbitrary predetermined size (S101). , Obtain the motion information of at least one available block among the neighboring blocks excluding the block belonging to the previous parallel motion information prediction unit of the current block (S103), and construct a motion information prediction list based on the obtained motion information. By doing so (S105), motion information prediction decoding processing for each mode based on the motion information prediction list can be performed (S107). However, in step S103, if there are no blocks available, the motion information prediction list in step S105 is constructed using a predetermined zero motion vector (zero MV) or a motion vector at the same position (collocated MV) of the previous frame. It can be.

More specifically, the encoding and decoding target blocks may be sequentially grouped into parallel motion information prediction units (PMUs; parallel MVP units) of a predetermined size according to the encoding and decoding order.

Also, if the nth parallel motion information prediction unit to which the current decoding target block belongs is called PMU(n), the motion compensation unit 160 or the motion information decoding unit 248 provides motion vector information of neighboring blocks, such as AMVP or Merge. When performing the necessary encoding and decoding processing, blocks included in the PMU (n-1) that were encoded or decoded immediately before the current block may be excluded from the neighboring blocks.

That is, the motion compensation unit 160 or the motion information decoder 248 determines that the motion information of the block included in the PMU (n-1) encoded/decoded immediately before the PMU (n) to which the current block belongs is the current block. It can be excluded when constructing the motion information prediction (MVP) candidate list. Accordingly, the MVP candidate list corresponding to the current block can be determined from the motion information of blocks not included in the PMU (n-1) among neighboring blocks.

However, because this may reduce the number of motion vectors of available neighboring blocks, in order to prevent a decrease in coding efficiency, as shown in FIG. 19, more neighboring blocks (F ~ E) are used compared to the existing neighboring blocks (A ~ E). I) The motion information can be further used to construct a motion vector prediction (MVP) candidate list.

Accordingly, at least one of the motion vectors of blocks available (e.g., blocks in the same picture/tile/slice) among neighboring blocks not included in the PMU (n-1) is entered into the final motion vector prediction (MVP) candidate list. can be decided.

Meanwhile, as shown in FIG. 18, the motion compensation unit 160 or the motion information decoding unit 248 decodes motion information using neighboring blocks at predetermined positions, and decodes the current block belonging to the PMU(n). Identify the surrounding blocks to be used for motion prediction (S201), and if there are neighboring blocks included in the PMU (n-1) (S203), substitute information for the motion information of the neighboring blocks included in the PMU (n-1) is acquired (S205), and motion prediction decoding of the current block using the replacement information is processed without motion information of neighboring blocks included in the PMU (n-1) (S207).

Here, the replacement information may be calculated as a predetermined default value even if the actual motion vector information of the corresponding location is not known, or may be a value derived from neighboring blocks included in the PMU(n). For example, replacement information for a neighboring block included in PMU(n-1) may consist of a zero vector (zero MV) or a motion vector (collocated MV) at the same location in the previous frame.

For this processing, the encoding device 10 may transmit PMU size information by including it in the form of syntax in sequence, picture, or slice header information. PMU size information can be signaled explicitly or implicitly as values related to CTU size, CU (or PU) minimum and maximum size.

In addition, the encoding device 10 may sequentially group PMUs according to a preset size, index each unit by assigning a parallel motion information prediction unit index, and the assigned index information may be explicitly transmitted, or Alternatively, index information may be derived from the decoding device 20.

For example, the size of the PMU may always be the same as the minimum PU size, or the PMU size may be the same as the size of the PU or CU currently being encoded/decoded. In such cases, PMU size information may not be transmitted explicitly, and the index may be identified according to sequential grouping.

Referring to FIG. 20, the thick line represents the CTU, the solid inner line represents the PU, and the red dotted line represents the PMU.

According to an embodiment of the present invention, the lower left motion vector among the surrounding motion vectors of the current PU may be the motion vector of the block included in the immediately previous PMU (n-1), depending on the encoding and decoding order. . Accordingly, in motion vector prediction encoding and decoding, the encoding device 10 and the decoding device 20 exclude the motion vector of the neighboring block included in the immediately previous PMU (n-1) when constructing the motion vector candidate list. You can. And, as replacement information for the block included in the PMU (n-1) of the immediately previous index, the zero vector (zero MV) or the motion vector at the same position in the previous frame (collocated MV) can be used when constructing the motion vector candidate list. there is.

Meanwhile, referring to FIG. 21, since the neighboring block in the upper right is included in the immediately previous PMU (n-1), it is excluded from the candidate motion vector in the motion vector prediction encoding and decoding of the current block, and instead uses the zero vector ( zero MV) or a motion vector at the same location (collocated MV) of the previous frame can be used as a candidate motion vector.

And, referring to FIG. 22, a case where all neighboring blocks on the left are included in the previous PMU(n-1) is illustrated. As shown in FIG. 22, in the motion vector prediction encoding and decoding process, the encoding device 10 and the decoding device 20 exclude the motion vectors of all left neighboring blocks from the candidate motion vectors, and instead use the top (upper left and Upper right) Using the motion vectors of neighboring blocks, a candidate list for motion vector prediction can be constructed.

With this configuration, it is possible to separate dependencies between parallel motion information prediction units, and motion information decoding can be processed without waiting for the completion of processing of blocks processed in the previous pipeline among neighboring blocks, resulting in pipeline stall (pipeline stall). stall) can be blocked in advance, thereby improving encoding and decoding efficiency.

The method according to the present invention described above can be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of computer-readable recording media include ROM, RAM, CD-ROM, and magnetic tape. , floppy disks, optical data storage devices, etc., and also includes those implemented in the form of carrier waves (for example, transmission via the Internet).

The computer-readable recording medium is distributed in a computer system connected to a network, so that computer-readable code can be stored and executed in a distributed manner. And, functional programs, codes, and code segments for implementing the method can be easily deduced by programmers in the technical field to which the present invention pertains.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

Claims (3)

In the video decoding method,
Deriving a first motion information candidate using spatial neighboring blocks of the current block;
Deriving a second motion information candidate using at least one block included in a specific area of the current picture excluding the spatial neighboring blocks;
generating a candidate list using at least one of the first motion information candidate and the second motion information candidate; and
Selecting a motion information candidate from the candidate list using motion information obtained from a bitstream, and generating a prediction block of the current block based on the selected motion information candidate,
The specific area included in the current picture is a Coding Tree Unit including the current block, and at least one block included in the specific area is a block that has been pre-coded and decoded before the current block,
An image decoding method, wherein the spatial neighboring blocks include a left block and an upper block of the current block. In the video encoding method,
Deriving a first motion information candidate using spatial neighboring blocks of the current block;
Deriving a second motion information candidate using at least one block included in a specific area of the current picture excluding the spatial neighboring blocks;
generating a candidate list using at least one of the first motion information candidate and the second motion information candidate;
selecting a motion information candidate from the candidate list and generating a prediction block of the current block using the selected motion information candidate; and
Including the step of encoding motion information to indicate the selected motion information candidate,
The specific area included in the current picture is a Coding Tree Unit including the current block, and at least one block included in the specific area is a block that is less coded than the current block,
An image encoding method, wherein the spatial neighboring blocks include a left block and an upper block of the current block. In a non-transitory computer-readable recording medium containing a bitstream generated by an image encoding method,
The video encoding method is,
Deriving a first motion information candidate using spatial neighboring blocks of the current block;
Deriving a second motion information candidate using at least one block included in a specific area of the current picture excluding the spatial neighboring blocks;
generating a candidate list using at least one of the first motion information candidate and the second motion information candidate;
selecting a motion information candidate from the candidate list and generating a prediction block of the current block using the selected motion information candidate; and
Including the step of encoding motion information to indicate the selected motion information candidate,
The specific area included in the current picture is a Coding Tree Unit including the current block, and at least one block included in the specific area is a block that is less coded than the current block,
A non-transitory computer-readable recording medium including a bitstream generated by an image encoding method, wherein the spatial neighboring blocks include a left block and an upper block of the current block.