KR20170058854A - Method and apparatus for adaptive motion search range control in the parallel processing encoding environment - Google Patents

Abstract

A method and an apparatus for performing motion search on an encoding unit included in an encoding target region are disclosed. According to an embodiment of the present invention, the method includes a step of determining the motion complexity of an encoding target region; a step of setting a motion search region based on the determined motion complexity, and a step of performing motion search for the encoding unit based on the set motion search region. So, an image can be encoded or decoded.

Description

TECHNICAL FIELD The present invention relates to a method and apparatus for setting an adaptive motion search area in a parallel processing coding environment,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present disclosure relates to image compression, and more particularly, to a coding method and apparatus using an adaptive search range setting in inter-picture prediction in a parallel processing coding environment.

Recently, the demand for high resolution and high quality images such as high definition (HD) image and ultra high definition (UHD) image is increasing in various applications. As the image data has high resolution and high quality, the amount of data increases relative to the existing image data. Therefore, when the image data is transmitted using a medium such as a wired / wireless broadband line or stored using an existing storage medium, The storage cost is increased. High-efficiency image compression techniques can be utilized to solve such problems as image data becomes high-resolution and high-quality.

Further, researches on dividing an image into a plurality of regions and parallel processing of the respective regions are underway, and researches are being conducted on techniques capable of minimizing coding loss due to parallel processing.

SUMMARY OF THE INVENTION The present invention provides a method and apparatus for encoding / decoding an image.

It is another object of the present disclosure to provide a method and apparatus for encoding / decoding an image in a parallel processing coding environment.

According to another aspect of the present invention, there is provided a method and apparatus for adaptively setting a motion search region in coding an image in a parallel processing coding environment.

It is another object of the present disclosure to provide a method and apparatus for performing a motion search based on an adaptively set search region.

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to an aspect of the present disclosure, there is provided a motion search method for a coding unit included in a coding target area, the method comprising: determining a motion complexity of the coding target area; Setting a motion search area based on the determined motion complexity; And performing a motion search for the coding unit based on the set motion search area.

The features briefly summarized above for this disclosure are only exemplary aspects of the detailed description of the disclosure which follow, and are not intended to limit the scope of the disclosure.

According to the present disclosure, it is possible to efficiently encode / decode an image.

Further, according to the present disclosure, it is possible to efficiently encode / decode an image in a parallel processing coding environment.

Further, according to the present disclosure, in encoding an image in a parallel processing coding environment, a motion search area can be adaptively set to minimize coding loss due to parallel processing.

Further, according to the present disclosure, it is possible to minimize coding complexity by performing adaptive motion search based on an adaptively set motion search area.

Further, according to the present disclosure, when a coding unit located at the boundary of a tile is coded in a parallel coding environment in which an image is divided into a plurality of tiles and each tile is processed in parallel, a search area is adaptively adjusted To-picture prediction mode so that the encoding loss due to the tile parallel processing can be minimized.

In addition, according to the present disclosure, it is possible to adaptively search only a coding unit that is located at a boundary of a tile and is likely to be coded by inter-picture prediction, instead of applying a uniformly resetting search region to all coding units included in a tile, It is possible to prevent the encoding complexity from becoming large.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.
2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.
FIG. 3 is a conceptual illustration of a parallel processing technique based on slices, tiles, and WPP among various data-level parallel processing techniques.
4 is a diagram illustrating a case where a picture including four tiles is coded in a single core and a case in which a plurality of cores are coded in parallel and a coding unit in a picture is coded Fig.
5 is a diagram for explaining encoding loss according to tile encoding.
6 is a flowchart illustrating a method of performing a motion search based on a search area adaptively set according to an embodiment of the present disclosure.
FIG. 7 is a diagram for explaining a diamond search and a raster search in the integer pixel-based motion search method.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . In addition, the content of "comprising" in this disclosure does not exclude a configuration other than the configuration of the present invention, and means that additional configurations can be included in the scope of the present invention or the practice of the present disclosure.

In addition, the components of the present disclosure are shown separately to represent different characteristic functions, and do not mean that each component is composed of separate hardware or a single software component unit. That is, each constituent unit is included in each constituent unit for convenience of explanation, and at least two constituent units of the constituent units may be combined to form one constituent unit, or one constituent unit may be divided into a plurality of constituent units to perform a function. The integrated and separate embodiments of the components are also included in the scope of the present disclosure without departing from the spirit of the present disclosure.

In addition, some of the elements are not essential components to perform essential functions in the present disclosure, but may be optional components only to improve performance. This disclosure can be implemented only with components that are essential to implementing the essentials of the present disclosure, except for those components used for performance enhancement, and only those components that include only the essential components, Are also included in the scope of the present disclosure.

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

1, the image encoding apparatus 100 includes a picture division unit 110, prediction units 120 and 125, a transform unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit An inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155. [

Each of the components shown in FIG. 1 is shown independently to represent different characteristic functions in the image encoding apparatus, and does not mean that each component is composed of separate hardware or one software configuration unit. That is, each constituent unit is included in each constituent unit for convenience of explanation, and at least two constituent units of the constituent units may be combined to form one constituent unit, or one constituent unit may be divided into a plurality of constituent units to perform a function. The integrated and separate embodiments of the components are also included in the scope of the present disclosure without departing from the spirit of the present disclosure.

The picture division unit 110 may divide the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture division unit 110 divides one picture into a plurality of coding units, a prediction unit, and a combination of conversion units, and generates a coding unit, a prediction unit, and a conversion unit combination So that the picture can be encoded.

For example, one picture may be divided into a plurality of coding units. In order to divide a coding unit in a picture, a recursive tree structure such as a quad tree structure can be used. In a coding or decoding scheme in which one picture or a largest coding unit is used as a root and divided into other coding units A unit can be divided with as many child nodes as the number of divided coding units. Under certain constraints, an encoding unit that is no longer segmented becomes a leaf node. That is, when it is assumed that only one square division is possible for one coding unit, one coding unit can be divided into a maximum of four different coding units.

Hereinafter, in the embodiment of the present disclosure, the encoding unit may be used as a unit for performing encoding, or may be used as a unit for performing decoding.

The prediction unit may be one divided into at least one square or rectangular shape having the same size in one coding unit, and one of the prediction units in one coding unit may be divided into another prediction Or may have a shape and / or size different from the unit.

If a prediction unit performing intra prediction on the basis of an encoding unit is not the minimum encoding unit at the time of generation, intraprediction can be performed without dividing the prediction unit into a plurality of prediction units NxN.

The prediction units 120 and 125 may include an inter prediction unit 120 for performing inter prediction and an intra prediction unit 125 for performing intra prediction. It is possible to determine whether to use inter prediction or intra prediction for a prediction unit and to determine concrete information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. At this time, the processing unit in which the prediction is performed may be different from the processing unit in which the prediction method and the concrete contents are determined. For example, the method of prediction, the prediction mode and the like are determined as a prediction unit, and the execution of the prediction may be performed in a conversion unit. The residual value (residual block) between the generated prediction block and the original block can be input to the conversion unit 130. [ In addition, the prediction mode information, motion vector information, and the like used for prediction can be encoded by the entropy encoding unit 165 together with the residual value and transmitted to the decoder. When a particular encoding mode is used, it is also possible to directly encode the original block and transmit it to the decoding unit without generating a prediction block through the prediction units 120 and 125.

The inter-prediction unit 120 may predict a prediction unit based on information of at least one of a previous picture or a following picture of the current picture, and may predict a prediction unit based on information of a partially- Unit may be predicted. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

In the reference picture interpolating section, the reference picture information is supplied from the memory 155 and pixel information of an integer pixel or less can be generated in the reference picture. In the case of a luminance pixel, a DCT-based interpolation filter having a different filter coefficient may be used to generate pixel information of an integer number of pixels or less in units of quarter pixels. In the case of a color difference signal, a DCT-based 4-tap interpolation filter having a different filter coefficient may be used to generate pixel information of an integer number of pixels or less in units of 1/8 pixel.

The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolating unit. Various methods such as Full Search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) can be used as methods for calculating motion vectors. The motion vector may have a motion vector value of 1/2 or 1/4 pixel unit based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by making the motion prediction method different. Various methods such as a skip method, a merge method, an AMVP (Advanced Motion Vector Prediction) method, and an Intra Block Copy method can be used as the motion prediction method.

The intra prediction unit 125 can generate a prediction unit based on reference pixel information around the current block which is pixel information in the current picture. In the case where the neighboring block of the current prediction unit is the block in which the inter prediction is performed so that the reference pixel is the pixel performing the inter prediction, the reference pixel included in the block in which the inter prediction is performed is referred to as the reference pixel Information. That is, when the reference pixel is not available, the reference pixel information that is not available may be replaced by at least one reference pixel among the available reference pixels.

In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction, and a non-directional mode in which direction information is not used in prediction. The mode for predicting the luminance information may be different from the mode for predicting the chrominance information and the intra prediction mode information or predicted luminance signal information used for predicting the luminance information may be utilized to predict the chrominance information.

When intraprediction is performed, when the size of the prediction unit is the same as the size of the conversion unit, intra prediction is performed on the prediction unit based on pixels existing on the left side of the prediction unit, pixels existing on the upper left side, Can be performed. However, when intra prediction is performed, when the size of the prediction unit differs from the size of the conversion unit, intraprediction can be performed using the reference pixel based on the conversion unit. It is also possible to use intra prediction using N x N divisions for only the minimum coding units.

The intra prediction method can generate a prediction block after applying an AIS (Adaptive Intra Smoothing) filter to the reference pixel according to the prediction mode. The type of the AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. In the case where the prediction mode of the current prediction unit is predicted using the mode information predicted from the peripheral prediction unit, if the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the current prediction unit, The prediction mode information of the current block can be encoded by performing entropy encoding if the prediction mode of the current prediction unit is different from the prediction mode of the neighbor prediction unit.

In addition, a residual block including a prediction unit that has been predicted based on the prediction unit generated by the prediction units 120 and 125 and a residual value that is a difference value from the original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130. [

The transform unit 130 transforms the residual block including the residual information of the prediction unit generated through the original block and the predictors 120 and 125 into a DCT (Discrete Cosine Transform), a DST (Discrete Sine Transform), a KLT You can convert using the same conversion method. The decision to apply the DCT, DST, or KLT to transform the residual block may be based on the intra prediction mode information of the prediction unit used to generate the residual block.

The quantization unit 135 may quantize the values converted into the frequency domain by the conversion unit 130. [ The quantization factor may vary depending on the block or the importance of the image. The values calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the reorder unit 160.

The reordering unit 160 can reorder the coefficient values with respect to the quantized residual values.

The reordering unit 160 may change the two-dimensional block type coefficient to a one-dimensional vector form through a coefficient scanning method. For example, the rearranging unit 160 may scan a DC coefficient to a coefficient in a high frequency region using a Zig-Zag scan method, and change the DC coefficient to a one-dimensional vector form. Instead of the jig-jag scan, a vertical scan may be used to scan two-dimensional block type coefficients in a column direction, and a horizontal scan to scan a two-dimensional block type coefficient in a row direction depending on the size of the conversion unit and the intra prediction mode. That is, it is possible to determine whether any scanning method among the jig-jag scan, the vertical direction scan and the horizontal direction scan is used according to the size of the conversion unit and the intra prediction mode.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160. For entropy encoding, various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used.

The entropy encoding unit 165 receives the residual value count information of the encoding unit, the block type information, the prediction mode information, the division unit information, the prediction unit information and the transmission unit information, and the motion information of the motion unit from the reordering unit 160 and the prediction units 120 and 125 Vector information, reference frame information, interpolation information of a block, filtering information, and the like.

The entropy encoding unit 165 can entropy-encode the coefficient value of the encoding unit input by the reordering unit 160. [

The inverse quantization unit 140 and the inverse transformation unit 145 inverse quantize the quantized values in the quantization unit 135 and inversely transform the converted values in the conversion unit 130. [ The residual value generated by the inverse quantization unit 140 and the inverse transform unit 145 is combined with the prediction unit predicted through the motion estimation unit, the motion compensation unit and the intra prediction unit included in the prediction units 120 and 125, A block (Reconstructed Block) can be generated.

The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by the boundary between the blocks in the reconstructed picture. It may be determined whether to apply a deblocking filter to the current block based on pixels included in a few columns or rows included in the block to determine whether to perform deblocking. When a deblocking filter is applied to a block, a strong filter or a weak filter may be applied according to the deblocking filtering strength required. In applying the deblocking filter, horizontal filtering and vertical filtering may be performed concurrently in performing vertical filtering and horizontal filtering.

The offset correction unit may correct the offset of the deblocked image with respect to the original image in units of pixels. In order to perform offset correction for a specific picture, pixels included in an image are divided into a predetermined number of areas, and then an area to be offset is determined and an offset is applied to the area. Alternatively, Can be used.

Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered reconstructed image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group may be determined and different filtering may be performed for each group. The information related to whether to apply the ALF may be transmitted for each coding unit (CU), and the shape and the filter coefficient of the ALF filter to be applied may be changed according to each block. Also, an ALF filter of the same type (fixed form) may be applied irrespective of the characteristics of the application target block.

The memory 155 may store the reconstructed block or picture calculated through the filter unit 150 and the reconstructed block or picture stored therein may be provided to the predictor 120 or 125 when the inter prediction is performed.

2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.

2, the image decoder 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, 240, and a memory 245 may be included.

When an image bitstream is input in the image encoder, the input bitstream may be decoded in a procedure opposite to that of the image encoder.

The entropy decoding unit 210 can perform entropy decoding in a procedure opposite to that in which entropy encoding is performed in the entropy encoding unit of the image encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied in accordance with the method performed by the image encoder.

The entropy decoding unit 210 may decode information related to intra prediction and inter prediction performed in the encoder.

The reordering unit 215 can perform reordering based on a method in which the entropy decoding unit 210 rearranges the entropy-decoded bitstreams in the encoding unit. The coefficients represented by the one-dimensional vector form can be rearranged by restoring the coefficients of the two-dimensional block form again. The reordering unit 215 can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing a reverse scanning based on the scanning order performed by the encoding unit.

The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values of the re-arranged blocks.

The inverse transform unit 225 may perform an inverse DCT, an inverse DST, and an inverse KLT on the DCT, DST, and KLT transformations performed by the transform unit on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit 225 of the image decoder, a transform technique (e.g., DCT, DST, KLT) may be selectively performed according to a plurality of information such as a prediction method, a size of a current block, and a prediction direction.

The prediction units 230 and 235 can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit 210 and the previously decoded block or picture information provided in the memory 245. [

As described above, when intra prediction is performed in the same manner as in the image encoder, when the size of the prediction unit is the same as the size of the conversion unit, pixels existing on the left side of the prediction unit, pixels existing on the upper left side, However, when the size of the prediction unit differs from the size of the prediction unit in intra prediction, intraprediction is performed using a reference pixel based on the conversion unit . It is also possible to use intra prediction using N x N divisions for only the minimum coding unit.

The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, and identifies prediction units in the current coding unit. It is possible to determine whether the unit performs inter prediction or intra prediction. The inter prediction unit 230 predicts the current prediction based on the information included in at least one of the previous picture of the current picture or the following picture including the current prediction unit by using information necessary for inter prediction of the current prediction unit provided by the image encoder, Unit can be performed. Alternatively, the inter prediction may be performed on the basis of the information of the partial region previously reconstructed in the current picture including the current prediction unit.

In order to perform inter prediction, a motion prediction method of a prediction unit included in a corresponding encoding unit on the basis of an encoding unit includes a skip mode, a merge mode, an AMVP mode, and an intra block copy mode It is possible to judge whether or not it is any method.

The intra prediction unit 235 can generate a prediction block based on the pixel information in the current picture. If the prediction unit is a prediction unit that performs intra prediction, the intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the image encoder. The intraprediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolator, and a DC filter. The AIS filter performs filtering on the reference pixels of the current block and can determine whether to apply the filter according to the prediction mode of the current prediction unit. The AIS filtering can be performed on the reference pixel of the current block using the prediction mode of the prediction unit provided in the image encoder and the AIS filter information. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

The reference pixel interpolator may interpolate the reference pixels to generate reference pixels in units of pixels less than or equal to an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values obtained by interpolating reference pixels. The reference pixel may not be interpolated in the prediction mode in which the prediction mode of the current prediction unit generates the prediction block without interpolating the reference pixel. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

The restored block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

When information on whether a deblocking filter is applied to a corresponding block or picture from the image encoder or a deblocking filter is applied, information on whether a strong filter or a weak filter is applied can be provided. In the deblocking filter of the video decoder, the deblocking filter related information provided by the video encoder is provided, and the video decoder can perform deblocking filtering for the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image and the offset value information during encoding.

The ALF can be applied to an encoding unit on the basis of ALF application information and ALF coefficient information provided from an encoder. Such ALF information may be provided in a specific parameter set.

The memory 245 may store the reconstructed picture or block to be used as a reference picture or a reference block, and may also provide the reconstructed picture to the output unit.

As described above, in the embodiments of the present disclosure, a coding unit (coding unit) is used as a coding unit for convenience of explanation, but it may be a unit for performing not only coding but also decoding.

The definitions of the terms used in the present disclosure are as follows.

Unit: A unit refers to a unit of image coding and decoding. An image coding or decoding unit refers to a unit for dividing one image into subdivided units and encoding or decoding the divided units. It can be called a block, a macroblock, a coding unit, a prediction unit (Prediction Unit), a conversion unit (Transform Unit), or the like. One unit may be further subdivided into smaller units.

Block: An MxN array of samples. M and N mean positive integer values, and the meaning of the block described in this disclosure refers to a coding unit.

In the image encoder, a rate-distortion optimization method may be used to provide a high coding efficiency using a combination of a coding unit size, a prediction mode, a prediction unit size, motion information, a conversion unit size, and the like.

The rate-distortion optimization scheme calculates a rate-distortion cost to select an optimal combination from among the combinations, and the rate-distortion cost can be calculated using Equation 1 below. In general, the combination in which the rate-distortion cost is minimized in the rate-distortion optimization method can be selected as the optimal combination.

In Equation (1), D representing the distortion means a mean square error of the difference values between the original transform coefficients and the restored transform coefficients in the transform block, It represents the bit rate using information. And [lambda] denotes a Lagrangian multiplier. In this case, R includes not only encoding parameter information such as a prediction mode, motion information, and coded block flag but also bits generated when the transform coefficients are encoded.

In order to calculate the correct D and R, the image encoder performs inter picture / intra picture prediction, conversion, quantization, entropy coding, inverse quantization, and inverse transformation, and this process greatly increases the complexity in the encoder.

The image compression of the present disclosure includes an inter prediction technique for predicting a pixel value included in a current picture from temporally preceding or following reference pictures, a prediction method using a pixel value included in a current picture using pixel information in the current picture , An entropy encoding technique for assigning a short code to a symbol having a high appearance frequency and allocating a long code to a symbol having a low appearance frequency can be applied.

The image compression of the present disclosure can be performed on a macroblock (16x16) basis. Or a block structure capable of achieving optimal compression efficiency for each coding unit (CU), prediction unit (PU), and transform unit (TU) in order to increase the optimum compression efficiency for high resolution video.

The image compression of the present disclosure can be applied to various data level parallel processing techniques (Data level parallelism). The data-level parallel processing technique refers to a method of dividing data to be processed into a plurality of unit data and then allocating the divided data to different cores or threads to perform the same operations in parallel . For example, in the image compression according to the present disclosure, an image can be divided into slice units, or can be divided into tiles and processed in parallel. Or parallel processing according to Wavefront parallel processing (WPP) technology. The WPP technology can refer to a method of parallel processing of encoding units in which there is no dependency between them.

The video compression of the present disclosure can minimize coding loss in various data-level parallel processing coding environments. For example, according to the data-level parallel processing, only encoding information of a neighboring block located in a slice or a tile area belonging to the current block can be used, resulting in a coding loss. In addition, in the case of WPP, it is possible to perform parallel processing using some peripheral information encoded before the current Wavefront coding, but it requires not only sophistication in implementation such as synchronization between threads and availability check of frequent peripheral block information, There is a problem that the parallel processing effect is relatively lowered. According to one embodiment of the present disclosure, the above-described problems caused by parallel processing of data levels can be solved.

More specifically, when one picture is divided into a plurality of tiles and processed in parallel, coding efficiency can be improved by using coding information between coding units in the same tile. However, in the case of a coding unit located at a tile boundary (upper, left), a peripheral coding unit (peripheral block) that can be referred to can be defined. Since the coding units at the boundary of the tile can not utilize more accurate surrounding encoding information (e.g., motion information), the mode determination according to the rate-distortion optimization is often determined as intra-picture prediction rather than inter-picture prediction. If the slice or the tile is not applied, the inter picture prediction mode can be determined with a relatively low bit amount. However, since the intra picture prediction mode is determined by applying the actual slice or tile, The encoding loss is generated.

In particular, when the image region belonging to the tile has a very large motion characteristic, most of the coding units belonging to the tile boundary are determined to be the in-picture mode, and the coding unit in the tile and the surrounding motion information can not be utilized The encoding loss becomes larger. In order to reduce the coding loss, it is necessary to set the coding units located at the tile boundary to be coded in the inter-picture prediction mode by performing inter-picture prediction through a wider search area, but the coding complexity increases according to the relatively large search area can do

According to one embodiment of the present disclosure, in the tile-based parallel processing coding environment, an adaptive search area can be set such that coding units located at the tile boundary can be coded by inter-picture prediction.

FIG. 3 is a conceptual illustration of a parallel processing technique based on slices, tiles, and WPPs among various data-level parallel processing techniques.

As shown in FIG. 3, the parallel processing in units of slices determines a plurality of consecutive units of encoding in a scan order in units of one slice, and each slice can be processed in parallel by separate cores.

In the case of a tile unlike a slice, the image may be divided into a plurality of tiles using one or more horizontal and / or vertical boundaries, and each tile may be processed in parallel by a separate core. The tile may consist of an integer number of encoding units (e.g., maximum size encoding units), and may have a rectangular (rectangular or square) shape. The size of the tile may be variably determined, for example, the minimum tile size may be preset or signaled. For example, the minimum tile size can be set to 256x64 in width x length. However, the minimum tile size is not limited to this, and may be set to various sizes. The maximum number of tiles in which one picture is divided according to the screen resolution can be preset or signaled. For example, the maximum number of tiles can be set to 25 for a full HD resolution image. However, the maximum number of tiles is not limited to this, and may be set to various numbers.

The WPP-based parallel processing technique can parallelize the encoding units that are not dependent on each other by separate cores.

The coding units in the same slice or the same tile can not refer to the coding information of the coding unit belonging to another slice or another tile being processed in parallel. Therefore, as described above, coding loss due to parallel processing can occur. In particular, when there are many regions with large motion in the tile, not only the coding units located in the tile boundary region are more likely to be determined as the intra-picture prediction mode, but also the coding units even located within the tile become peripheral coding units The coding unit located in the inner prediction mode is determined to be the intra prediction mode, the surrounding motion information can not be utilized, and thus the coding loss can be propagated throughout the tile.

4 is a diagram illustrating a case where a picture including four tiles is coded in a single core and a case in which a plurality of cores are coded in parallel and a coding unit in a picture is coded Fig. When each tile is processed in parallel using a plurality of cores, the encoding of the basic block can proceed according to the scan order indicated in each tile. The basic block may be a block serving as a unit of coding and may be a coding unit (CU) obtained by dividing the coding unit from a maximum coding unit (LCU) or a maximum coding unit. The maximum coding unit may be divided into a plurality of coding units using a tree structure, and the maximum coding unit may be a coding tree unit (CTU).

In Fig. 4, when a plurality of cores are used for parallel processing of respective tiles, the coding information of coding units belonging to different tiles can not be used. For example, all the encoding units belonging to tile 3 can not use the encoding information of the encoding units belonging to tile 0 to tile 2. Thus, the coding units located at the boundary (top and / or left) of the tile 3 may be defined with respect to neighboring blocks to be referenced.

5 is a diagram for explaining encoding loss according to tile encoding. FIG. 5 shows a 4K image having a large motion divided into 30 tiles, and a solid line in the image indicates a boundary of a tile. The image 502 in the left part of FIG. 5 shows the encoding result when the image 501 is set as the reference image and the motion search area (search range) is set as 64. FIG. The image 504 in the right part of FIG. 5 shows the result of encoding when the image 503 is set as the reference image and the motion search area is set as 256. In FIG. 5, white portions inside the images 501, 502, 503, and 504 are portions where intra prediction is performed, and gray portions and black portions indicate portions where inter prediction is performed. The inter-picture prediction is expressed in gray and black in the case where the picture is coded in the SKIP mode or the merge mode even within the inter-picture prediction, and the area in which the motion prediction is actually performed is expressed differently.

As can be seen from FIG. 5, when the search area is 64, most of the area is determined to be intra picture prediction (white), so that the coding loss is larger. This makes it possible to effectively utilize the motion information of the surrounding coded block in the tile boundary area And the search area is set to be relatively small as 64, which is caused by the fact that the motion can not efficiently cope with a large image.

On the other hand, when the search area is set to a relatively large value of 256, it can be seen that the inter-view prediction is performed in a relatively large area.

6 is a flowchart illustrating a method of performing a motion search based on an adaptively set search area according to an embodiment of the present disclosure;

FIG. 6 can be applied to a case in which a search region is set adaptively when coding a coding unit of a tile boundary region in a tile-based parallel processing coding environment, and a motion search is performed based on a set search region. However, the present invention is not limited to the case of tile-based parallel processing, and can be applied to any case where an image is divided into a plurality of regions for parallel processing.

First, in step S601, a motion complexity (MC) for a current tile area to be coded can be determined. The motion complexity may be determined by acquisition of an associated parameter. The motion complexity of the tile area to be coded at present may be a motion characteristic of an image area belonging to the current tile area. The motion characteristics may mean, for example, the size of the motion, and the motion complexity is determined based on the magnitude of the motion, and the size of the search region can be determined adaptively based on the determined motion complexity.

The motion complexity of the tile to be coded at present can be determined by various methods, for example, through the following two measurements.

(1) Zero motion based TD (tile difference)

In order to minimize the complexity, the sum of the absolute values of the zero motion-based tile difference between the current tile and the tile area belonging to the same position of the reference picture is measured and used as a parameter for determining the motion complexity of the current tile to be coded have. For example, let T _curr (i, j) be the pixel value of the pixel position (i, j) belonging to the current tile and T _ref (i, j) be the pixel value of the pixel position (i, j) of the tile region belonging to the same position of the reference picture, , j), the sum of the absolute values of the zero motion based TD (Tile Difference) can be measured, for example, by the following equation (2).

(2) Norm of the motion information of the tile in the same area of the picture referred to by the current tile

In the case of bidirectional prediction, motion information of a reference picture may be classified according to List 0 and List 1. At this time, a motion vector (MV, motion vector) of a tile in the same area of a reference picture of the current tile is obtained according to List 0 and List 1, and a norm for each motion vector can be calculated. For the calculation of Norm for each motion vector, for example, Equation (3) below may be used.

은 x 가 실수인 경우, x에 가까운 가장 큰 정수값을 의미한다.In Equation (3)

Means the largest integer value close to x if x is a real number.

After calculating the norm for each motion vector, the mean (AvgNorm) and maximum norm (MaxNorm) of the motion vectors of List 0 and List 1 can be calculated. For example, the following equation (4) can be used to calculate the norm of the norm of the motion vector and the maximum Norm.

In Equation (4), N _L0 and N _L1 represent the number of motion vectors for List 0 and List 1, respectively, acquired in the same tile region of the reference picture.

The average and / or the maximum norm of the norm of the motion vector calculated as described above can be used as a parameter for determining the motion complexity of the current tile to be coded, and the motion complexity of the tile to be coded can be determined based on this.

Specifically, the motion complexity can be determined by comparing the AvgNorm for each TD and / or each List calculated as described above with a predetermined or signaled threshold value. For example, if the computed TD and / or AvgNorm is greater than the threshold, it can be determined that the motion complexity of the tile is large. Or a plurality of threshold values and comparing the calculated TD and / or AvgNorm with the calculated TD and / or AvgNorm so that the calculated TD and / or AvgNorm is divided into a plurality of levels separated by a plurality of thresholds And determine the motion complexity of the tile as one of a plurality of levels. The threshold value may be manually tuned based on various images on the off-line, or dynamically set according to statistical characteristics of previously encoded images during actual encoding. Also, the above-described threshold value may be set to a threshold value for TD and a threshold value for AvgNorm, respectively, and a plurality of threshold values may be set for one or both of them. For example, as shown in Equation (5) below, only when TD and AvgNorm are both greater than the respective thresholds, it is determined that the motion complexity of the corresponding tile is large, and the search region can be adaptively set.

In Equation (5), Th _TD denotes a threshold value for TD, and Th _Norm denotes a threshold value for AvgNorm. According to an embodiment of the present disclosure, only when the condition of Equation (5) is satisfied, it is determined that the motion complexity of the tile is large, and thus the search region can be adaptively set. For example, when AvgNorm in Equation (5) is related to List 0, when TD and AvgNorm (L0) are larger than predetermined threshold values, the search area is adaptively Can be set. At this time, the adaptive setting of the search area may mean enlarging the search area. The enlargement of the search area can be performed according to a predetermined scaling, which will be described later.

If the motion complexity of the current tile to be coded is determined in step S601, the motion search area can be adaptively set based on the motion complexity (step S602). The adaptively set motion search area may be applied only during inter-picture coding of the coding unit located at the boundary (e.g., left and / or above) of the current tile to be coded, Or may be applied to inter-picture coding of some coding units belonging to the current tile to be coded.

When it is determined to enlarge the search area based on the motion complexity of the current tile to be coded, a predetermined scaling is applied to the search area (SearchRange) basically set for the encoding of the current image to obtain an enlarged search area (New Search Range) Can be set. For example, an enlarged search area can be set using the following equation (6).

In Equation (6), since MaxNorm and AvgNorm are calculated for each of List 0 and List 1, the magnification of the search area and the magnitude of the enlarged search area can be set differently for each of List 0 and List 1. For example, in the inter-picture prediction of the coding unit to be currently coded, the search area is enlarged only for List 0, and the search area for List 1 may be the same as the existing one, and vice versa. Alternatively, the search area may be enlarged for both List 0 and List 1.

When the search area is enlarged as described above, motion search for the currently coded coding unit can be performed based on the enlarged search area (step S603). The motion search can be composed of integer pixel motion search and subpixel motion search.

The integer pixel-based motion search can be performed to search for the position of an optimal reference block having the highest degree of correlation with the coding unit currently coded in the search area of the reference picture, in units of integer pixels. The correlation between the currently encoded coding unit and the reference block can be calculated by various methods, for example, a sum of absolute difference (SAD) can be used. Various algorithms can be applied to motion search in integer pixel units. For example, an algorithm composed of three steps of a diamond search step, a raster search step, and a refinement step may be applied for motion detection on an integer pixel basis.

FIG. 7 is a diagram for explaining a diamond search and a raster search in the integer pixel-based motion search method.

In the diamond search of the first step shown in FIG. 7, a diamond search is performed based on an initially set predictive motion vector (PMV) to find a search point in an integer pixel unit satisfying a predefined early decision condition . In the case where a search point in the unit of an integer pixel satisfying the predefined early determination condition is not found in the diamond search of the first step, the raster search of the second step shown in FIG. 7 can be performed. In the raster search of the second stage, a search point in the integer pixel unit can be found with a constant pixel interval according to the raster scan order within the set search area. The constant pixel spacing can be set, for example, at 5 pixel intervals. In the refinement step of the third step, an optimum search point can be found by performing a diamond search based on the search point determined in the first-step diamond search or the second-step raster search.

Even when the motion search area is enlarged according to the embodiment of the present disclosure, the above-described motion search method can be applied. Alternatively, the above-described motion search method may be modified and applied in accordance with the enlargement of the search area. For example, when the above-described motion search method is applied as it is according to the enlargement of the search area, the complexity of the encoder may increase, and the motion search method described above may be modified and applied.

More specifically, when the search area is enlarged, the intervals of the integer pixels that are search targets of the raster search in the above-described motion search method can be set differently from the general case. For example, if a raster search is performed at 5 pixel intervals in a general case where the search area is not enlarged, a 5 pixel raster search interval (RSI) can be adjusted when the search area is enlarged. Adjustment of the raster search interval may be performed based on the extent to which the search area is enlarged. For example, the following Equation (7) may be applied to calculate the adjusted raster search interval (New RSI) for the enlarged search area.

In Equation (7), scaling may mean the degree to which the search area is enlarged. For scaling, for example, the scaling calculated in Equation (6) above may be applied.

As described above, if the raster search interval for the enlarged search area is adjusted to decrease the calculation frequency for raster search in the enlarged search area, it is possible to prevent the drawback that the complexity of the encoder increases .

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (1)

A motion search method for a coding unit included in an encoding target area,
Determining a motion complexity of the area to be coded;
Setting a motion search area based on the determined motion complexity; And
And performing a motion search for the encoding unit based on the set motion search area.

Images