KR102425722B1 - Method and apparatus for encoding/decoding a video signal - Google Patents

Abstract

A video signal decoding method according to the present invention comprises the steps of determining whether there is a change in brightness between a current image including a current block and a reference image of the current image; if it is determined to be, determining a weight prediction parameter candidate for the current block, determining a weight prediction parameter for the current block based on index information specifying any one of the weight prediction parameter candidates, and the weight The method may include performing prediction on the current block based on the prediction parameter.

Description

Video signal encoding/decoding method and apparatus

The present invention relates to a method and apparatus for encoding/decoding a video signal.

Recently, the demand for multimedia data such as moving pictures is rapidly increasing on the Internet. However, it is difficult to keep up with the rapidly increasing amount of multimedia data at the rate of development of the bandwidth of the channel. Accordingly, in February 2014, ITU-T's Video Coding Expert Group (VCEG) and ISO/IEC's MPEG (Moving Picture Expert Group) adopted HEVC (High Efficiency Video Coding) version 1, a video compression standard, in February 2014. enacted.

HEVC defines techniques such as intra prediction, inter prediction, transform, quantization, entropy encoding, and in-loop filter. Among them, inter prediction refers to performing prediction using previously reconstructed images and motion information such as a motion vector, a reference picture index, and an inter prediction indicator. it means.

In the prediction between screens, the higher the correlation between images, the higher the prediction efficiency can be obtained. However, if there is a change in brightness between images, such as fade-in or fade-out, and the correlation between images is lowered, there is a concern that the prediction result between the screens may be inaccurate. In order to solve such a problem, in the present invention, weight prediction is proposed. Here, the weighted prediction may mean that when there is a brightness change between images, the weight is estimated by the degree of the brightness change, and the estimated weight is applied to the inter prediction.

A main object of the present invention is to improve inter prediction efficiency by performing inter prediction using weights in encoding/decoding an image.

The present invention provides an apparatus and method for effectively performing inter-screen prediction even when a plurality of light sources exist in an image or a change in brightness exists only in a local area by selectively using weights in encoding/decoding an image Its main purpose is to provide

A video signal decoding method and apparatus according to the present invention determines whether there is a change in brightness between a current image including a current block and a reference image of the current image, and there is a change in brightness between the current image and the reference image If it is determined to be, a weight prediction parameter candidate for the current block is determined, and a weight prediction parameter for the current block is determined based on index information specifying any one of the weight prediction parameter candidates, and the weight prediction parameter is determined. Based on , prediction may be performed on the current block.

In the method and apparatus for decoding an image signal according to the present invention, the weighted prediction parameter candidate may include a first weighted prediction parameter for the reference image.

In the video signal decoding method and apparatus according to the present invention, when at least one region from which a second weight prediction parameter can be derived is included in the current image, the weight prediction parameter candidates include at least one second weight It may further include a prediction parameter.

In the method and apparatus for decoding an image signal according to the present invention, the first weighted prediction parameter may be derived based on a prediction value of the first weighted prediction parameter and a residual value of the first weighted prediction parameter.

In the method and apparatus for decoding an image signal according to the present invention, the predicted value for the first weighted prediction parameter may be determined according to the precision of the current block.

In the method and apparatus for decoding an image signal according to the present invention, the maximum number of weight prediction parameter candidates may be adaptively determined according to the size of the current block.

In the method and apparatus for decoding an image signal according to the present invention, the weight prediction parameter candidate may include an initial weight prediction parameter having a preset weight value.

A video signal encoding method and apparatus according to the present invention determines whether there is a change in brightness between a current image including a current block and a reference image of the current image, and there is a change in brightness between the current image and the reference image If it is determined that a weight prediction parameter candidate for the current block is determined, a weight prediction parameter for the current block is determined from among the weight prediction parameter candidates, and index information specifying the determined weight prediction parameter is encoded, and Prediction may be performed on the current block based on the weighted prediction parameter.

In the video signal encoding method and apparatus according to the present invention, the weighted prediction parameter candidate may include a first weighted prediction parameter for the reference image.

In the video signal encoding method and apparatus according to the present invention, when at least one region from which a second weight prediction parameter can be derived is included in the current image, the weight prediction parameter candidates include at least one second weight It may further include a prediction parameter.

The video signal encoding method and apparatus according to the present invention may encode a difference value indicating a difference between the first weighted prediction parameter and a predicted value for the first weighted prediction parameter.

In the video signal encoding method and apparatus according to the present invention, the prediction value for the first weighted prediction parameter may be determined according to the precision of the current block.

In the video signal encoding method and apparatus according to the present invention, the maximum number of weight prediction parameter candidates may be adaptively determined according to the size of the current block.

In the video signal encoding method and apparatus according to the present invention, the weight prediction parameter candidate may include an initial weight prediction parameter having a preset weight value.

The present invention can improve inter prediction efficiency by performing inter prediction using weights when encoding/decoding an image.

In the present invention, by selectively using weights in encoding/decoding an image, even when a plurality of light sources exist in an image or a change in brightness exists only in a local area, inter prediction can be effectively performed.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present invention.
3 is a block diagram schematically illustrating a motion estimation method according to an embodiment of the present invention.
4 is an example showing the positions of neighboring blocks from which motion information to be applied to a block to be currently encoded according to an embodiment of the present invention.
5 is a diagram illustrating a change in brightness between a current image including a current block and a reference image.
6 is a flowchart illustrating a method of estimating a weight prediction parameter in an image encoding apparatus.
7 is a diagram illustrating an example of performing Rate Distortion Optimization (RDO) on a prediction block using a weight parameter.
8 is a flowchart illustrating a method of encoding information related to a weight prediction parameter for a current block.
9 is a diagram illustrating an example of decoding a weight prediction parameter in a decoding apparatus.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , the image encoding apparatus 100 includes an image segmentation unit 110 , prediction units 120 and 125 , a transform unit 130 , a quantization unit 135 , a rearrangement unit 160 , and an entropy encoding unit ( 165 ), an inverse quantization unit 140 , an inverse transform unit 145 , a filter unit 150 , and a memory 155 .

Each of the constituent units shown in FIG. 1 is independently illustrated to represent different characteristic functions in the image encoding apparatus, and does not mean that each constituent unit is composed of separate hardware or one software constituent unit. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention except for components used for improving performance, and only having a structure including essential components excluding optional components used for improving performance Also included in the scope of the present invention.

The image dividing unit 110 may divide the input image into at least one block. In this case, the block may mean a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The division may be performed based on at least one of a quadtree and a binary tree. A quad tree is a method in which the upper block is divided into lower blocks whose width and height are half that of the upper block. Binary tree is a method in which the upper block is divided into lower blocks whose either width or height is half that of the upper block. Through the aforementioned quad-tree or binary tree-based partitioning, a block may have a non-square shape as well as a square shape.

Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit for performing encoding or may be used as a meaning for a unit for performing decoding.

The prediction units 120 and 125 may include an inter prediction unit 120 performing inter prediction and an intra prediction unit 125 performing intra prediction. Whether to use inter prediction or to perform intra prediction for a prediction unit may be determined, and specific information (eg, intra prediction mode, motion vector, reference image, etc.) according to each prediction method may be determined. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit.

A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130 . Also, prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoder 165 together with a residual value and transmitted to a decoder. When a specific encoding mode is used, the original block may be encoded and transmitted to the decoder without generating the prediction block through the predictors 120 and 125 .

The inter prediction unit 120 may predict a prediction unit based on information on at least one of a previous image or a subsequent image of the current image, and in some cases, based on information of a partial region in the current image that has been encoded. A prediction unit may be predicted. The inter prediction unit 120 may include a reference image interpolator, a motion information generator, and a motion compensator.

The reference image interpolator may receive reference image information from the memory 155 and generate pixel information of integer pixels or less from the reference image. In the case of luminance pixels, a DCT-based 8-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/4 pixels may be used. In the case of the color difference signal, a DCT-based 4-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/8 pixels may be used.

The motion information generator may generate motion information based on the reference image interpolated by the reference image interpolator. Here, the motion information means a motion vector, a reference image index, a prediction direction, and the like. As a method for estimating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. Also, the motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixels. In inter prediction, the current prediction unit may be predicted by using a different method of generating motion information. As a method of generating motion information, various methods such as a merge method using a motion vector of a neighboring block and a motion estimation method (eg, Adaptive Motion Vector Prediction (AMVP)) may be used.

As an example, FIG. 3 is a diagram illustrating an example of generating motion information through motion estimation. Motion estimation is to determine a motion vector, a reference picture index, and an inter prediction direction of the current block according to the determination when a reference block identical to or similar to a prediction block in a reference picture that has already been encoded and decoded is determined.

When the AMVP method is used, the encoding apparatus predicts a motion vector estimated in the current block to generate a motion vector prediction (MVP), and a difference value (MVD) between the motion vector and the generated prediction motion vector: Motion Vector Difference) can be encoded.

A method of using a motion vector of a neighboring block is to apply motion information of a neighboring block adjacent to the current block to the current block. In this case, the neighboring block may include a spatial neighboring block adjacent to the current block and a temporal neighboring block existing at the same position as the current block included in the reference image. As an example, FIG. 4 illustrates a neighboring block of the current block. The encoding apparatus may determine the motion information of the current block by applying motion information of neighboring blocks (spatial neighboring blocks: A to E , temporal neighboring blocks: Col) of the current block shown in FIG. 4 to the current block. Here, Col means a block at the same or similar position as the current block existing in the reference image.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current image. If the neighboring block of the current prediction unit is a block on which inter prediction is performed, and the reference pixel is a pixel restored by performing inter prediction, the reference pixel included in the block on which inter prediction is performed is used for the neighboring intra prediction. It can be used by replacing it with reference pixel information of the executed block. That is, when the reference pixel is not available, the unavailable reference pixel information may be replaced with at least one reference pixel among the available reference pixels.

In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which directional information is not used when prediction is performed. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and in-screen prediction mode information or predicted luminance signal information used to predict luminance information may be utilized to predict chrominance information. .

In the intra prediction method, a prediction block may be generated after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of the AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When the intra prediction mode of the current prediction unit is predicted by using the intra prediction mode information predicted from the neighboring prediction unit, if the intra prediction mode of the current prediction unit and the neighboring prediction unit are the same, the current prediction unit using predetermined flag information Information that the prediction mode of the prediction unit and the neighboring prediction unit is the same may be transmitted, and if the prediction modes of the current prediction unit and the neighboring prediction unit are different, entropy encoding may be performed to encode the prediction mode information of the current block.

In addition, a residual block including residual information that is a difference value from the original block of the prediction unit and the prediction unit in which prediction is performed based on the prediction unit generated by the prediction units 120 and 125 may be generated. The generated residual block may be input to the transform unit 130 .

The transform unit 130 may transform the residual block including the residual data using a transform method such as DCT, DST, or Karhunen Loeve Transform (KLT). In this case, the transformation method may be determined based on the intra prediction mode of the prediction unit used to generate the residual block. For example, depending on the intra prediction mode, DCT may be used in a horizontal direction and DST may be used in a vertical direction.

The quantization unit 135 may quantize values transformed in the frequency domain by the transform unit 130 . The quantization coefficient may vary according to blocks or the importance of an image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160 .

The transform unit 130 and/or the quantizer 135 may be selectively included in the image encoding apparatus 100 . That is, the image encoding apparatus 100 may encode the residual block by performing at least one of transform and quantization on the residual data of the residual block, or skipping both transform and quantization. Even if either transform or quantization is not performed in the image encoding apparatus 100 or neither transform nor quantization is performed, a block entering the input of the entropy encoder 165 is generally referred to as a transform block.

The reordering unit 160 may rearrange the coefficient values on the quantized residual values.

The rearranging unit 160 may change the two-dimensional block form coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the reordering unit 160 may scan from DC coefficients to coefficients in a high frequency region using a predetermined scan type and change the scan type into a one-dimensional vector form.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160 . For entropy encoding, various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used.

The entropy encoding unit 165 receives the residual value coefficient information and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, and motion of the coding unit from the reordering unit 160 and the prediction units 120 and 125 . Various information such as vector information, reference image information, block interpolation information, and filtering information may be encoded. In the entropy encoding unit 165 , the coefficients of the transform block include a flag indicating whether 0 in units of partial blocks within the transform block, a flag indicating whether the absolute value of the coefficient is greater than 1, and whether the absolute value of the coefficient is greater than 2 A flag indicating , etc. may be encoded. The entropy encoding unit 165 encodes the sign of the coefficient only for non-zero coefficients. And, for a coefficient whose absolute value is greater than 2, the remaining value obtained by subtracting 2 from the absolute value is encoded.

The entropy encoder 165 may entropy-encode the coefficient values of the coding units input from the reordering unit 160 .

The inverse quantizer 140 and the inverse transform unit 145 inversely quantize the values quantized by the quantizer 135 and inversely transform the values transformed by the transform unit 130 . The residual value generated by the inverse quantizer 140 and the inverse transform unit 145 may be combined with a prediction block generated for each prediction unit through the prediction units 120 and 125 to generate a reconstructed block.

The filter unit 150 may include at least one of a deblocking filter, an offset correcting unit, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion caused by the boundary between blocks in the reconstructed image. In order to determine whether to perform the deblocking filter, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be concurrently processed when performing vertical filtering and horizontal filtering.

The offset correcting unit may correct an offset from the original image in units of pixels with respect to the image on which the deblocking has been performed. In order to perform offset correction on a specific image, the method of dividing pixels included in an image into arbitrary regions, determining the region to be offset and applying the offset to the region, or calculating the offset by considering the edge information of each pixel application method can be used.

Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the corresponding group is determined to perform differential filtering for each group. Information related to whether to apply ALF may be transmitted for each coding unit (CU) in the case of a luminance signal, and the shape and filter coefficients of the ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the application target block.

The memory 155 may store the reconstructed block or image calculated through the filter unit 150 , and the stored reconstructed block or image may be provided to the predictors 120 and 125 when inter prediction is performed.

2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present invention.

2, the image decoder 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, and a filter unit ( 240) and a memory 245 may be included.

When an image bitstream is input from an image encoder, the input bitstream may be decoded by a procedure opposite to that of the image encoder.

The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to that performed by the entropy encoding unit of the image encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied corresponding to the method performed by the image encoder. In the entropy decoding unit 210, the coefficients of the transform block include a flag indicating whether 0 in units of partial blocks within the transform block, a flag indicating whether the absolute value of the coefficient is greater than 1, and whether the absolute value of the coefficient is greater than 2 A flag indicating , etc. may be decoded. Then, the entropy decoding unit 210 decodes a sign of a non-zero coefficient. For coefficients having an absolute value greater than 2, the remaining values obtained by subtracting 2 may be decoded.

The entropy decoder 210 may decode information related to intra prediction and inter prediction performed by the encoder.

The reordering unit 215 may perform reordering based on a method of rearranging the entropy-decoded bitstream by the entropy decoding unit 210 by the encoder. Coefficients expressed in a one-dimensional vector form may be restored and rearranged in a two-dimensional block form. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform reordering by performing a reverse scanning method based on the scanning order performed by the corresponding encoder.

The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the reordered coefficient values of the blocks.

The inverse transform unit 225 may perform inverse transform on the inverse quantized transform coefficient using a predetermined transform method. In this case, the transformation method may be determined based on information on a prediction method (inter-picture/intra-picture prediction), a size/shape of a block, an intra prediction mode, and the like.

The prediction units 230 and 235 may generate a prediction block based on the prediction block generation related information provided from the entropy decoder 210 and the previously decoded block or image information provided from the memory 245 .

The prediction units 230 and 235 may include a prediction unit determiner, an inter prediction unit, and an intra prediction unit. The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and separates the prediction unit from the current coding unit. , it may be determined whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information required for inter prediction of the current prediction unit provided from the image encoder, based on information included in at least one of a previous image or a subsequent image of the current image including the current prediction unit. Inter-screen prediction for the current prediction unit may be performed. Alternatively, inter prediction may be performed based on information on a pre-restored partial region in the current image including the current prediction unit.

In order to perform inter prediction, based on a coding unit, it may be determined whether a method of generating motion information of a prediction unit included in a corresponding coding unit is generated by either a merge method or a motion estimation method.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current image. When the prediction unit is a prediction unit on which intra prediction is performed, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can be applied by determining whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on the reference pixel of the current block by using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. . When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter may generate the prediction block through filtering when the prediction mode of the current block is the DC mode.

The reconstructed block or image may be provided to the filter unit 240 . The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to a corresponding block or image and information on whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the image encoder. The deblocking filter of the image decoder may receive deblocking filter-related information provided from the image encoder, and the image decoder may perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding, offset value information, and the like.

ALF may be applied to a coding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoder. Such ALF information may be provided by being included in a specific parameter set.

The memory 245 may store the reconstructed image or block to be used as a reference image or reference block, and may also provide the reconstructed image to an output unit.

5 is a diagram illustrating a change in brightness between a current image including a current block and a reference image.

When inter prediction is performed on the current block, the greater the change in brightness between the current image and the reference image, the greater the change in brightness between the current block and the prediction block to be selected from the reference image. Therefore, as the error due to the inter prediction of the current block increases, it can be expected that the energy for the residual signal of the current block also increases. And, as the energy of the residual signal increases, it can be expected that the error due to quantization also increases. As a result, when there is a brightness change between the current image and the reference image, the error for the residual block will increase compared to when there is no brightness change.

Accordingly, the present invention intends to propose a method of generating a weighted prediction parameter by estimating the brightness change between images and performing inter prediction using the weighted prediction parameter. By using the weighted prediction parameter in inter prediction, it is possible to prevent the energy of the residual block from rapidly increasing and to increase the prediction efficiency.

The weight prediction parameter may be generated in consideration of changes in brightness of the current image and the reference image, such as fade in or fade out. However, when determining the weight prediction parameter in consideration of only the above factors, it is difficult to respond when the brightness of the current image and the reference image is changed by a plurality of light sources or when the brightness of only a local area within the current image is changed. Accordingly, in the present invention, a method for estimating a weight prediction parameter is also proposed in consideration of a case in which the brightness is changed by a plurality of light sources or a case in which the brightness of only a local area is changed.

Hereinafter, inter prediction using a weight prediction parameter will be described in detail with reference to the drawings.

6 is a flowchart illustrating a method of estimating a weight prediction parameter in an image encoding apparatus.

The weight prediction parameter may be generated based on a change in brightness between the current image and the reference image. For convenience of description, in an embodiment to be described later, a weighted prediction parameter generated based on a change in brightness between the current image and a reference image is referred to as a 'first weighted prediction parameter', and a partial region in the current image and a reference image A weighted prediction parameter generated based on a change in brightness between partial regions of the inner region will be referred to as a 'second weighted prediction parameter'. The prefix in front of the weight prediction parameters, such as 'first' and 'second', is merely imposed for technical convenience, and limits that the first weight prediction parameter and the second weight prediction parameter must be of different properties. it is not

The encoding apparatus may estimate a first weight prediction parameter for the reference image by using the current image and the reference image before encoding the image ( S601 ). As an example, the encoding apparatus may assign a first weight prediction parameter to each of a plurality of reference images included in the reference image list.

The first weight prediction parameter is a value generated based on a brightness variation between the reference image and the current image.

When the first weighted prediction parameter is used for inter prediction of the current block, the prediction block may be generated based on the reference block indicated by the motion vector and the first weighted parameter for the reference image including the reference block.

The first weighted prediction parameter may include at least one of a multiplication parameter multiplied by the prediction pixel or an addition parameter added to the prediction pixel. In this case, the multiplication parameter and the addition parameter may be derived based on regression analysis. As an example, Equation 1 below shows an example of a regression analysis model.

In Equation 1, Y is the data of the current image, X is the data of the reference image, w is the slope of the regression line, o is the intercept value of the regression line, and e is the prediction error of the regression line. In this case, Y is a pixel value of the current image, and the entire or partial region of the current image may be the range, and X is the pixel value of the reference image, and the entire or partial region of the reference image may be the range.

The first weight prediction parameter may be obtained by partial differentiation of Equation 1 by w and o, respectively. As an example, when Equation 1 is partially differentiated by w and o, respectively, w and o at which the square of the error e is minimized may be set as a multiplication parameter and an addition parameter, respectively.

The value of the first weight prediction parameter calculated based on Equation 1 may have a real value. The first weight prediction parameter may be set as a real value calculated based on Equation 1 or may be set as an integer value obtained by converting a real value calculated based on Equation 1 to an integer. As an example, the first weight prediction parameter may be derived as an integer value derived by multiplying a real value calculated based on Equation 1 by ^2N . The variable N used for integerizing the first weight prediction parameter may be encoded in block units, region units, or through higher headers.

When the first weight parameter is determined, the encoding apparatus may calculate a cost depending on whether the first weight parameter is applied ( S602 ). For example, the encoding apparatus applies the first weight parameter to the entire reference image to calculate a sum of absolute difference (SAD) between the current image and the reference image and the SAD between the reference image and the current image to which the first weight parameter is not applied. can If the SAD with the reference image to which the first weighting parameter is applied is smaller than the SAD with the reference image to which the first weighting parameter is not applied, it may be determined that there is a change in brightness between the current image and the reference image. Conversely, if the SAD with the reference image to which the first weight parameter is applied is greater than the SAD with the reference image to which the wp1 weight parameter is not applied, it may be determined that there is no brightness change ( S603 ).

In addition to the above-described SAD, cost calculation may be performed using sum of squared difference (SSD) or sum of absolute transformed difference (SATD).

When it is determined that there is no change in brightness between the current image and the reference image ( S603 ), it may be determined that the weight prediction parameter is not used in inter prediction for the current image. Accordingly, inter prediction may be used for the current block included in the current image without using a weight parameter.

On the other hand, when it is determined that there is a change in brightness between the current image and the reference image ( S603 ), a process of deriving the second weight parameter for each predetermined region of the current image may be performed. For example, the second weight parameter for a predetermined region in the current image is obtained by comparing pixel values of blocks included in the predetermined region with pixel values of blocks included in a region at the same location as the predetermined region in a reference image. can be

In order to calculate a second weight prediction parameter for a predetermined region in the current image, the current image and the reference image may be divided into a plurality of regions ( S604 ). The current image and the reference image may be divided into a plurality of blocks having the same size. The current image and the reference image may be divided in the same manner, and accordingly, the number of blocks and the positions of the blocks in the current image and the reference image may be set to be the same.

Thereafter, the encoding apparatus may determine whether at least one region in which a local brightness change occurs in the current image is included by using pixel values for each block in the current image and the reference image ( S605 ).

In step S605, at least one region may be configured as a set of blocks having the same or similar pixel value ratio. For example, if the average pixel value of the first divided block in the current image is 100 and the average pixel value of the first divided block in the reference image is 80, the first block in the current image and the first block in the reference image are 1.25 It can be seen that there is a difference in the brightness of the ship. The encoding apparatus may group a block having a brightness difference of 1.25 times with the reference block or a block having a brightness difference similar to 1.25 times with the first block. As such, by grouping blocks having the same or similar brightness difference, a region in which a local brightness change exists can be identified.

That is, the encoding apparatus may compare the pixel values of the blocks included in the current image with the pixel values included in the reference image, and then group blocks having the same or similar pixel value ratio and treat them as one region.

The encoding apparatus may generate a second weight prediction parameter for at least one region included in the current image (S606). When a plurality of regions are generated as a set of blocks having a similar pixel value ratio in the current image, a plurality of second weight prediction parameters may be generated.

The second weight prediction parameter for each region may be obtained using Equation (1). In this case, Y may be data in a predetermined area in the current image for which the second weight estimation parameter is to be estimated, and X may be data in the same location as the predetermined area in the reference image. For example, Y may be a pixel value within a predetermined region, and all or some regions may be a range, and X may be a pixel value within a same location region, and all or a partial region may be a range. The value of the second weight prediction parameter calculated based on Equation 1 may have a real value. The second weight prediction parameter may be set as a real value calculated based on Equation 1, or may be set as a value obtained by integerizing a real value calculated based on Equation 1. As an example, the second weight prediction parameter may be derived as an integer value derived by multiplying a real value calculated based on Equation 1 by 2 ^N. A variable N used for integerizing the second weight prediction parameter may be encoded in block units, region units, or through higher headers. In addition, it may have the same value as N used for integerizing the first weight prediction parameter, and it is also possible to use different Ns.

7 is a diagram illustrating an example of performing Rate Distortion Optimization (RDO) on a prediction block using a weight parameter.

6 , the encoding apparatus obtains the first weighted prediction parameter when there is a change in brightness between the current image and the reference image, and when there is a region in which a local change in brightness occurs in the current image, A second weight prediction parameter may be obtained ( S701 ). That is, the encoding apparatus obtains a weighted prediction parameter candidate that can be applied to the current block, such as a first weighted prediction parameter or a second weighted prediction parameter, according to whether there is a brightness change between the current image and a reference image or a local brightness change. can

When there is a change in brightness between the current block and the reference image, the weighted prediction parameter candidate for the current block may include the first weighted prediction parameter. In addition, when there is a region in which a local brightness change exists in the current image, the weight prediction parameter candidate for the current block may further include a second weight prediction parameter. If a plurality of second weighted prediction parameters exist for the current image, the weighted prediction parameter candidates may include a plurality of second weighted prediction parameters.

The encoding apparatus may apply the weight parameters set as weight prediction parameter candidates to the current block, calculate each cost (S702), and then determine an optimal weight prediction parameter for the current block based on the calculation result (S703) .

Determining the weighted prediction parameter with respect to the current block corresponds to selecting one exhibiting optimal inter prediction performance for the current block from among a plurality of weighted prediction parameter candidates. As an example, if a first weighted prediction parameter is derived with respect to a reference image and a second weighted prediction parameter is derived for a predetermined region in the current image, among the first weighted prediction parameter and the second weighted prediction parameter, the optimal value for the current block is obtained. A weighted prediction parameter can be selected.

To this end, the encoding apparatus may select an optimal weight prediction parameter by comparing results of inter prediction of each of the weight prediction parameter candidates. As an example, the encoding apparatus may determine an optimal weighted prediction parameter of the current block according to a result of performing inter prediction of each of the first weighted prediction parameter and the plurality of second weighted prediction parameters for the current block.

In addition, by comparing the result of performing inter prediction using the weighted prediction parameter candidate with the result of performing the inter prediction without using the weighted prediction parameter candidate, it is possible to determine whether inter prediction is performed using the weighted prediction parameter. You may decide whether to

In the above-described example, it has been described that the weighted prediction parameter includes the first weighted prediction parameter and, in some cases, may further include the second weighted prediction parameter. Contrary to the described example, the first weight prediction parameter may be used as a weight prediction parameter candidate only when the second weight prediction parameter cannot be used or the number of the second weight prediction parameters is less than or equal to a predetermined number.

The weight prediction parameter candidates may have a fixed number or a variable number. When the number of weight prediction parameter candidates is variable, the encoding apparatus may encode information indicating the number of weight prediction parameter candidates available for the current block through a bitstream (eg, an upper header). In this case, the number of candidates for the weight prediction parameter may be variably set according to the size or depth of the current block. Accordingly, the encoding apparatus may encode information about the number of weight prediction parameter candidates that can be used according to block size or depth information through a bitstream (eg, an upper header).

As another example, the encoding apparatus and the decoding apparatus may be defined to use the same number of weight prediction parameter candidates according to a preset condition. For example, the number of weight prediction parameter candidates may be adaptively determined according to the size, shape, or intra prediction mode of the current block. Assuming that the number of available weighted prediction parameters is 5, when the size of the current block is 8x8 or less, 5 weighted prediction parameter candidates are used, and when the size of the current block is 16x16, 4 weighted prediction parameter candidates are used. When the size of the current block is 32x32, three weighted prediction parameter candidates can be used, and when the size of the current block is 64x64, two weighted prediction parameters can be used.

If the second weighted prediction parameter is not obtained for the current image, the current block may be encoded using the first weighted prediction parameter. Also, if there is no change in brightness between the current image and the reference image, the current block may be encoded without using the weight prediction parameter or may be encoded using the initial weight prediction parameter.

The initial weight prediction parameter means that the multiplication parameter and the addition parameter are set as initial values. Here, the initial values of the multiplication parameter and the addition parameter may be 1 and 0, respectively. As another example, the initial value of the multiplication parameter may be set to 1<<N, and the initial value of the addition parameter may be set to zero.

In the above-described example, when there is no change in brightness between the current image and the reference image, it is exemplified that the initial weight prediction parameter is used. Even when there is a change in brightness between the current image and the reference image, it is also possible to encode the current block using the initial weighted prediction parameter. In this case, in order to select the initial weight prediction parameter, the initial weight prediction parameter may be added as a weight prediction parameter candidate. As the initial weight prediction parameter is added as a weight prediction parameter candidate, a new index may be assigned to the initial weight prediction parameter. In this case, the index assigned to the initial weight prediction parameter may be encoded through a bitstream or may have a preset value.

In addition, by performing RDO using weighted prediction parameter candidates including the first weighted prediction parameter, the second weighted prediction parameter, and the initial weighted prediction parameter, the current among the first weighted prediction parameter, the second weighted prediction parameter, and the initial weighted prediction parameter A weight prediction parameter suitable for a block can be specified.

Next, a method of encoding information related to a weight prediction parameter will be described.

8 is a flowchart illustrating a method of encoding information related to a weight prediction parameter for a current block.

Referring to FIG. 8 , the encoding apparatus may encode information on whether a change in brightness exists for each reference image of the current image ( S801 ). Whether or not to use the weighted prediction parameter is determined according to whether there is a brightness change, so the information may be used to indicate whether or not the weighted prediction parameter is present in the bitstream. In this case, the information on whether there is a change in brightness may be a 1-bit flag, but is not limited thereto. Also, information on whether there is a brightness change may be encoded in units of prediction blocks or may be encoded through a header higher than the prediction block. For example, the information may be encoded in units of sequences, pictures, slices, or tiles. The decoding apparatus may determine whether there is a change in brightness between the current image and the reference image based on information decoded from the bitstream.

When there is no change in brightness between the current image and the reference image, the encoding apparatus does not encode information related to the weight prediction parameter for the current block.

On the other hand, when there is a change in brightness between the current image and the reference image ( S802 ), the encoding apparatus provides information on whether the first weighted prediction parameter and the current image include a region to which at least one second weighted prediction parameter is assigned. can be encoded (S803, S804). When the current image includes a region to which the second weighted prediction parameter is assigned, information on the second weighted prediction parameter is additionally encoded, so that the information includes an additional weighted prediction parameter in addition to the first weighted predictive parameter for the current image. It can also be used to indicate whether or not Here, information on whether the current image includes at least one region for deriving the second weight prediction parameter may be a 1-bit flag, but is not limited thereto. Also, information on whether there is a change in brightness may be encoded through a header higher than the prediction block. For example, the information may be encoded in units of sequences, pictures, slices, or tiles.

If it is determined that the current image includes a region to which at least one second weight prediction parameter is assigned ( S805 ), the encoding apparatus may encode the second weight prediction parameter for each of the M regions included in the current image ( S805 ). S806).

Information on the first weight prediction parameter and the second weight prediction parameter may be encoded in units of prediction blocks or may be encoded through a header higher than the prediction block. For example, information on the first weight prediction parameter and the second weight prediction parameter may be encoded in units of sequences, pictures, slices, or tiles.

It is also possible to encode the first weighted prediction parameter and the second weighted prediction parameter in different layers. As an example, the first weight prediction parameter may be encoded in units of images, and the second weight prediction parameters may be encoded in units of slices.

The first weight prediction parameter and the second weight prediction parameter may include a multiplication parameter and an addition parameter. In this case, the multiplication parameter may be divided into a prediction value determined based on the precision N of the multiplication parameter and a difference value between the multiplication parameter and the prediction value. As an example, 1<<N may be set as a predicted value of the multiplication parameter, and a difference between the multiplication parameter and the predicted value (1<<N) may be set as a difference value. The encoding apparatus may encode information about a multiplication parameter by encoding the difference value. The precision N of the current block may be encoded in units of blocks, slices, or pictures, and may be transmitted to a decoding apparatus through a bitstream.

Alternatively, it is possible to determine the precision N of the current block by the same method in the encoding apparatus and the decoding apparatus. For example, the precision N of the current block may be adaptively determined according to the size, shape, etc. of the current block.

If it is determined that the current image does not include a region to which at least one second weighted prediction parameter is assigned, encoding of the second weighted prediction parameter may be omitted.

The number of regions included in the current block (that is, the number of second weighted prediction parameters or the number of regions from which the second weighted prediction parameter can be derived) information about 'M' is to be encoded in units of blocks or through higher headers, etc. can Alternatively, the encoding apparatus and the decoding apparatus may equally determine the number of regions included in the current block based on a preset condition.

If it is determined that the current image includes at least one region for deriving the second weight prediction parameter, index information indicating the weight prediction parameter of the current block among the plurality of weight prediction parameter candidates may be encoded (S807). In this case, the index information may be encoded in units of prediction blocks.

As another example, the encoding apparatus may encode information for identifying a region from which a weight parameter and a weight prediction parameter are derived. As an example, the encoding apparatus may encode the second weight prediction parameter and the location and size of the region from which the second weight parameter is derived, or an index assigned to the region.

If it is assumed that the indexes of the weight prediction parameter candidates start from index 0, the index information will indicate any one of '0' to 'the number of weight prediction parameter candidates - 1'. Here, the number of weight prediction parameter candidates may have a fixed value or a variable value, as in the example described above.

The plurality of weight prediction parameter candidates may include at least one of a first weight prediction parameter, a second weight prediction parameter, and an initial weight prediction parameter.

Next, an example of decoding information about a weight prediction parameter in a decoding apparatus will be described in detail.

9 is a diagram illustrating an example of decoding a weight prediction parameter in a decoding apparatus. Referring to FIG. 9 , first, the decoding apparatus may decode, from the bitstream, information indicating whether the contrast of the current image and the brightness of each reference image have changed ( S901 ).

The decoding apparatus may determine whether a first weight prediction parameter exists for the current block based on the information (S902). For example, when it is determined that there is no change in brightness between the current image and the reference image based on the information, the decoding apparatus may not decode information related to the weight prediction parameter anymore.

On the other hand, when it is determined that there is a change in brightness between the current image and the reference image (S902), the decoding apparatus includes at least one region in which the first weighted prediction parameter and the second weighted prediction parameter can be derived from the current image. It is possible to decode the information indicating whether or not it is (S903, S904).

When the current image includes at least one region from which the second weight prediction parameter can be derived, the decoding apparatus may decode information related to the second weight prediction parameter from the bitstream (S905). The decoding apparatus may determine whether a second weight prediction parameter exists with respect to the current block based on the information.

If there are a plurality of M regions from which the second weight prediction parameter can be derived in the current image, the decoding apparatus may decode information about the M second weight prediction parameters ( S906 ).

In this case, information on the first weighted prediction parameter and the second weighted prediction parameter may be decoded in units of prediction blocks or may be decoded through a header higher than the prediction block. For example, information on the first weight prediction parameter and the second weight prediction parameter may be decoded in units of sequences, pictures, slices, or tiles.

The first weighted prediction parameter and the second weighted prediction parameter may be decoded in different layers. For example, the first weight prediction parameter may be decoded in units of images, while the second weight prediction parameters may be decoded in units of slices.

The first weight prediction parameter and the second weight prediction parameter may include a multiplication parameter and an addition parameter. In this case, the decoding apparatus may decode information indicating a difference value between the multiplication parameter and the multiplication parameter predicted value according to the precision N of the multiplication parameter. When the precision of the multiplication parameter is N, 1<<N may be set as the predicted value of the multiplication parameter.

Thereafter, the decoding apparatus may decode index information specifying the weight prediction parameter of the current block from among the plurality of weight prediction parameter candidates ( S907 ). When a weighted prediction parameter for the current block is specified by the index information, inter prediction for the current block may be performed based on the specified weighted prediction parameter.

Specifically, the decoding apparatus may obtain a first prediction pixel by performing inter prediction on the current block, and obtain a second prediction pixel by applying a weighted prediction parameter to the obtained first prediction pixel. As an example, the second prediction pixel may be obtained by adding a multiplication parameter to the first prediction pixel and then adding an addition parameter.

As another example, the decoding apparatus may decode the weight prediction parameter and information identifying the region from which the weight prediction parameter is derived. For example, the decoding apparatus may decode the second weight prediction parameter and the position and size of the region to which the second weight parameter is allocated, or an index allocated to the region, and the like. In this case, the decoding apparatus may adaptively select the weighted prediction parameter according to whether the current block is included in a region to which the second weighted prediction parameter is allocated. For example, when the current block is included in a region to which the second weighted prediction parameter is not allocated, the decoding apparatus may select the weighted prediction parameter for the current block by using the first weighted prediction parameter. On the other hand, when the current block is included in the region to which the second weighted prediction parameter is allocated, the decoding apparatus performs inter prediction on the current block by using the second weighted prediction parameter allocated to the region including the current block. can do.

In the above-described embodiment, it is determined whether the first weighted prediction parameter can be used as a weighted prediction parameter candidate based on information indicating whether there is a change in brightness between the current image and the reference image, and a weight is applied to the current image. It has been described that whether the second weighted prediction parameter can be used as a weighted prediction parameter candidate is determined according to whether a region from which the prediction parameter can be derived is included. That is, it has been described that whether or not to add the second weight prediction parameter to the weight prediction candidate is determined according to whether the current image includes a region from which the weight prediction parameter can be derived.

Unlike the described example, when there is a change in brightness between the current image and the reference image, the encoding apparatus determines the number of available weighted prediction parameter candidates instead of information indicating whether the second weighted prediction parameter is included in the current image. It can also be encoded.

In this case, the decoding apparatus may decode at least one weight prediction parameter candidate based on the received number information. For example, if the number of available weight prediction parameters is N, a weight prediction parameter candidate may be constructed using one first weight prediction parameter and N-1 second weight prediction parameters.

In the above-described embodiment, the weighted prediction parameter includes the first weighted prediction parameter generated by the difference in brightness between the current image and the reference image, and the first weighted prediction parameter generated by the difference in brightness between the partial region in the current image and the partial region in the reference image. It has been described as including 2 weighted prediction parameters. However, the above-described first weight prediction parameter and second weight prediction parameter describe an embodiment in which the weight prediction parameter is generated, and the present invention is not limited thereto.

Example methods of the present disclosure are expressed as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, other steps may be included in addition to the illustrated steps, other steps may be included except some steps, or additional other steps may be included except some steps.

Various embodiments of the present disclosure do not list all possible combinations, but are intended to describe representative aspects of the present disclosure, and matters described in various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure includes software or machine-executable instructions (eg, operating system, application, firmware, program, etc.) that cause an operation according to the method of various embodiments to be executed on a device or computer, and such software or and non-transitory computer-readable media in which instructions and the like are stored and executed on a device or computer.

Claims (14)

obtaining a first prediction pixel by performing inter prediction on the current block;
determining a weight prediction parameter candidate of the current block;
determining a weighted prediction parameter of the current block from the weighted prediction parameter candidates; and
obtaining a second prediction pixel of the current block by applying the weighted prediction parameter to the first prediction pixel of the current block;
The number of weight prediction parameter candidates available for the current block is variably determined based on the size of the current block,
When the size of the current block is the first block size, the number of weight prediction parameter candidates available for the current block is 5;
When the size of the current block is the second block size, the number of weight prediction parameter candidates available for the current block is less than five. delete delete The method of claim 1,
and decoding index information specifying the weight prediction parameter of the current block from among the plurality of weight prediction parameter candidates when the determined number of weight prediction parameter candidates is plural. obtaining a first prediction pixel by performing inter prediction on the current block;
determining a weight prediction parameter candidate of the current block;
determining a weighted prediction parameter of the current block from the weighted prediction parameter candidates; and
obtaining a second prediction pixel of the current block by applying the weighted prediction parameter to the first prediction pixel of the current block;
The number of weight prediction parameter candidates available for the current block is variably determined based on the size of the current block,
When the size of the current block is the first block size, the number of weight prediction parameter candidates available for the current block is 5;
When the size of the current block is the second block size, the number of weight prediction parameter candidates available for the current block is less than five. A computer-readable recording medium storing a bitstream, comprising:
The bitstream includes a coded residual coefficient of a residual block obtained based on a second prediction pixel of the current block,
The second prediction pixel of the current block is obtained by applying a weighted prediction parameter of the current block to the first prediction pixel of the current block,
The weight prediction parameter is determined from weight prediction parameter candidates of the current block,
The first prediction pixel is obtained by performing inter prediction on the current block,
The number of weight prediction parameter candidates available for the current block is variably determined based on the size of the current block,
When the size of the current block is the first block size, the number of weight prediction parameter candidates available for the current block is 5;
When the size of the current block is the second block size, the number of weight prediction parameter candidates available for the current block is less than five. delete delete delete delete delete delete delete delete

Images