KR101187580B1 - Method and apparatus for compensating illumination compensation and method and apparatus for encoding moving picture based on illumination compensation, and method and apparatus for encoding moving picture based on illumination compensation - Google Patents

Abstract

The present invention relates to an illuminance compensation method, wherein an illuminance compensation method of a reference block according to the present invention receives a pixel value of the reconstructed neighboring pixels of the current block and a pixel value of the reconstructed neighboring pixels of the reference block, Based on the pixel values of the reconstructed neighboring pixels of the reference block and the pixel values of the reconstructed neighboring pixels of the reference block, illuminance compensation may be performed on the reference block so that video encoding and decoding can be performed without transmitting parameters for illuminance compensation. By doing so, there is an effect that the coding efficiency can be increased.

Illumination Compensation Video Coding Decoding Multiview

Description

Illumination compensation method and apparatus and method and apparatus for video encoding using same and apparatus {Method and apparatus for compensating illumination compensation and method and apparatus for encoding moving picture based on illumination compensation, and method and apparatus for encoding moving picture based on illumination compensation}

The present invention relates to an illumination compensation method and apparatus, and more particularly, to an illumination compensation method and apparatus for video encoding and decoding.

Multi-view coding (MVC) for three-dimensional display applications is characterized by incompletely calibrated cameras, different perspective projection directions, and different reflections when predicting between adjacent views. An illumination change between adjacent views occurs due to effects, etc., which causes a problem of lowering coding efficiency. In addition, even in the case of a single view, there is a problem in that the coding efficiency due to the change in illuminance decreases in the case of a screen change.

To solve this problem, the H.264 standard uses a technique called weighted prediction. This weighted prediction technique is applied to motion compensation at the slice level, and roughness is compensated according to the appropriate weighted factor W and additional offset O. An improved method for such a weighted prediction technique is illumination change-adaptive motion estimation / compensation (ICA ME / MC).

According to the illuminance transform adaptive motion prediction and compensation scheme, in the case of the Y component, ICA ME / MC is performed in units of 16 × 16 blocks, and a difference value of illumination change (DVIC) is obtained for each macroblock. Lose.

There are two modes in ICA ME / MC. One of the two modes is the IC-Inter 16x16 mode using ICA ME / MC, used in P or B slices. Another mode is IC-Direct 16x16 mode, which does not use ICA ME, which is used only in B slices. To compensate for coarse illuminance transformation, one bit of flag, mb_ic_flag, is required for each inter 16x16 and each direct 16x16 block mode.

Since there is a very high correlation between the DVIC of the current block and the DVIC of adjacent blocks, the DVIC of the current block has a method of encoding and sending a difference between the DVICs of adjacent blocks.

Hereinafter, ICA ME / MC in macroblock units for inter 16x16 mode will be described.

For ICA ME / MC, a new sum of absolute difference (SAD) must be defined. Suppose that a pixel at position (i, j) of the current frame is f (i, j), and a pixel at position (i, j) of a reference frame is r (i, j). Is calculated as in Equation 1 below. Here, SxT may be 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4, and the like.

Here, (x, y) is a candidate motion vector, and (m, n) means the position of the current block.

To compensate for illuminance conversion, a new SAD is needed. This is obtained by the following equations (2) and (3).

Here, Mcur is an average value of the pixels in the current block, and Mref is an average value of the pixels in the reference block. Also, (p, q) is the position of the reference block. The new SAD, NewSAD (x, y), is obtained by the following equation.

In the ICA ME / MC method, a block for minimizing NewSAD (x, y), for example, a 16 × 16 block, is searched based on Equation 3, and a corresponding MV is found.

When the motion vector MV (x`, y`) is set such that NewSAD (x, y) is minimized, the illuminance compensation residual signal NewR (i, j) is determined by Equation 4 below.

At this time, the 1-bit flag mb_ic_flag is stored in a syntax to indicate whether ICA ME / MC is used. The DPCM value of DVIC is also included in the syntax. In this embodiment, if mb_ic_flag is 0, it indicates that ICA MC has not been performed for the current block. In addition, when mb_ic_flag is 1, it indicates that ICA MC is performed for the current block.

If mb_ic_flag is 1, the ICA ME / MC unit of the decoding apparatus obtains a reconstructed pixel by using Equation 5 below.

In Equation 5, NewR`` (i, j) is a reconstructed illuminance compensation residual signal, and f` (i, j) is a pixel in the reconstructed current frame.

In the conventional ICA ME / MC method, since the DVIC information needs to be transmitted, there is a problem that the coding efficiency is lowered.

SUMMARY OF THE INVENTION The present invention provides an improved illuminance compensation method in which the transmission of DVIC information can be omitted by improving the conventional illuminance compensation scheme, and a video encoding and decoding method and apparatus employing the illuminance compensation method. It is.

In order to achieve the above object, the illumination compensation method of the reference block according to the present invention comprises the steps of receiving the pixel value of the reconstructed peripheral pixels of the current block and the reconstructed peripheral pixels of the reference block; Performing illuminance compensation on the reference block based on the input pixel values of the reconstructed neighboring pixels of the current block and the reconstructed neighboring pixels of the reference block.

In addition, in order to achieve the above object, the step of performing the illuminance compensation is based on the correlation of the pixel value of the reconstructed peripheral pixels of the current block to be encoded of the current frame and the pixel value of the reconstructed peripheral pixels of the reference block Calculating an illuminance compensation parameter for the illuminance compensation of; Preferably, based on the calculated illuminance compensation parameter, generating an illuminance compensated reference block.

 In addition, to achieve the above object, performing the illumination compensation

Equation

Determining a _{x , y} and b _{x , y} to minimize the J value;

Using the determined a _{x , y} and b _{x , y}

Generating a reference block whose illuminance is compensated for, wherein a _{x, y} and b _{x , y} vary in value depending on the motion vector (x, y), for each motion vector Is constant, f` (i, -1) and f` (-1, j) are pixel values of the reconstructed neighboring pixels of the current block, and r`x, y (i, -1) and r`x, y (-1, j) is a pixel value of the reconstructed neighboring pixels of the motion compensated reference block, r`x, y (i, j) represents a motion compensated reference block, and is preferably an illuminance compensated reference block. Do.

In addition, in order to achieve the above object, performing illuminance compensation generates a reference block whose illuminance is compensated for by using an average value of a difference between the values of the reconstructed neighboring pixels of the current block and the values of the reconstructed neighboring pixels of the reference block. It is preferable to further include a step.

In addition, in order to achieve the above object, performing illuminance compensation may be performed by using the difference between the average value of the pixel values of the restored neighboring pixels of the current block and the average value of the values of the restored neighboring pixels of the reference block. It is preferable to include the step of generating a block.

In addition, in order to achieve the above object, it is preferable that the reference block is a block in one reference frame of the reconstructed frames of the adjacent view in multi-view coding.

The object of the present invention also includes an illuminance compensation unit configured to perform illuminance compensation on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block and the pixel values of the reconstructed neighboring pixels of the reference block. It can be achieved by the device.

In addition, the illuminance compensator according to the present invention for achieving the above object based on the correlation between the pixel value of the reconstructed peripheral pixels of the current block to be encoded of the current frame and the pixel value of the reconstructed peripheral pixels of the reference block. An illuminance compensation parameter calculator configured to calculate an illuminance compensation parameter for illuminance compensation of the block; The illuminance-compensated reference block generator may be configured to generate an illuminance-compensated reference block based on the calculated illuminance compensation parameter.

The object of the present invention is to perform illuminance compensation on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be encoded and the pixel values of the reconstructed neighboring pixels of the reference block; It may be achieved by a video encoding method based on illuminance compensation, which includes performing motion prediction based on a reference block on which illuminance compensation is performed.

In addition, the task includes an illuminance compensator for performing illuminance compensation on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be encoded and the pixel values of the reconstructed neighboring pixels of the reference block; It may be achieved by the video encoding apparatus based on the illuminance compensation including a motion predictor which performs the motion prediction based on the reference block on which the illuminance compensation is performed.

The object of the present invention is to perform illuminance compensation on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be decoded and the pixel values of the reconstructed neighboring pixels of the reference block; It may be achieved by a video decoding method based on illuminance compensation including performing motion prediction based on a reference block on which illuminance compensation is performed.

In addition, the object may include an illuminance compensation unit configured to perform illuminance compensation on the reference block based on the pixel values of the restored neighboring pixels of the current block to be decoded and the pixel values of the restored neighboring pixels of the reference block; It may be achieved by the video decoding apparatus based on the illumination compensation, characterized in that it comprises a motion prediction unit for performing a motion prediction based on the reference block on which the illumination compensation is performed.

In addition, the task includes the steps of receiving the pixel value of the reconstructed peripheral pixels of the current block and the pixel value of the reconstructed peripheral pixels of the reference block; Performing illuminance compensation on the reference block based on the input pixel values of the reconstructed neighboring pixels of the current block and the reconstructed neighboring pixels of the reference block. The program can be achieved by a computer readable recording medium having recorded thereon.

The object of the present invention is to perform illuminance compensation on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be encoded and the pixel values of the reconstructed neighboring pixels of the reference block; A computer-readable recording medium having recorded thereon a program for executing a video encoding method based on illuminance compensation, including performing motion prediction based on a reference block on which illuminance compensation has been performed, may be achieved.

The object of the present invention is to perform illuminance compensation on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be decoded and the pixel values of the reconstructed neighboring pixels of the reference block; A computer readable recording medium having recorded thereon a program for executing a motion compensation decoding-based video decoding method comprising performing motion prediction based on the reference block on which the illumination compensation has been performed may be achieved.

As described above, in the video encoding and decoding method to which the illumination compensation method according to the present invention and the illumination compensation method according to the present invention are applied, there is an effect that the coding efficiency can be increased because the parameter for illumination compensation is not transmitted. .

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing an illuminance compensation device according to the present invention.

The illuminance compensator 100 includes an illuminance compensation parameter calculator 120 and an illuminance compensated reference block generator 140.

The illuminance compensation parameter calculator 120 calculates an illuminance compensation parameter based on the reconstructed neighbor pixel value of the current block and the reconstructed neighbor pixel value of the reference block, and outputs it to the illuminance compensated reference block generator 140. do. In the case of single view coding, the reference block is the reference block of the reconstructed previous frame. Also, in the case of optional multi-view coding, it may be a block in one reference frame of the reconstructed frames of the adjacent view.

Hereinafter, a method of calculating illuminance compensation parameters according to the present invention will be described with reference to FIGS. 2A and 2B. Hereinafter, each case of the 16x16 mode, the 8x8 mode, the 4x4 mode, and the adaptive mode will be described.

First, a method of calculating illuminance compensation parameters in 16x16 mode will be described with reference to FIGS. 2A and 2B. Illuminance compensation method according to the remaining modes will be described later.

The thick line block of FIG. 2 (a) is a 16-by-16 sized Y block, which is a reconstructed reference block corresponding to the motion vector (x, y) in the reference frame. The restored pixel value of the motion vector (x, y) in the reference block (i, j) corresponding to the position is represented by <sub>x r`, y</sub> (i, j). Here, the subscript `means the restored value. Meanwhile, the pixels displayed in dark colors are reconstructed peripheral pixel values. The reference frame is the previous frame in the case of a single view, and the restored frame of the adjacent view in the case of the MVC scheme.

The thick line block in FIG. 2 (b) is a 16-by-16 block of Y, the current block. The original pixel value at the position (i, j) in the current block is represented by f (i, j). Peripheral pixels shown in dark colors are used for motion estimation compensation. The peripheral pixels used for the motion estimation compensation are reconstructed pixels. As in FIG. 2 (a), the subscript `means a restored value.

As shown in Equation 6, the predictor of f (i, j) in the current block may be represented by a linear _function of r` _{x , y} (i, j).

The predictors obtained in Equation 6 are clipped to [0,255] for an 8-bit image.

As in Equation 6, if a _{x , y} and b _{x , y} are determined, the predictor may be calculated. In this embodiment, the difference between the illuminance of the pixels in the current block and the illuminance of the neighboring pixels of the current block is not large. Compute illuminance compensation parameters a _{x , y} and b _{x , y corresponding} to.

That is, since a _{x , y} and b _{x , y} can be determined using the neighboring reconstructed pixels of the current block and the reference block, even if a _{x , y} and b _{x , y} are not transmitted through the syntax from the encoder to the decoder. In the decoder, a _{x , y} and b _{x , y} can be calculated. Therefore, there is an effect that the transmission data can be reduced.

In the following, three embodiments for determining a _{x , y} and b _{x , y} according to the present invention are described. Three embodiments are a linear regression method, an average of difference based prediction (ADP) method, and a difference of average based prediction (DAP) method.

First, the linear regression method will be described with reference to Equations 7 and 8 below.

According to the linear regression method, it is possible to determine a _{x , y} and b _{x , y} which minimize the value of Equation 7 below.

Here, f` (i, -1) and f` (-1, j) are pixel values of neighboring pixels of the current block. <sub>Also, r` x, y (i,</sub> -1) and f` (-1, j) is a pixel value of peripheral pixels of the motion-compensation reference block. Also, a _{x , y} and b _{x , y} vary in value depending on the motion vector (x, y), and are constant for each motion vector.

By calculating the partial derivatives, a _{x , y} and b _{x , y} minimizing J in Equation 7 are given by Equation 8 below.

When the size of the current block and the reference block is 16x16, N in Equation 8 is 32, and f` (n) denotes a restored peripheral pixel of the current block, and f` (-1, j) of FIG. corresponds to any one of f` (i, -1). Where i, j is a value between 0 and 15. Also, r` _x, and _y (n) corresponds to any one of the pixels of the _{r` x, y (i, -1</sub> ) and <sub>r` x, y (-1, j} ).

In the following, a _{x , y} and b _{x , y} calculation methods according to an average of difference based prediction (ADP) method will be described with reference to Equation (9).

In the mean-based prediction method, a _{x and y} are fixed to 1 and only b _{x and y} are obtained. That is, a _{x b, y} A method for determining the average of a difference between the peripheral restored pixel, b _{x, y} can be determined by Equation (9) below.

In the following, with reference to equation (10) by taking the average difference based prediction will be described in accordance with the (Average Difference of DAP based Prediction) method a _{x, y,} and _{x b, y} calculated.

In the average difference-based prediction method, a _{x and y} are fixed to 1 and only b _{x and y} are obtained. That is, there is provided a method of determining the average of the drive between the _{x b, y} around the restored pixel, b _{x, y} can be determined by the equation (10) below.

As such, the illuminance compensation parameter calculation unit 120 calculates a _{x , y} and b _{x , y} in the same manner as the linear regression method, the difference average based prediction method, and the mean difference based prediction method, and calculates the illumination compensation. Output to the reference block generator 140.

The illuminance-compensated reference block generation unit 140 uses the input illuminance compensation parameters a _{x , y} and b _{x , y} to compensate the illuminance of the predictor corresponding to f (i, j) according to Equation (6). A reference block is generated, and the generated roughness compensated reference block is output to a motion predictor (not shown).

3 is a flowchart illustrating an illuminance compensating method performed by the illuminance compensating apparatus of FIG. 1.

In operation 310, the pixel values of the restored neighboring pixels of the current block and the pixel values of the restored neighboring pixels of the reference block are input. In the case of single view coding, the reference block is the reference block in the reconstructed previous frame. Also, optionally in the case of multi-view coding, it is a block in one reference frame of the reconstructed frames of the adjacent view.

In operation 320, illuminance compensation is performed on the reference block based on the pixel values of the restored neighboring pixels of the current block and the pixel values of the restored neighboring pixels of the reference block. In operation 320, an illuminance compensation parameter is generated based on a correlation between the pixel values of the reconstructed neighboring pixels of the current block and the pixel values of the reconstructed neighboring pixels of the reference block, as shown in Equations 7 to 10. In addition, an illuminance compensated reference block is generated according to Equation 6 using the generated illuminance compensation parameter.

4 is a block diagram illustrating a video encoding apparatus to which an illumination compensation method according to the present invention is applied.

The video encoder according to the present invention includes a transform and quantizer 410, an inverse transform and inverse quantizer 420, a frame storage unit 430, an illumination compensation unit 440, a ME / MC unit 450, and a first adder. 460, a second adder 462, and an entropy encoder 470.

The transform and quantization unit 410 transforms the input image data in order to remove spatial redundancy of the image data. Further, transform coefficient values obtained by transform encoding are quantized according to a predetermined quantization step to obtain N × M data, which is two-dimensional data composed of quantized transform coefficient values. An example of an image transform to be used is a discrete cosine transform (DCT). Quantization is performed according to a predetermined quantization step.

The inverse transform and inverse quantizer 420 inverse quantizes the quantized image data in the transform and quantizer 410, and inverse image transforms, for example, inverse DCT the inverse quantized image data.

The second adder 462 generates a reconstructed image by adding the predicted image output from the ME / MC unit 450 and the data reconstructed by the inverse transform and inverse quantization unit 420.

The frame storage unit 430 stores the image reconstructed by the second adder 462 in units of frames.

The illuminance compensator 440 receives the reconstructed neighbor pixel value of the current block and the reconstructed neighbor pixel value of the reference block from the frame storage unit 430, generates an illuminance compensated prediction block, and generates the generated illuminance compensation. The predicted block is output to the ME / MC unit 450. Optionally, if the input image is an MVC-based image, the peripheral pixel values of the reference block are the reconstructed peripheral pixel values of the reference block located within the frame of the adjacent view, in which case the peripheral pixel values of the reference block are MVC based reference frame storage. It is input from a part (not shown).

Since the illuminance compensator 440 performs the same function as the illuminance compensator 100 of FIG. 1 according to the present invention, a detailed description of the illuminance compensator 440 will be omitted for simplicity.

The ME / MC unit 450 estimates a motion vector per macroblock (MV) based on the image data of the current frame input and the illumination compensation reference block output from the illumination compensation unit 440. In addition, a motion compensated prediction region P based on the estimated motion vector, for example, a 16 × 16 region selected by motion estimation is generated and output to the first adder 460.

That is, the ME / MC unit 450 uses the illuminance-compensated reference block obtained by the illuminance compensator 440, and according to Equation 11 below, the SAD value whose illuminance corresponding to the current block is compensated, that is, IC-SAD ( Obtain the illumination compensated SAD value. In addition, the final motion vector is searched through a comparison of IC SAD values.

In Equation 11, a _{x , y} and b _{x , y} vary depending on the motion vector (x, y), and are constant for each motion vector.

Optionally, the ME / MC unit 450 further includes a comparator (not shown).

The comparator is not based on the encoding method based on the image data of the current frame inputted from the illuminance compensator 440 and the illuminance compensation output from the image data of the current frame inputted from the frame storage unit 430. The efficiency of coding schemes based on unreferenced blocks are compared, and a scheme of high coding efficiency is selected. In this case, the ME / MC unit generates a motion compensated prediction region P obtained according to the selected motion prediction and compensation scheme, and outputs it to the first adder 460.

In addition, optionally 1-bit flag information indicating whether illumination compensation is provided in units of macroblocks may be transmitted to the decoder through syntax. In addition, it is also possible to selectively evaluate the performance of the illuminance compensation method in units of GOP or slices, and transmit flag information indicating whether illuminance is compensated in units of GOP or slices.

The first adder 460 converts the difference between the original image and the predictor output from the ME / MC unit 40 into a predetermined block unit and inputs the transformed and quantized unit 410. If the final motion vector for the current block determined by the motion vector search process is (p, q), the first adder 460 calculates the residual (p, q) of the current block as shown in Equation 12 below. And it is output to the transform and quantization unit 410.

In Equation 12, a _{p , q} and b _{p , q} vary depending on the motion vector (p, q), and are constant for each motion vector.

The entropy encoder 470 receives the quantized transform coefficients output from the transform and quantization unit 410 and information about the motion vector output from the motion prediction and compensation unit to entropy-encode the finally encoded bitstream. Output

FIG. 5 is a flowchart illustrating an encoding method performed in a video encoding apparatus to which the illumination compensation method according to the present invention of FIG. 4 is applied.

In step 510, transformation and quantization are performed.

In operation 520, a reconstructed image is generated by performing inverse transform and inverse quantization on the transformed and quantized data.

In operation 530, illuminance compensation is performed on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be encoded and the pixel values of the reconstructed neighboring pixels of the reference block. In this embodiment, the reference block is a block in the previous frame. However, optionally, if the reference block is an image based on MVC, the reference block is a block in a frame of an adjacent view.

In operation 540, the prediction block is generated by performing motion prediction and compensation based on the reference block on which the illumination compensation is performed.

In operation 550, a residual image is generated based on the original image and the generated prediction block. A transform and quantization process is performed on the generated residual image, and then the transformed and quantized residual image is entropy coded together with a motion vector obtained at the time of motion prediction.

6 is a block diagram illustrating a video decoding apparatus to which an illumination compensation method according to the present invention is applied.

The video decoding apparatus illustrated in FIG. 6 includes an entropy decoder 610, an inverse quantization and inverse transform unit 620, a frame storage unit 630, an illuminance compensator 640, an ME / MC unit 650, and an adder. 660.

The entropy decoder 610 entropy decodes the encoded input stream to extract image data, a motion vector, and the like. Entropy-decoded image data is input to the inverse quantization and inverse transform unit 620, and motion vector information is input to the ME / MC unit 650.

The inverse transform and inverse quantizer 620 performs inverse transform and inverse quantization on the image data extracted by the entropy decoder 610.

The frame storage unit 630 stores image data inversely quantized and inversely transformed by the inverse transform and inverse quantizer 620 in units of frames.

The illuminance compensator 640 receives the reconstructed peripheral pixel value of the current block and the reconstructed peripheral pixel value of the reference block input from the frame storage unit 630, generates an illuminance compensated reference block, and generates the generated illuminance compensation. The predicted block is output to the ME / MC unit 650. Since the illuminance compensator 640 performs the same function as the illuminance compensator of FIG. 1 according to the present invention, a detailed description of the illuminance compensator 640 is omitted for simplicity.

Optionally, when the video decoding apparatus of FIG. 6 is an MVC-based decoding apparatus, the peripheral pixel value of the reference block is a reconstructed peripheral pixel value of the reference block located in the frame of the adjacent view, in which case the peripheral pixel value of the reference block. Is input from an MVC based frame storage (not shown).

The ME / MC unit 650 estimates a motion vector per macroblock MV based on the image data of the current frame input and the illumination compensation reference block output from the illumination compensation unit 640. In addition, a motion compensated prediction region P based on the estimated motion vector, for example, a 16 × 16 region selected by motion estimation is generated and output to the adder 660.

That is, the ME / MC unit 650 uses the illuminance-compensated reference block obtained by the illuminance compensator 640 to obtain an SAD value whose luminance is compensated for the current block according to Equation 11, that is, an IC-SAD value. Obtain In addition, the final motion vector is searched through a comparison of IC SAD values.

Also, optionally, the ME / MC unit 650 further includes a comparing unit (not shown). The comparator includes an encoding method based on the image data of the current frame input and the illumination compensation reference block output from the illumination compensator 640, and the illumination compensation output from the image data and frame storage 630 of the current frame input. Efficiency of coding schemes based on unreferenced reference blocks are compared, and a scheme according to motion prediction and compensation with high coding efficiency is selected. In this case, the ME / MC unit generates a motion compensated prediction region P obtained according to the selected motion prediction and compensation scheme, and outputs it to the adder 660.

Also, optionally, the motion prediction based on the illuminance-compensated reference block determined based on flag information indicating whether illuminance compensation extracted by the entropy decoding unit 610 is based on the compensated prediction block or the non-illumination uncompensated reference block. The motion prediction and the compensated prediction block are output to the adder 660.

The adder 660 adds the image reconstructed by the inverse transform and inverse quantizer 620 and the predictor output from the ME / MC unit 650 to the display unit (not shown) and the frame storage unit 630. Output

According to the present exemplary embodiment, the adder 660 may include residue` (p, q), which is a restored residual signal obtained by reconstructing the residual signal residue (p, q) input from the inverse quantization and inverse transform unit 620, and ME / MC. The final reconstructed pixel f ′ (x, y) in the current block is calculated by adding pixel values of the prediction block input from the block 650 according to Equation 13 below.

In Equation 13, a _{p , q} and b _{p , q} vary depending on the motion vector (p, q), and are constant for each motion vector.

7 is a flowchart illustrating a video decoding method performed in the video decoding apparatus shown in FIG. 6.

In operation 710, the encoded input stream is entropy decoded to extract image data and motion vectors.

In operation 720, inverse transform and inverse quantization are performed on the image data extracted in operation 710.

In operation 730, image data reconstructed by inverse quantization and inverse transformation is stored in units of frames.

In operation 740, illuminance compensation is performed on the reference block based on the pixel values of the reconstructed neighboring pixels of the current block to be decoded and the pixel values of the reconstructed neighboring pixels of the reference block. At this time, when the input stream is based on MVC, the peripheral pixel value of the reference block is the reconstructed peripheral pixel value of the reference block in the frame of the adjacent view.

In operation 750, the prediction block is generated by performing motion prediction and compensation based on the reference block on which the illumination compensation is performed and the extracted motion vector.

In operation 760, a reconstructed image is generated using the prediction block generated in operation 750. For example, a reconstructed image is generated by adding a prediction block to an image on which inverse transformation and inverse quantization are performed.

Hereinafter, the illuminance compensation method and the motion compensation method in the 8x8 mode will be described with reference to FIG. 8.

In the case of the 8x8 mode, four 8x8 subblocks exist in a 16x16 block, and each has a different motion vector. In this case, an example of the processing procedure is shown in FIG. In the 8x8 mode, only the processing unit that performs illuminance compensation is changed to the 8x8 subblock, and the processing method is the same as in the 16x16 mode. That is, a _{x , y} and b _{x , y} are obtained through linear regression, ADP, and DAP. In addition, the motion compensation process in the video encoding apparatus and the video decoding apparatus is the same as in the case of the 16x16 mode except that the 8x8 block unit.

Hereinafter, the illuminance compensation method and the motion compensation method in the 4x4 mode will be described with reference to FIG. 9.

When processing in 4x4 mode, 16 4x4 subblocks are present in the 16x16 block, and each has a different motion vector. In this case, an example of the processing procedure is shown in FIG. In the 4x4 mode, only the processing unit that performs illuminance compensation is changed to the 4x4 subblock, and the processing method is the same as in the 16x16 mode. That is, a _{x , y} and b _{x , y} are obtained through linear regression, ADP, and DAP. In addition, the motion compensation process in the video encoding apparatus and the video decoding apparatus is the same as in the case of the 16x16 mode except for the 4x4 block unit.

Hereinafter, an illuminance compensation method and a motion compensation method in the adaptive mode will be described.

In the adaptive mode, the video encoding apparatus may perform illumination compensation and encoding according to 16x16 mode, 8x8 mode, and 4x4 mode in macroblock units, and select one of them. At this time, the residual block according to the mode is encoded and transmitted. At this time, the information indicating the mode is transmitted to the video decoding apparatus. In addition, if the illuminance compensation and motion estimation are possible for each of the 16x16 mode, the 8x8 mode, and the 4x4 mode, the decoding apparatus may optionally not transmit the mode information.

In the present embodiment, only the 16x16 mode, the 8x8 mode, and the 4x4 mode are dealt with, but the illumination compensation method according to the present invention can be applied to any other type of block compensation.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1 is a block diagram showing an illuminance compensation device according to the present invention.

2 (a) and 2 (b) are diagrams for explaining a method of calculating illuminance compensation parameters according to the present invention.

3 is a flowchart illustrating a roughness compensation method according to the present invention.

4 is a block diagram illustrating a video encoding apparatus to which an illumination compensation method according to the present invention is applied.

5 is a flowchart illustrating a video encoding method to which the illumination compensation method according to the present invention is applied.

6 is a block diagram illustrating a video decoding apparatus to which an illumination compensation method according to the present invention is applied.

7 is a flowchart illustrating a video decoding method to which the illumination compensation method according to the present invention is applied.

8 is a diagram illustrating an encoding sequence in the 8x8 mode.

9 is a diagram illustrating an encoding sequence in the 4x4 mode.

Claims (21)

delete In the illumination compensation method of the reference block for motion prediction, Receiving pixel values of the reconstructed neighboring pixels of the current block and the pixel values of the reconstructed neighboring pixels of the reference block; Performing illuminance compensation on the reference block based on a difference between a pixel value of the restored neighboring pixels of the current block and a pixel value of the restored neighboring pixels of the reference block; Performing the illuminance compensation is Calculating an illuminance compensation parameter for illuminance compensation of the reference block based on a correlation between pixel values of reconstructed neighboring pixels of the current block to be encoded of the current frame and pixel values of the reconstructed neighboring pixels of the reference block; ; Generating an illuminance compensated reference block based on the calculated illuminance compensation parameter. In the illumination compensation method of the reference block for motion prediction, Receiving pixel values of the reconstructed neighboring pixels of the current block and the pixel values of the reconstructed neighboring pixels of the reference block; Performing illuminance compensation on the reference block based on a difference between a pixel value of the restored neighboring pixels of the current block and a pixel value of the restored neighboring pixels of the reference block; Performing the illuminance compensation is Equation

Determining a _{x, y} and b _{x, y} to minimize the J value; Using the determined a _{x, y} and b _{x, y} the following equation

Generating a reference block whose illumination is compensated based on the; Here, a _{x, y} and b _{x, y} vary in value depending on the motion vector (x, y), are constants for each motion vector, and f` (i, -1) and f` (-1 is the pixel value of the reconstructed neighboring pixels of the current block, and r`x, y (i, -1) and r`x, y (-1, j) are the reconstructed neighboring pixels of the reference block. And r'x, y (i, j) is a motion compensated reference block, and is an illuminance compensated reference block. In the illumination compensation method of the reference block for motion prediction, Receiving pixel values of the reconstructed neighboring pixels of the current block and the pixel values of the reconstructed neighboring pixels of the reference block; Performing illuminance compensation on the reference block based on a difference between a pixel value of the restored neighboring pixels of the current block and a pixel value of the restored neighboring pixels of the reference block; Performing the illuminance compensation is And generating an illuminance-compensated reference block by using an average value of a difference between the values of the reconstructed neighboring pixels of the current block and the values of the reconstructed neighboring pixels of the reference block. In the illumination compensation method of the reference block for motion prediction, Receiving pixel values of the reconstructed neighboring pixels of the current block and the pixel values of the reconstructed neighboring pixels of the reference block; Performing illuminance compensation on the reference block based on a difference between a pixel value of the restored neighboring pixels of the current block and a pixel value of the restored neighboring pixels of the reference block; Performing the illuminance compensation is And generating a reference block whose illuminance is compensated for by using a difference between an average value of pixel values of the restored neighboring pixels of the current block and an average value of the restored neighboring pixels of the reference block. Compensation method. delete In the illumination compensation device of the reference block for motion prediction, An illuminance compensator configured to perform illuminance compensation on the reference block based on a difference between a pixel value of the reconstructed neighboring pixels of the current block and a pixel value of the reconstructed neighboring pixels of the reference block, The illuminance compensation unit Illumination compensation for calculating an illumination compensation parameter for illuminance compensation of the reference block based on a correlation between pixel values of reconstructed neighboring pixels of the current block to be encoded of the current frame and pixel values of the reconstructed neighboring pixels of the reference block. A parameter calculator; And an illuminance-compensated reference block generator for generating an illuminance-compensated reference block based on the calculated illuminance compensation parameter. In the illumination compensation device of the reference block for motion prediction, An illuminance compensator configured to perform illuminance compensation on the reference block based on a difference between a pixel value of the reconstructed neighboring pixels of the current block and a pixel value of the reconstructed neighboring pixels of the reference block, The illuminance compensation unit Equation

In the above, a _{x, y} and b _{x, y} are determined to minimize the J value _, and by using the determined a _{x, y} and b _{x, y} ,

And an illuminance-compensated reference block generator that generates an illuminance-compensated reference block based on the Here, a _{x, y} and b _{x, y} vary in value depending on the motion vector (x, y), and are constant for each motion vector, and f` (i, -1) and f` (-1, j) is a pixel value of the reconstructed peripheral pixels of the current block, and r`x, y (i, -1) and r`x, y (-1, j) are the pixels of the reconstructed peripheral pixels of the reference block. Wherein r′x, y (i, j) is a motion compensated reference block, and is an illuminance compensated reference block. In the illumination compensation device of the reference block for motion prediction, An illuminance compensator configured to perform illuminance compensation on the reference block based on a difference between a pixel value of the reconstructed neighboring pixels of the current block and a pixel value of the reconstructed neighboring pixels of the reference block, The illuminance compensation unit And an illuminance-compensated reference block generator for generating an illuminance-compensated reference block by using an average value of a difference between the value of the reconstructed neighboring pixels of the current block and the values of the reconstructed neighboring pixels of the reference block. Illuminance compensation device. In the illumination compensation device of the reference block for motion prediction, An illuminance compensator configured to perform illuminance compensation on the reference block based on a difference between a pixel value of the reconstructed neighboring pixels of the current block and a pixel value of the reconstructed neighboring pixels of the reference block, The illuminance compensation unit And an illuminance-compensated reference block generation unit configured to generate an illuminance-compensated reference block by using a difference between an average value of pixel values of the restored neighboring pixels of the current block and an average value of the restored neighboring pixels of the reference block. Illuminance compensation device, characterized in that. delete In the video encoding method based on the illumination compensation, Performing illuminance compensation on the reference block based on a difference between the pixel values of the reconstructed neighboring pixels of the current block to be encoded and the pixel values of the reconstructed neighboring pixels of the reference block for motion prediction; Performing motion prediction based on the reference block on which the illuminance compensation is performed, Performing the illuminance compensation is Generating an illuminance compensation parameter for illuminance compensation of the reference block based on a correlation between the pixel values of the reconstructed neighboring pixels of the current block to be encoded of the current frame and the pixel values of the reconstructed neighboring pixels of the reference block; ; And generate an illuminance compensated reference block based on the generated illuminance compensation parameter. delete In the video encoding apparatus based on the illumination compensation, An illuminance compensator for performing illuminance compensation on the reference block based on a difference between the pixel values of the reconstructed neighboring pixels of the current block to be encoded and the pixel values of the reconstructed neighboring pixels of the reference block for motion prediction; A motion predictor configured to perform motion prediction based on the reference block on which the illumination compensation is performed; The illuminance compensation unit Illumination compensation for generating an illuminance compensation parameter for illuminance compensation of the reference block based on a correlation between pixel values of reconstructed neighboring pixels of the current block to be encoded of the current frame and pixel values of the reconstructed neighboring pixels of the reference block A parameter generator; And an illuminance-compensated reference block generator for generating an illuminance-compensated reference block based on the generated illuminance compensation parameter. delete In the video decoding method based on the illumination compensation, Performing illuminance compensation on the reference block based on a difference between the pixel values of the reconstructed neighboring pixels of the current block to be decoded and the pixel values of the reconstructed neighboring pixels of the reference block for motion compensation; Performing motion compensation based on the reference block on which the illuminance compensation is performed, Performing the illuminance compensation is Generating an illuminance compensation parameter for illuminance compensation of the reference block based on a correlation between the pixel values of the reconstructed neighboring pixels of the current block to be decoded of the current frame and the pixel values of the reconstructed neighboring pixels of the reference block; ; And generating an illuminance compensated reference block based on the generated illuminance compensation parameter. delete In the video decoding apparatus based on the illumination compensation, An illuminance compensator for performing illuminance compensation on the reference block based on a difference between the pixel values of the reconstructed neighboring pixels of the current block to be decoded and the pixel values of the reconstructed neighboring pixels of the reference block for motion compensation; A motion predictor configured to perform motion compensation based on the reference block on which the illumination compensation is performed; The illuminance compensation unit An illuminance compensation for generating an illuminance compensation parameter for illuminance compensation of the reference block based on a correlation between the pixel values of the reconstructed neighboring pixels of the current block to be decoded of the current frame and the pixel values of the reconstructed neighboring pixels of the reference block. A parameter generator; And an illuminance-compensated reference block generator for generating an illuminance-compensated reference block based on the generated illuminance compensation parameter. A computer-readable recording medium having recorded thereon a program for executing the illuminance compensation method according to any one of claims 2 to 5. A computer-readable recording medium having recorded thereon a program for executing the video encoding method of claim 12. A computer-readable recording medium having recorded thereon a program for executing the video decoding method of claim 16.

Images