WO2013027950A2 - Apparatus and method for encoding/decoding a depth image using variable quantization parameters - Google Patents

Abstract

The present invention relates to an apparatus and method for encoding/decoding a depth image using variable quantization parameters. The encoding apparatus can extract a high-quality image even with an identical bit amount by variably setting quantization parameters for each block of a depth image.

Description

Apparatus and method for encoding / decoding of depth image using variable quantization parameter

Embodiments of the present disclosure relate to an apparatus and method for encoding / decoding a depth image, and more particularly, to an apparatus and method for encoding / decoding a depth image by setting variable quantization parameters for each block of the depth image. will be.

The conventional compression method treats the depth image as an independent image and compresses it by applying the existing image compression method such as H.264 / MPEG-4 AVC. However, the depth image has a very different property from the color image.

For example, the depth image is composed of a piece-wise flat portion, which has more low frequency components than the color image, and the flat portions form distinct outlines to form a frequency component of the intermediate band. Due to these properties, it is difficult to expect large compression efficiency with image compression schemes such as H.264 / MPEG-4 AVC based on block quantization.

In addition, the color image and the depth image representing the image at the same time and at the same time in color and depth have a large correlation with each other. Therefore, it is inefficient to encode and transmit each of them, and it is necessary to use the correlation to improve the coding efficiency.

An encoding apparatus according to an embodiment of the present specification includes a parameter setting unit configured to variably set a quantization parameter for each of a plurality of blocks included in a depth image; And an image encoder which encodes the depth image by using the quantization parameter.

In the encoding apparatus according to the embodiment of the present specification, the parameter setting unit may determine a difference value of a quantization parameter to be applied to a global quantization parameter for each of the plurality of blocks.

In the encoding apparatus according to the embodiment of the present specification, the parameter setting unit increases the quantization parameter when a block included in the depth image belongs to a flat region and quantizes when the block included in the depth image belongs to a complex region. The parameter can be reduced.

In the encoding apparatus according to the embodiment of the present specification, the parameter setting unit may set a quantization parameter that minimizes a cost function for a block included in a depth image.

In the encoding apparatus according to the embodiment of the present specification, the parameter setting unit may set the quantization parameter by using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.

In the encoding apparatus according to the embodiment of the present specification, the parameter setting unit is configured to adjust the quantization parameter based on a distortion function which is a difference between before and after the movement of the entire image and the blocks of the color image, respectively. Can be set.

In the encoding apparatus according to the embodiment of the present specification, the parameter setting unit may set the quantization parameter by linearizing the distortion function when the distortion function is not a linear function.

An encoding apparatus according to an embodiment of the present disclosure includes a parameter setting unit configured to set a first quantization parameter to be applied to the entire depth image for each of a plurality of blocks included in the depth image; A cost function calculator for calculating a cost function by respectively applying second parameters preset to the first quantization parameter; And a parameter selector for selecting an optimal second parameter in consideration of the calculated cost function.

An encoding apparatus according to an embodiment of the present disclosure includes a parameter setting unit configured to set a first quantization parameter to be applied to the entire depth image for each of a plurality of blocks included in the depth image; A parameter determination unit that determines a second quantization parameter to be variably set for each of the plurality of blocks; A cost function calculator for calculating a cost function by applying a plurality of preset third parameters to the first quantization parameter in which the second quantization parameter is reflected; And a parameter selector that selects an optimal third parameter in consideration of the calculated cost function.

Decoding apparatus according to an embodiment of the present specification includes a bitstream receiving unit for receiving a bitstream in which a depth image is encoded; A parameter setting unit that variably sets a quantization parameter for each of the plurality of blocks included in the depth image; And an image decoder which decodes the depth image by using the quantization parameter.

In the decoding apparatus according to an embodiment of the present specification, the parameter setting unit increases the quantization parameter when a block included in the depth image belongs to a flat region, and quantizes when the block included in the depth image belongs to a complex region. The parameter can be reduced.

In the decoding apparatus according to an embodiment of the present specification, the parameter setting unit may set a quantization parameter that minimizes a cost function for a block included in a depth image.

In the decoding apparatus according to the exemplary embodiment of the present specification, the parameter setting unit may set the quantization parameter by using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.

According to an embodiment of the present disclosure, a decoding apparatus includes: a parameter receiver configured to receive an optimal second parameter calculated through a first quantization parameter and a cost function to be applied to an entire depth image for each of a plurality of blocks included in a depth image; And an image decoder which decodes a depth image by using the first quantization parameter and the second quantization parameter.

The decoding apparatus according to an embodiment of the present specification uses a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image, a second quantization parameter to be variably set for each of the plurality of blocks, and a cost function. A parameter receiver configured to receive the calculated optimal third parameter; And an image decoder configured to decode a depth image in consideration of the first quantization parameter, the second quantization parameter, and the third quantization parameter.

An encoding method according to an embodiment of the present specification comprises the steps of: variably setting a quantization parameter for each of a plurality of blocks included in a depth image; And encoding the depth image using the quantization parameter.

An encoding method according to an embodiment of the present specification may variably set a quantization parameter for each of a plurality of blocks using neighboring blocks in a decoded color image or a decoded depth image.

The encoding method according to an embodiment of the present disclosure may increase the quantization parameter when the block included in the depth image belongs to the flat region and decrease the quantization parameter when the block included in the depth image belongs to the complex region. .

An encoding method according to an embodiment of the present specification may set a quantization parameter for minimizing a cost function for a block included in a depth image.

The encoding method according to an embodiment of the present specification may set the quantization parameter by using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.

The encoding method according to an embodiment of the present disclosure may set a quantization parameter based on a distortion function that is a difference between before and after moving the entire image of the color image and the blocks of the color image, respectively.

In the encoding method according to the exemplary embodiment of the present specification, when the distortion function is not a linear function, the distortion function may be linearized to set a quantization parameter.

An encoding method according to an embodiment of the present disclosure includes setting a first quantization parameter to be applied to an entire depth image for each of a plurality of blocks included in the depth image; Calculating a cost function by applying preset second parameters to the first quantization parameter, respectively; And selecting the optimal second parameter in consideration of the calculated cost function.

An encoding method according to an embodiment of the present disclosure includes setting a first quantization parameter to be applied to an entire depth image for each of a plurality of blocks included in the depth image; Determining a second quantization parameter to be variably set for each of the plurality of blocks; Calculating a cost function by applying each of a plurality of preset third parameters to a first quantization parameter in which the second quantization parameter is reflected; And selecting the optimal third parameter in consideration of the calculated cost function.

A decoding method according to an embodiment of the present specification comprises the steps of: receiving a bitstream encoded with a depth image; Variably setting a quantization parameter for each of the plurality of blocks included in the depth image; And decoding the depth image using the quantization parameter.

According to an embodiment of the present disclosure, when a block included in a depth image belongs to a flat region, the quantization parameter may be increased, and when the block included in the depth image belongs to a complex region, the quantization parameter may be reduced. .

The decoding method according to an embodiment of the present specification may variably set a quantization parameter for each of a plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image.

The decoding method according to the exemplary embodiment of the present specification may set the quantization parameter by using the relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.

According to an embodiment of the present disclosure, a decoding method includes receiving an optimal second parameter calculated through a first quantization parameter and a cost function to be applied to an entire depth image for each of a plurality of blocks included in the depth image; And decoding the depth image using the first quantization parameter and the second quantization parameter.

A decoding method according to an embodiment of the present specification uses a first quantization parameter to be applied to an entire depth image for each of a plurality of blocks included in a depth image, a second quantization parameter to be variably set for each of the plurality of blocks, and a cost function. Receiving the calculated optimal third parameter; And decoding the depth image in consideration of the first quantization parameter, the second quantization parameter, and the third quantization parameter.

According to one embodiment of the present specification, by setting a variable quantization parameter for each block of the depth image, it is possible to reduce the bit amount compared to the same image quality.

According to one embodiment of the present specification, the image quality can be increased compared to the same bit amount by setting a variable quantization parameter for each block of the depth image.

According to an embodiment of the present disclosure, the encoding apparatus may directly set the quantization parameter without encoding and transmitting the quantization parameter set for each block to the decoding apparatus, thereby improving the coding efficiency of the depth image.

1 is a diagram illustrating an encoding apparatus according to a first embodiment.

2 is a diagram illustrating an encoding apparatus according to a second embodiment.

3 is a diagram illustrating an encoding device according to a third embodiment.

4 is a diagram illustrating a decoding apparatus according to the first embodiment.

5 is a diagram illustrating a decoding apparatus according to a second embodiment.

6 is a diagram illustrating a result of variably setting a quantization parameter.

7 is a diagram illustrating distortion of a composite image.

8 is a diagram for describing a process of setting a quantization parameter using a distortion function.

9 is a diagram illustrating a change in distortion of a color image due to pixel shift.

10 is a diagram illustrating an encoding method according to an embodiment of the present specification.

11 is a diagram illustrating a decoding method according to an embodiment of the present specification.

12 is a diagram illustrating a process of selecting a quantization parameter according to an embodiment of the first specification.

13 is a diagram illustrating a process of selecting a quantization parameter according to a second embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an encoding apparatus according to a first embodiment.

Referring to FIG. 1, the encoding apparatus 100 may include a parameter setting unit 101 and an image encoder 102.

The parameter setting unit 101 may variably set a quantization parameter (QP) for each of the plurality of blocks included in the depth image. That is, the parameter setting unit 101 may set different quantization parameters for each of the plurality of blocks included in the depth image.

In general, when encoding a color image, the image processing apparatus predetermines a global quantization parameter applied to the entirety of the color image, and applies the global quantization parameter uniformly to all blocks of the color image. This is because similar picture quality can be expected by applying the same global quantization parameter to each block when encoding a color image.

However, the depth image, unlike the color image, no matter how the same global quantization parameter is applied to each block, the actual quality of the synthesized image is different for each region of the depth image. Because the depth area of the object represented in the depth image is complex, accurate depth information with high quality is required, but since the flat area of the object is not complicated, it is possible to derive a composite image of good quality even with the depth information with low quality. can do.

For this reason, the encoding apparatus 100 according to an embodiment of the present specification may set a variable quantization parameter (QP) for each block of the depth image in view of the synthesized image. By setting different quantization parameters for each block of the depth image, the image actually displayed has a similar picture quality for each block, so that the user can view a more natural image.

In addition, in the flat region of the object, since the same image quality can be displayed even if the amount of bits required for encoding is reduced, the amount of bits compared to the same image quality of the synthesized image may be reduced, or the image quality of the synthesized image may increase.

For example, the parameter setting unit 101 may variably set a quantization parameter for each of the plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image. In this case, the parameter setting unit 101 may determine a difference value of the quantization parameter to be applied to the global quantization parameter for each of the plurality of blocks. That is, the parameter setting unit 101 is configured to perform global quantization parameters applied to the entire depth image.

Can be determined.

Denotes the amount of increase or decrease of the quantization parameter to be applied to the global quantization parameter applied to the entire depth image for the quantization parameter to be variably set for each block of the depth image.

The parameter setting unit 101 may set a quantization parameter that minimizes a cost function for blocks included in the depth image. In detail, when the block included in the depth image belongs to the flat region, the parameter setting unit 101 may increase the quantization parameter. On the contrary, when a block included in the depth image belongs to a complicated region, the parameter setting unit 101 may reduce the quantization parameter.

In particular, the parameter setting unit 101 may set the quantization parameter by using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.

In detail, the parameter setting unit 101 may move the entire image of the color image and the blocks of the color image, respectively, and then set the quantization parameter based on a distortion function that is a difference between before and after the color image. In this case, when the distortion function is not a linear function, the parameter setting unit 101 may set the quantization parameter by linearizing the distortion function.

The image encoder 102 may encode the depth image by using the quantization parameter. At this time, the image encoder 102 does not encode the quantization parameter. This is because, when encoding a quantization parameter that is set variably for each block, information required for transmission increases rapidly. Instead, the decoding apparatus 200 may improve encoding efficiency by predicting the same parameter as the quantization parameter set by the direct encoding apparatus 100.

2 is a diagram illustrating an encoding apparatus according to a second embodiment.

The encoding apparatus 200 of FIG. 2 may include a parameter setting unit 201, a cost function calculating unit 202, a parameter selecting unit 203, and an image encoder 204.

The parameter setting unit 201 may set a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image. Here, the first quantization parameter refers to a global quantization parameter that is equally applied to each block. As mentioned in FIG. 1, different quantization parameters may be applied to a plurality of blocks of a depth image.

The cost function calculator 202 may calculate a cost function by applying a plurality of preset second parameters to the first quantization parameter. That is, the cost function calculator 202 may encode the depth image based on the sum of the first quantization parameter and the second quantization parameter, and then calculate the cost function with respect to the encoded result. At this time, the cost function is calculated by measuring the rate-distortion cost.

The parameter selector 203 may select an optimal second parameter in consideration of the calculated cost function. That is, the parameter selector 203 may select a second parameter having a minimum cost function among the plurality of second parameters.

The image encoder 204 may encode the depth image by using the first quantization parameter and the selected second parameter. In this case, the first quantization parameter and the second quantization parameter may also be encoded. The bitstream may be generated according to the encoding result.

3 is a diagram illustrating an encoding device according to a third embodiment.

The encoding apparatus 300 of FIG. 3 may include a parameter setting unit 301, a parameter determination unit 302, a cost function calculator 303, a parameter selector 304, and an image encoder 305.

The parameter setting unit 301 may set a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image. Here, the first quantization parameter refers to a global quantization parameter that is equally applied to each block.

The parameter determiner 302 may determine a second quantization parameter to be variably set for each of the plurality of blocks. Here, the second quantization parameter is

It may mean. At this time,

Denotes the amount of increase or decrease of the quantization parameter to be applied to the global quantization parameter for the quantization parameter to be variably set for each block of the depth image. In one example, the second quantization parameter may be determined using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.

The cost function calculator 303 may calculate a cost function by applying a plurality of third parameters preset to the first quantization parameter in which the second quantization parameter is reflected. The cost function calculator 303 may replace the result of the sum of the first quantization parameter and the second quantization parameter with the first quantization parameter. Then, the cost function calculator 303 may encode the depth image based on the sum of the first quantization parameter and the third quantization parameter, and then calculate the cost function with respect to the encoded result. At this time, the cost function is calculated by measuring the rate-distortion cost.

In one example, the cost function calculator 303 may change the range of the third quantization parameter according to the range of the second quantization parameter. For example, if the second quantization parameter represents a range of -6 to 6, the third quantization parameter may represent a range of 0 to 12 or -12 to 0.

The cost function calculator 303 may determine the cost function using the result of dividing the third quantization parameter by a predetermined interval. For example, if the third quantization parameter has a value from 0 to 12, the cost function calculation unit 303 uses only values of a predetermined interval such as 0, 6, 12, etc., rather than using all the values of the third quantization parameter. The cost function can be calculated.

The parameter selector 304 may select the optimal third parameter in consideration of the calculated cost function. That is, the parameter selector 203 may select the third parameter having the least cost function among the plurality of third parameters.

The image encoder 305 may encode the depth image by using the first quantization parameter, the second quantization parameter, and the third parameter. In this case, the first quantization parameter, the second quantization parameter, and the third parameter may also be encoded. The bitstream may be generated according to the encoding result.

4 is a diagram illustrating a decoding apparatus according to the first embodiment.

Referring to FIG. 4, the decoding apparatus 400 may include a bitstream receiver 401, a parameter setting unit 402, and an image decoder 403.

The bitstream receiver 401 may receive a bitstream in which a depth image is encoded.

The parameter setting unit 402 may variably set the quantization parameter for each of the plurality of blocks included in the depth image. According to the exemplary embodiment of the present specification, the quantization parameter variably set for each block in the encoding apparatus 100 is not encoded and transmitted to the decoding apparatus 400.

As described above, the encoding efficiency may be reduced when the encoding apparatus 100 encodes and transmits the quantization parameter set for each block. As a result, the decoding apparatus 400 may improve encoding efficiency by predicting the same parameter as the quantization parameter used in the encoding apparatus 100. Since the parameter setting unit 402 operates in the same manner as the parameter setting unit 101 of the encoding apparatus 100, a detailed description thereof will be omitted.

The image decoder 403 may decode the depth image by using a quantization parameter.

5 is a diagram illustrating a decoding apparatus according to a second embodiment.

Referring to FIG. 5, the decoding apparatus 500 may include a parameter receiver 501 and an image decoder 502.

For example, the first second parameter calculated through the first quantization parameter and the cost function to be applied to the entire depth image may be received for each of the plurality of blocks included in the depth image. Here, reference may be made to FIG. 2 regarding the first quantization parameter and the second quantization parameter.

As another example, the parameter receiver 501 may receive an optimal second parameter calculated through a first quantization parameter and a cost function to be applied to the entire depth image for each of the blocks included in the depth image. Here, reference may be made to FIG. 3 regarding the first quantization parameter, the second quantization parameter, and the third quantization parameter.

The image decoder 502 may decode the depth image in consideration of the first quantization parameter, the second quantization parameter, and the third quantization parameter.

FIG. 6 is a diagram illustrating a result of variably setting a quantization parameter according to an embodiment of the present specification.

Referring to FIG. 6, different quantization parameters are set for each block included in the depth image. If the global quantization parameter applied to the entire depth image is QP, a difference value from the global quantization parameter may be set for each block.

When = 0, it is equal to the global quantization parameter.

And,

If> 0, the final quantization parameter applied to the block has a larger value than the global quantization parameter, so that the coding strength is relatively increased. At this time,

Blocks> 0 belong to an uncomplicated area, such as the planar area of an object. Contrary,

If <0, the final quantization parameter applied to the block has a smaller value than the global quantization parameter, so that the coding strength is relatively decreased. At this time,

Blocks of <0 belong to complex areas such as the boundary area of the object.

That is, the encoding apparatus 100 does not significantly reduce the quality of the synthesized image displayed to the user even if the encoding is strongly performed by increasing the quantization parameter in an uncomplicated region such as a flat region of the object.

The encoding apparatus 100 may maintain the image quality of the synthesized image displayed to the user by weakly encoding the quantization parameter by reducing the quantization parameter in a complicated region such as the boundary region of the object. The same applies to the decoding apparatus 200.

7 illustrates distortion of a composite image according to an exemplary embodiment of the present specification.

Setting variable quantization parameters for each block of the depth image is associated with generation of distortion of the synthesized image obtained by combining the color image and the depth image. In detail, the interpolated curve A (x) is warped after the original depth image is warped, resulting in a curve B (x) due to an error generated during the encoding / decoding process of the depth image. According to one embodiment of the present specification, an area existing between curves A (x) and B (x) may be defined as a distortion function.

The distortion of the synthesized image is called intermediate image (V) and ground-truth, which are synthesized by a warping function (fw) based on the original color image (C) and the original depth image (D). The difference between the color image C ′ generated and the intermediate image V ′ synthesized into the depth image D ′ is calculated.

Accordingly, the distortion D _syn of the synthesized image may be determined according to Equation 1 below.

As can be seen in Equation 1, the actual disparity error has a linear relationship with the error of depth. In Equation 1, C denotes a color image and D denotes a depth image. V is an original intermediate image, and V 'is an intermediate image synthesized from a color image and a depth image.

Depth error caused by noise such as encoding / decoding of depth image

When we say epth, the error of actual disparity (

) Is as shown in Equation 2 below.

Accordingly, the prediction error D _syn of the synthesized image is expressed by Equation 3 below.

Here, Δp is a translation error in the x-axis direction and represents a linear relationship with the error of the depth image. C 'means a color image in which noise is included in the original color image C due to encoding or decoding. Pixel C (x, y) of any color image should be moved to the position of (x ', y) in the composite image of the intermediate view by the depth value D (x, y). However, the D (x, y) value is changed to D '(x, y) due to an error generated through the encoding / decoding process. Accordingly, the pixel C (x, y) of the color image is D (x, y). (x '+ not actually (x', y) due to an error in y)

Go to (x, y), y).

Therefore, according to Equation 3, pixels C '(x', y) of the color image and pixels of the color image

The difference between C '(x + Δp (x, y), y) is defined as the distortion of the composite image at the intermediate view.

 According to an embodiment of the present disclosure, the encoding apparatus 101 may predict the distortion of the synthesized image based on the distortion of the current block and the distortion of the current block even without actually generating the synthesized image.

8 is a diagram for describing a process of setting a quantization parameter by using a distortion function, according to an exemplary embodiment.

When encoding the input image, a predetermined global quantization parameter for determining the image quality of the frame is set. The encoding apparatus 100 may determine whether the error of the depth image generated by the global quantization parameter finally affects the quality of the synthesized image by using Equation 1.

That is, the encoding apparatus 100 may influence the error of the depth image on the image quality of the synthesized image based on the result of shifting the color image by 1 pixel (or 1/2, 1/4 pixel) using Equation 1. You can judge.

That is, if the error of the depth image generated by the result of encoding using the given global quantization parameter exists by 1 pixel in every pixel, the difference between the image shifted by the color image by 1 pixel and the original image is the distortion of the synthesized image. Has a value similar to

According to this method, the encoding apparatus 100 shifts a pixel into 1, 2, 3, ..., etc. within a frame to determine how much the error of the depth image affects the error of the actual composite image, and then The trend can be used to predict.

When the above-described method is applied for each block instead of the entire frame, the relationship between the error of the depth image in the block and the distortion of the synthesized image is shown in FIG.

If the relationship between the error of the depth image and the distortion of the composite image is assumed to be represented by the distortion function f (x) associated with one frame and the distortion function ((g (x)) associated with the block. 101 denotes a global quantization parameter for each block through the ratio of two distortion functions.

Can be set.

According to an embodiment of the present disclosure, the encoding apparatus 101 assumes that the distortion function f (x) associated with a frame and the distortion function (g (x)) associated with a block are both linearly related. The encoding apparatus 101 assumes f (x) to be y = kx, and assumes g (x) to be y = hx, where k and h mean arbitrary constants.

The encoding apparatus 100 has a minimum cost function of a distortion function f (x) associated with a frame and a distortion function associated with a block ((g (x)).

Can be determined.

In this case, assuming that the distortion and the ratio in the cost function of the distortion function are inversely proportional to each other, that is, (D ~ 1 / R), the distortion function associated with the frame (f (x)) and the distortion function associated with the block Each slope ratio of ((g (x)) is always assigned a constant.

This minimizes the cost function of the distortion function f (x) associated with the frame and the distortion function associated with the block ((g (x)).

Is determined according to equation (4).

In Equation 4, k and h denote a distortion function f (x) associated with a frame and a distortion function associated with a block ((g (x)), respectively, and c denotes a distortion function associated with a frame f ( x)) and the ratio of the distortion function ((g (x)) associated with the block.

The relationship between the bit amount currently allocated to the block and the bit amount to be newly allocated to the block is determined according to Equation 5.

Here, the bit amount currently allocated to the block is R _1, and the bit amount to be newly allocated to the block is R ₂ .

In the case where the quantization interval is increased by twice each time the quantization parameter that determines the image quality of the image according to the bit amount is generally increased by k,

May be determined according to Equation 6.

In Equation 6, c is derived through Equation 4, and k represents the degree to which the quantization parameter is increased.

Equation 3 described above refers to an equation for simply predicting the distortion of the composite image. Such distortion modeling methods may exist in various ways. According to an embodiment of the present disclosure, the distortion of the synthesized image may be variously changed according to the distortion modeling method.

For example, in the case of using the distortion modeling method as shown in Equation 3, the graphs shown in FIG. 5 are in the form of y = x ² , respectively. In this case, Equation 6 is changed into a simple form as shown in Equation 7 below.

Here, E _f and E _b denote spatial complexity of the entire frame and each block, respectively. The spatial complexity E is determined by the following equation (8).

As another example, the distortion modeling method may be applied to consider not only a synthesized image but also a depth image. The distortion function finally derived is derived as in Equation 9 below.

Here, D denotes a distortion function finally derived, D _syn denotes a distortion of a synthesized image, and D _d denotes a distortion of a depth image.

Wow

Denote weights for the distortion of the composite image and the distortion of the depth image, respectively.

When a distortion modeling method such as Equation 9 is applied, Equation 7 may be modified to Equation 10 below.

In Equation 10, Γ represents a ratio (Dsyn / Dd) of D _syn to D _d , and represents a different value for each image.

For Equation 3 described above,

In setting the, a decoded color image is always required. As in the case where the 3DV system encodes the depth image before the color image,

If it is not possible to utilize a color image when setting the, basically it is difficult to apply the method proposed in the present invention.

However, when the decoded color image of the neighboring view exists, the decoded color image of the neighboring view may be warped and moved to the current view. Derived from Equation 6

Since the encoding apparatus 100 and the decoding apparatus 200 are derived through the same operation, the encoding apparatus 100 does not need to be separately transmitted to the decoding apparatus 200.

Meanwhile, when setting the quantization parameter for each block of the depth image, after applying various preset quantization parameters in the encoding apparatus 100, the decoding apparatus 200 selects a quantization parameter having a minimum RD cost. Can be sent to.

According to one embodiment of the present specification, the above-described methods for determining the quantization parameter may be used in combination. Specifically, the quantization parameter is a quantization parameter (derived through Equation 6).

) Is applied to the global quantization parameter (QP) that is the quantization reference.

Can be set. Then, in the next step,

QP + using the value

+

Calculates the furnace rate-distortion cost, and the best QP + with the lowest calculated cost

+

Can be set. In the following,

Optimal rate-distortion cost among candidates

Is defined as dQP.

At this time,

Based on the range of dQP may be set differently. E.g,

Is in the range of -6 to 6, the dQP may be changed to 0-12 or -12-0. In addition, when dQP is in the range of 0 to 12, the rate-distortion cost may be calculated at a predetermined interval such as 0, 6, and 12 without calculating the rate-distortion cost for all values.

9 is a diagram illustrating a change in distortion of a color image due to pixel shift.

Referring to FIG. 9, it can be seen that the distortion of the color image increases in response to the pixel shift in the depth image. 9 shows that the distortion increases in proportion to the pixel shift.

Equations 4, 5, and 6 described above are all derived assuming that the distortion function f (x) associated with a frame and the distortion function ((g (x)) associated with a block are linear functions as shown in FIG. 5. .

If the distortion function f (x) associated with the frame and the distortion function (g (x)) associated with the block are not linear functions, the encoding apparatus 100 linearizes the distortion function through a polynomial function or an exponential function. Then, Equations 4, 5, and 6 may be applied.

10 is a diagram illustrating an encoding method according to an embodiment of the present specification.

In operation 1001, the parameter setting unit 101 of the encoding apparatus 100 may variably set a quantization parameter for each of the plurality of blocks included in the depth image.

In this case, the parameter setting unit 101 may variably set a quantization parameter for each of the plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image.

When the block included in the depth image belongs to the flat region, the parameter setting unit 101 may increase the quantization parameter. If the block included in the depth image belongs to a complex region, the parameter setting unit 101 may reduce the quantization parameter.

In addition, the parameter setting unit 101 may set a quantization parameter that minimizes a cost function for blocks included in the depth image. In this case, the parameter setting unit 101 may set the quantization parameter by using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image. In detail, the parameter setting unit 101 may move the entire image of the color image and the blocks of the color image, respectively, and then set the quantization parameter based on a distortion function that is a difference between before and after the color image. If the distortion function is not a linear function, the parameter setting unit 101 may set the quantization parameter by linearizing the distortion function.

In operation 1002, the image encoder 102 of the encoding apparatus 100 may encode the depth image by using a quantization parameter.

11 is a diagram illustrating a decoding method according to an embodiment of the present specification.

In operation 1101, the bitstream receiver 201 of the decoding apparatus 200 may receive a bitstream in which a depth image is encoded.

In operation 1102, the parameter setting unit 202 of the decoding apparatus 200 may variably set a quantization parameter for each of the plurality of blocks included in the depth image. Since the operation of the parameter setting unit 202 is the same as the parameter setting unit 201 described with reference to FIG. 6, a detailed description thereof will be omitted. According to an embodiment of the present specification, the encoding apparatus 100 does not encode a quantization parameter that is variably set for each block, and the decoding apparatus 100 predicts the same value as the quantization parameter set in the encoding apparatus 100 so that the quantization parameter is quantized. The amount of bits required for transmission can be reduced, and as a result, the coding efficiency for the depth image can be improved.

In operation 1103, the decoding apparatus 200 may decode the depth image using the quantization parameter.

12 is a diagram illustrating a process of selecting a quantization parameter according to an embodiment of the first specification.

12 may be performed by the encoding apparatus 200 of FIG. 2. Referring to FIG. 12, the encoding apparatus 200 may set QP0, which is a first quantization parameter, for each of a plurality of blocks (1201). In this case, QP0 means a global quantization parameter to be applied to the entire depth image.

The encoding apparatus 200 may calculate a cost function using dQP1 to dQPn, which are second quantization parameters preset in QP0. As an example, the encoding apparatus 200 may calculate a cost function for the depth image by using QP0 + dQP1 1202 to QP0 + dQPn 1205 to which the second quantization parameters dQP1 to dQPn are applied to QP0. The process of calculating the cost function may be performed separately for each of the plurality of blocks included in the depth image.

Then, the encoding apparatus 200 may select dQPn, which is the second quantization parameter when the cost function is minimum, from QP0 + dQP1 1202 to QP0 + dQPn 1205 (1206). Thereafter, the encoding apparatus 200 may perform quantization using the first quantization parameter and the second quantization parameter (1207), and may encode the depth image using the quantization result (1208).

13 is a diagram illustrating a process of selecting a quantization parameter according to a second embodiment.

FIG. 13 may be performed by the encoding apparatus 300 of FIG. 3. Referring to FIG. 13, it is assumed that a first quantization parameter QP0 is set for each of a plurality of blocks. Then, the encoding apparatus 300 is a second quantization parameter to be variably applied to each of the plurality of blocks.

It may be determined (1301). The encoding apparatus 300 compares the first quantization parameter QP0 with QP0.

It can be replaced by the sum of the results (1302).

The encoding apparatus 300 may calculate a cost function by using the third quantization parameters dQP1 to dQPn preset in QP0 derived in operation 1302. For example, the encoding apparatus 300 may calculate a cost function for the depth image by using the QP0 + dQP1 1303 to the QP0 + dQPn 1306 to which the third quantization parameters dQP1 to dQPn are applied to QP0. The process of calculating the cost function may be performed separately for each of the plurality of blocks included in the depth image.

Then, the encoding apparatus 300 may select dQPn, which is the third quantization parameter when the cost function is minimum, from QP0 + dQP1 1203 to QP0 + dQPn 1206 (1307). Thereafter, the encoding apparatus 300 may perform quantization using the first quantization parameter and the third quantization parameter (1308), and may encode the depth image using the quantization result (1309). Here, the second quantization parameter may be derived in the same manner in the encoding apparatus and the decoding apparatus.

Methods according to an embodiment may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of one embodiment of the present specification should not be limited to the described embodiments, but should be defined not only by the claims below but also by the equivalents of the claims.

Claims (37)

1. A parameter setting unit that variably sets a quantization parameter for each of the plurality of blocks included in the depth image; And

   An image encoder which encodes the depth image by using the quantization parameter.

   Encoding apparatus comprising a.
2. The method of claim 1,

   The parameter setting unit,

   And a quantization parameter is variably set for each of the plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image.
3. The method of claim 1,

   The parameter setting unit,

   If the block included in the depth image belongs to the flat region, increase the quantization parameter,

   And a quantization parameter is reduced when a block included in the depth image belongs to a complex region.
4. The method of claim 1,

   The parameter setting unit,

   And a quantization parameter for minimizing a cost function for blocks included in the depth image.
5. The method of claim 1,

   The parameter setting unit,

   And a quantization parameter is set using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.
6. The method of claim 5,

   The parameter setting unit,

   And after each of the entire image of the color image and the blocks of the color image are moved, a quantization parameter is set based on a distortion function which is a difference between before and after the movement.
7. The method of claim 6,

   The parameter setting unit,

   And if the distortion function is not a linear function, linearize the distortion function to set a quantization parameter.
8. A parameter setting unit configured to set a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image;

   A cost function calculator for calculating a cost function by respectively applying second parameters preset to the first quantization parameter;

   Parameter selection unit for selecting an optimal second parameter in consideration of the calculated cost function

   Encoding apparatus comprising a.
9. A parameter setting unit configured to set a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image;

   A parameter determination unit that determines a second quantization parameter to be variably set for each of the plurality of blocks;

   A cost function calculator for calculating a cost function by applying a plurality of preset third parameters to the first quantization parameter in which the second quantization parameter is reflected;

   Parameter selection unit for selecting an optimal third parameter in consideration of the calculated cost function

   Encoding apparatus comprising a.
10. The method of claim 9,

    The parameter determiner,

    And a second quantization parameter is determined using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.
11. The method of claim 9,

    The cost function calculation unit,

    And the range of the third quantization parameter is changed according to the range of the second quantization parameter.
12. The method of claim 8,

    The cost function calculation unit,

    And a cost function is determined based on a result of dividing the third quantization parameter by a predetermined interval.
13. A bitstream receiver for receiving a bitstream encoded with a depth image;

    A parameter setting unit that variably sets a quantization parameter for each of the plurality of blocks included in the depth image; And

    An image decoder which decodes the depth image using the quantization parameter.

    Decoding apparatus comprising a.
14. The method of claim 13,

    The parameter setting unit,

    If the block included in the depth image belongs to the flat region, increase the quantization parameter,

    And if the block included in the depth image belongs to a complex region, the quantization parameter is reduced.
15. The method of claim 13,

    The parameter setting unit,

    And a quantization parameter is variably set for each of the plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image.
16. The method of claim 8,

    The parameter setting unit,

    And a quantization parameter is set using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.
17. A parameter receiver configured to receive an optimal second parameter calculated through a first quantization parameter and a cost function to be applied to the entire depth image for each of the plurality of blocks included in the depth image;

    An image decoder which decodes a depth image by using the first quantization parameter and the second quantization parameter.

    Decoding apparatus comprising a.
18. A parameter for receiving a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image, a second quantization parameter to be variably set for each of the plurality of blocks, and an optimal third parameter calculated through a cost function. Receiving unit;

    An image decoder which decodes a depth image by considering the first quantization parameter, the second quantization parameter, and the third quantization parameter.

    Decoding apparatus comprising a.
19. The method of claim 18,

    The second quantization parameter is

    The decoding apparatus of claim 1, wherein the decoding apparatus is determined using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.
20. The method of claim 18,

    The cost function is

    And the third quantization parameter is calculated using a result of changing the range of the third quantization parameter according to the range of the second quantization parameter.
21. The method of claim 18,

    The cost function is

    And the third quantization parameter is determined using a result of dividing the third quantization parameter by a predetermined interval.
22. Variably setting a quantization parameter for each of the plurality of blocks included in the depth image; And

    Encoding the depth image using the quantization parameter

    Encoding method comprising a.
23. The method of claim 22,

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And a quantization parameter is variably set for each of the plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image.
24. The method of claim 22,

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    If the block included in the depth image belongs to the flat region, increasing the quantization parameter; or

    If the block included in the depth image belongs to a complex region, reducing the quantization parameter

    Encoding method comprising a.
25. The method of claim 22,

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And a quantization parameter for minimizing a cost function for blocks included in the depth image.
26. The method of claim 22,

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And a quantization parameter is set using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.
27. The method of claim 26,

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And after each of the entire image of the color image and the blocks of the color image are moved, a quantization parameter is set based on a distortion function which is a difference between before and after the color image.
28. The method of claim 27,

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And if the distortion function is not a linear function, linearize the distortion function to set a quantization parameter.
29. Setting a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image;

    Calculating a cost function by applying preset second parameters to the first quantization parameter, respectively; And

    Selecting an optimal second parameter in consideration of the calculated cost function

    Encoding method comprising a.
30. Setting a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image;

    Determining a second quantization parameter to be variably set for each of the plurality of blocks;

    Calculating a cost function by applying each of a plurality of preset third parameters to a first quantization parameter in which the second quantization parameter is reflected; And

    Selecting an optimal third parameter in consideration of the calculated cost function

    Encoding method comprising a.
31. Receiving a bitstream encoded with a depth image;

    Variably setting a quantization parameter for each of the plurality of blocks included in the depth image; And

    Decoding the depth image using the quantization parameter

    Decryption method comprising a.
32. The method of claim 31, wherein

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    If the block included in the depth image belongs to the flat region, increasing the quantization parameter; or

    If the block included in the depth image belongs to a complex region, reducing the quantization parameter

    Decryption method comprising a.
33. The method of claim 31, wherein

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And a quantization parameter is variably set for each of the plurality of blocks using neighboring blocks in the decoded color image or the decoded depth image.
34. The method of claim 31, wherein

    The step of variably setting the quantization parameter for each of the plurality of blocks,

    And a quantization parameter is set using a relationship between the entire image of the color image corresponding to the depth image and the blocks of the color image.
35. Receiving an optimal second parameter calculated through a first quantization parameter and a cost function to be applied to the entire depth image for each of the plurality of blocks included in the depth image; And

    Decoding a depth image using the first quantization parameter and the second quantization parameter

    Decryption method comprising a.
36. Receiving a first quantization parameter to be applied to the entire depth image for each of the plurality of blocks included in the depth image, a second quantization parameter to be variably set for each of the plurality of blocks, and an optimal third parameter calculated through a cost function ; And

    Decoding a depth image by considering the first quantization parameter, the second quantization parameter, and the third quantization parameter

    Decryption method comprising a.
37. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 22-36.

Images