WO2012177015A2 - Image decoding/decoding method and device - Google Patents

Abstract

According to the present invention, an image decoding method comprises the steps of: receiving a bit stream; generating a prediction block and a residual block for a current picture on the basis of the bit stream; performing up-sampling on the residual block on the basis of reduced resolution update (RRU)-type information derived from the bit stream; generating a restoration block on the basis of the prediction block and the up-sampled residual block; and performing filtering on the restoration block on the basis of the RRU-type information. According to the present invention, image encoding efficiency is improved and complexity is reduced.

Description

Image coding / decoding method and apparatus

The present invention relates to image processing, and more particularly, to a resolution reduction update method and apparatus thereof.

Digital broadcasting services using 3D video are attracting attention as next generation broadcasting services. The 3D video may provide a realism and immersion by using a plurality of view channels, and the 3D video providing the stereoscopic effect may include a binocular 3D video, a multiview 3D video, a hologram, and the like.

In the case of 3D video, unlike a conventional 2D video, not only a texture video but also a depth video may be encoded and / or decoded together. Unlike a texture image, which is a target of a conventional video compression standard, a depth image may include depth information of objects in a screen. In the case of 3D video, the three-dimensional effect can be represented by the above-described depth information.

An object of the present invention is to provide an image encoding method and apparatus capable of improving image encoding efficiency and reducing complexity.

Another object of the present invention is to provide an image decoding method and apparatus capable of improving image encoding efficiency and reducing complexity.

Another technical problem of the present invention is to provide a resolution reduction update method and apparatus capable of improving image encoding efficiency and reducing complexity.

Another technical problem of the present invention is to provide a filtering method and apparatus for improving image encoding efficiency.

1. An embodiment of the present invention is a video decoding method. The method includes receiving a bitstream, generating a prediction block and a residual block for a current picture based on the bitstream, and reduced-resolution update (RRU) type information derived from the bitstream. Performing up-sampling on the residual block, generating a reconstruction block based on the prediction block and the residual block on which the upsampling is performed, and based on the RRU type information And performing filtering on the reconstruction block, wherein the RRU type information includes sampling ratio information and filter type information. Upsampling is performed based on a sampling rate indicated, and the filter indicated by the filter type information is performed in the filtering step. Based filtering to perform.

2. In 1, the sampling rate information may indicate a higher sampling rate as the frequency of the region corresponding to the residual block is lower in the current block.

3. The method of claim 1, wherein the performing of the filtering comprises: determining a window of a predetermined size and shape for a pixel to be filtered in the residual block and for the pixel to be filtered based on the pixels in the window. The method may further include performing filtering.

4. The method of 3, wherein when the current picture is a picture in a texture video, the filter indicated by the filter type information is a bi-laterally weighted filter that performs weighted averaging based on pixels in the window. filter).

5. The method of 4, wherein the bidirectional weight filter performs weak smoothing when the pixel to be filtered is a pixel in an edge region including an edge, and the pixel to be filtered is configured to perform edge smoothing. Strong smoothing may be performed in the case of a pixel in a non-edge region that does not include.

6. The method of 3, wherein the current picture is a depth picture in a depth video, and a texture picture corresponding to the current picture is available in a texture video corresponding to the depth video. In the performing of the filtering, the filtering may be performed based on texture pixels in an area corresponding to the window in the texture picture.

7. The method of claim 6, wherein the filter indicated by the filter type information may be a bi-laterally weighted Winner_Take_All filter, and the bidirectional weighted WTA filter is a depth pixel in the window based on the texture pixel. A weighting score may be derived for each, and a pixel value of a depth pixel having the highest weight score among the depth pixels may be determined as an output value for the pixel to be filtered, wherein the depth pixel is within the depth picture. It may be a pixel.

8. The method of claim 6, wherein the filter indicated by the filter type information may be a bi-laterally weighted median filter, and the bidirectional weighted median filter may include a depth in the window based on the texture pixel. A weighting score is derived for each pixel, a median value is derived among the depth pixels based on the weight score, and a pixel value of a depth pixel corresponding to the derived median value is obtained for the filtering target pixel. The depth pixel may be a pixel in the depth picture.

9. When the current picture is a depth picture in a depth image, and the texture picture corresponding to the current picture is not valid in the texture image corresponding to the depth image, the filter indicated by the filter type information is And a bidirectional weighted median filter, wherein the bidirectional weighted median filter derives a weighting score for each of the depth pixels based on depth pixels in the window, and based on the weight score The intermediate value may be derived from the pixels, and the pixel value of the depth pixel corresponding to the derived intermediate value may be determined as an output value for the filtering target pixel, and the depth pixel may be a pixel in the depth picture.

10. The method of claim 9, wherein the bidirectional weighted median filter performs weak smoothing when the pixel to be filtered is a pixel in an edge region including an edge, and the pixel to be filtered does not include an edge. In the case of non-edge pixels, strong smoothing may be performed.

11. Another embodiment of the present invention is a video decoding apparatus. The apparatus includes: a receiver for receiving a bitstream, a prediction block and a residual block generator for generating a prediction block and a residual block for a current picture based on the bitstream, and an RRU derived from the bitstream A reconstruction block for generating a reconstruction block based on the up-sampler for performing up-sampling on the residual block, the prediction block, and the residual block on which the up-sampling is performed, based on -Resolution Update) type information A filter unit configured to perform filtering on the reconstruction block based on a generation unit and the RRU type information, wherein the RRU type information includes sampling ratio information and filter type information; The upsampler performs upsampling based on the sampling rate indicated by the sampling rate information, and the filter unit performs the filter type information. Performs filtering based on the filter indicated by.

In 12. 11, the sampling rate information may indicate a higher sampling rate as the frequency of the region corresponding to the residual block is lower in the current block.

13. The method of claim 11, wherein the filter unit is configured to determine a window having a predetermined size and shape with respect to the filtering target pixel in the residual block and perform filtering on the filtering target pixel based on the pixel in the window. Can be.

14. The method of claim 13, wherein when the current picture is a picture in a texture video, the filter indicated by the filter type information is a bi-laterally weighted filter that performs weighted averaging based on pixels in the window. filter).

15. The method of 15. 14, wherein the bidirectional weighting filter performs weak smoothing when the pixel to be filtered is a pixel in an edge region including an edge, and the pixel to be filtered is configured to perform edge smoothing. Strong smoothing may be performed in the case of a pixel in a non-edge region that does not include.

16. The method of 16. 13, wherein the current picture is a depth picture in a depth video, and a texture picture corresponding to the current picture is available in a texture video corresponding to the depth video. The filter unit may perform filtering based on texture pixels in an area corresponding to the window in the texture picture.

17. The method of claim 16, wherein the filter indicated by the filter type information may be a bi-laterally weighted Winner_Take_All filter, wherein the bidirectional weighted WTA filter is a depth pixel in the window based on the texture pixel. A weighting score may be derived for each, and a pixel value of a depth pixel having the highest weight score among the depth pixels may be determined as an output value for the pixel to be filtered, wherein the depth pixel is within the depth picture. It may be a pixel.

18. The method of claim 16, wherein the filter indicated by the filter type information may be a bi-laterally weighted median filter, and the bidirectional weighted median filter is configured to determine a depth in the window based on the texture pixel. A weighting score is derived for each pixel, a median value is derived among the depth pixels based on the weight score, and a pixel value of a depth pixel corresponding to the derived median value is obtained for the filtering target pixel. The depth pixel may be a pixel in the depth picture.

19. The method of claim 13, wherein when the current picture is a depth picture in a depth image and a texture picture corresponding to the current picture is not valid in a texture image corresponding to the depth image, the filter indicated by the filter type information is And a bidirectional weighted median filter, wherein the bidirectional weighted median filter derives a weighting score for each of the depth pixels based on depth pixels in the window, and based on the weight score The intermediate value may be derived from the pixels, and the pixel value of the depth pixel corresponding to the derived intermediate value may be determined as an output value for the filtering target pixel, and the depth pixel may be a pixel in the depth picture.

20. The method of 19, wherein the bidirectional weighted median filter performs weak smoothing when the pixel to be filtered is a pixel in an edge region including an edge, and the pixel to be filtered does not include an edge. In the case of non-edge pixels, strong smoothing may be performed.

According to the image encoding method according to the present invention, image encoding efficiency may be improved and complexity may be reduced.

According to the image decoding method according to the present invention, image encoding efficiency may be improved and complexity may be reduced.

According to the resolution reduction update method according to the present invention, image encoding efficiency may be improved and complexity may be reduced.

According to the filtering method according to the present invention, image encoding efficiency may be improved.

1 is a block diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.

2 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

3 is a block diagram schematically illustrating an image encoding apparatus according to another embodiment of the present invention.

4 is a block diagram schematically illustrating an image decoding apparatus according to another embodiment of the present invention.

5 is a diagram illustrating an upsampling process and a filtering process according to the present invention.

6 is a flowchart schematically showing an embodiment of a decoding method according to the present invention.

7 is a flowchart schematically showing another embodiment of a decoding method according to the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the invention to the specific embodiments. The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the spirit of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

On the other hand, each of the components in the drawings described in the present invention are shown independently for the convenience of the description of the different characteristic functions in the image encoding / decoding apparatus, each component is implemented by separate hardware or separate software It does not mean to be. For example, two or more of each configuration may be combined to form one configuration, or one configuration may be divided into a plurality of configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of the present invention without departing from the spirit of the present invention.

In addition, some of the components may not be essential components for performing essential functions in the present invention, but may be optional components for improving performance. The present invention can be implemented including only the components essential for implementing the essentials of the present invention except for the components used for improving performance, and the structure including only the essential components except for the optional components used for improving performance. Also included within the scope of the present invention.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant description of the same components is omitted.

1 is a block diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus according to the embodiment of FIG. 1 may be applied to both a texture video and a depth video. Referring to FIG. 1, the apparatus for encoding an image may include a motion estimator (ME) 110, a motion compensator (MC) 120, a down-sampler 130, and a converter ( Transformer (T) 135, Quantizer (Q) 140, Entropy coder 150, Inverse Quantizer or Dequantizer (IQ) 155, Inverse Transformer : IT) 160, an up-sampler 170, a filter unit 180, and a reference picture buffer 190.

In the input image input to the image encoding apparatus, the picture may be divided into at least one processing unit. In this case, the processing unit may be a macro block, a prediction unit (PU), a transform unit (TU), or a coding unit (CU). have. Hereinafter, embodiments to be described below are described based on a processing unit generated by dividing a picture to be encoded / decoded (or a current picture), and a unit may be referred to as a block in some cases.

Meanwhile, the texture image and the depth image may include a low frequency region which is high in flatness and a monotonous region, and a high frequency region in which detail characteristics are important. In this case, when the encoding process is performed on all pixel values of the low frequency region where detail is not important, complexity may be excessively high compared to coding efficiency. Since the coding efficiency may not be significantly reduced even when the low frequency region is processed at a low resolution, the low frequency region may be subjected to an encoding process by reducing the resolution through a down sampler, and upsampling through an up sampler during reconstruction and / or decoding. A method of carrying out this can be used. In this case, the sampling ratio of the down sampler and the up sampler may be determined differently depending on the degree of monotony of the image, the degree of importance of the detail of the image, and / or the frequency of the image. Here, the down sampling rate and the up sampling rate may be the same. A specific embodiment of the down / up sampling rate determination method will be described later.

As described above, a method of reducing the resolution in the encoding process and performing upsampling in the reconstruction and / or decoding process according to the frequency of the image may include a reduced resolution update (RRU, hereinafter, a reduced resolution update). And RRU both have the same meaning. Also, resolution reduction updates may sometimes be referred to as reduced resolution updates. In the embodiment of FIG. 1, the above-described resolution reduction update method may be used.

Referring to FIG. 1, the motion predictor 110 may obtain a motion vector based on a reference picture stored in the reference picture buffer 190. The motion compensator 120 may generate a prediction block by performing motion compensation using the motion vector. The motion information used for prediction (eg, a motion vector and a reference picture index) may be encoded by the entropy encoder 150 together with the residual value and transmitted to the decoder.

The down sampler 130 may perform down sampling on a residual value (residual block) between the generated prediction block and the original block. In this case, the down-sampled residual block may be input to the converter 135. Depending on the frequency characteristic of the image, the residual block may be directly input to the converter 135 without passing through the down sampler 130. For example, when the region corresponding to the current block is a high frequency region, since the detailed characteristic of the image is important, the residual block may be directly input to the converter 135 without downsampling.

The transformer 135 may transform the residual block (or downsampled residual block) in a transform unit and generate transform coefficients, and the quantization unit 140 may convert the residual values transformed by the transformer. Quantization may be performed to generate quantization coefficients. In addition, the entropy encoder 150 may output a bitstream by performing entropy encoding on the quantization coefficients.

The inverse quantization unit 155 may inverse quantize values quantized by the quantization unit 140. In addition, the inverse transformer 160 may inversely transform the inverse quantized values by the inverse quantizer 155.

The up sampler 170 may perform upsampling on the residual value and / or the residual block generated by the inverse quantizer 155 and the inverse transformer 160. The upsampled residual block may be combined with the prediction block generated by the motion compensator 120 to generate a reconstructed block. In this case, the filter unit 180 may perform filtering according to the present invention on the reconstruction block. Specific embodiments of the filter unit will be described later. Meanwhile, when downsampling is not performed on the residual block, the residual block generated by the inverse transformer 160 may be combined with the prediction block generated by the motion compensator 120 without passing through the up sampler 170. In this case, for example, the filter unit 180 may not perform filtering on the generated reconstruction block.

The reference picture buffer 190 may store the generated reconstruction block and / or the reconstruction block on which the filtering is performed. The reconstructed blocks and / or pictures stored in the reference picture buffer 190 may be provided to the motion predictor 110 and the motion compensator 120.

2 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention. Referring to FIG. 2, the image decoding apparatus includes an entropy decoder 210, an inverse quantizer IQ 220, an inverse transformer IT 230, an up sampler 240, a motion compensator MC, 250, a filter unit 260, and a reference picture buffer 270.

The image decoding apparatus according to the embodiment of FIG. 2 may be applied to both texture video and depth video, similarly to the image encoding apparatus. In addition, in the embodiment of FIG. 2, the above-described resolution reduction update method may be used.

Referring to FIG. 2, the entropy decoder 210 may perform entropy decoding on the input bitstream. The motion information for generating the prediction block among the information decoded by the entropy decoder 210 may be provided to the motion compensator 250, and the residual value on which entropy decoding is performed may be input to the inverse quantizer 220. The inverse quantization unit 220 may perform inverse quantization based on the residual value on which entropy decoding is performed, and the inverse transform unit 230 performs the inverse transformation on the inverse quantization result performed by the inverse quantization unit 220. The block can be derived.

The up sampler 240 may increase the resolution reduced by the down sampler 130 of the image encoding apparatus by performing up sampling on the residual block derived from the inverse transformer 230. If down sampling is not performed on the residual block in the image encoding apparatus, upsampling may not be performed on the residual block derived by the inverse transformer 230.

The motion compensator 250 may generate a prediction block based on the motion information provided by the entropy decoder 210 and previously decoded blocks and / or picture information provided by the reference picture buffer 270. The reconstruction block may be generated by adding the residual block derived from the inverse transform unit 230 and / or the residual block provided by the up sampler 240 to the prediction block. When a reconstruction block is generated based on the up-sampled residual block, the filter unit 260 may perform filtering according to the present invention on the reconstructed block. Specific embodiments of the filtering will be described later.

The reference picture buffer 270 may store the generated reconstructed picture (or reconstructed block) and / or reconstructed reconstructed picture (or reconstructed block) on which the filtering has been performed, so that the reconstructed picture and / or the reconstructed picture may be used as a reference picture or a reference block. Alternatively, the restoration block may be provided as an output unit.

3 is a block diagram schematically illustrating an image encoding apparatus according to another embodiment of the present invention. Referring to FIG. 3, the apparatus for encoding an image includes a predictor 310, a down sampler 320, a transformer T and 330, a quantizer Q and 340, an entropy encoder 350, and an inverse quantizer IQ. 360, an inverse transform unit IT 370, an up sampler 380, a filter unit 385, and a reference picture buffer 390.

The image encoding apparatus according to the embodiment of FIG. 3 may be applied to both a texture image and a depth image. In addition, as in the embodiment of FIG. 1, the above-described resolution reduction update (RRU) method may be used in the embodiment of FIG. 3.

Meanwhile, in FIG. 1, an encoding method when inter prediction is performed is described, but the present invention is not limited thereto. The present invention can be applied in the same or similar manner when not only inter prediction but also intra prediction is performed.

Referring to FIG. 3, the predictor 310 may include an inter predictor that performs inter prediction and an intra predictor that performs intra prediction. The inter prediction unit may generate a prediction block by performing prediction based on information of at least one picture of a previous picture or a subsequent picture of the current picture. The detailed operation of the inter prediction unit is the same as in the embodiment of FIG. 1 and will be omitted. In addition, the intra predictor may generate a prediction block by performing prediction based on pixel information in the current picture. Prediction mode information and motion information used for prediction may be encoded by the entropy encoder 350 together with the residual value and transmitted to the decoder.

Operation of the remaining components except for the above-described configuration is the same as in FIG. 1, and thus descriptions thereof will be omitted.

4 is a block diagram schematically illustrating an image decoding apparatus according to another embodiment of the present invention. Referring to FIG. 4, the image decoding apparatus includes an entropy decoder 410, an inverse quantizer IQ 420, an inverse transformer IT 430, an up sampler 440, a predictor 450, and a filter 455. ) And a reference picture buffer 460.

The image decoding apparatus according to the embodiment of FIG. 4 may be applied to both the texture image and the depth image similarly to the image encoding apparatus of FIG. 3. In addition, as in the embodiment of FIG. 2, the above-described resolution reduction update (RRU) method may be used in the embodiment of FIG. 4.

Meanwhile, although the decoding method in the case where inter prediction is performed in FIG. 2 is described, the present invention is not limited thereto. The present invention can be applied in the same or similar manner when not only inter prediction but also intra prediction is performed.

Referring to FIG. 4, the predictor 450 may include an inter predictor that performs inter prediction and an intra predictor that performs intra prediction. The inter prediction unit may generate the prediction block based on the motion information provided by the entropy decoder 410 and previously decoded blocks and / or picture information provided by the reference picture buffer 460. In addition, the intra predictor may generate the predictive block by performing prediction based on the prediction block generation related information provided by the entropy decoder 410 and the pixel information in the current picture.

Operation of the remaining components except for the above-described configuration is the same as in FIG.

5 is a diagram illustrating an upsampling process and a filtering process according to the present invention.

5 is a diagram for describing an operation of a filter applied to an operation and a reconstruction block of the up-sampler illustrated in the embodiments of FIGS. 1 to 4. 5 shows the location of the sample to be generated or interpolated by the up sampling process and the down sampled low-resolution sample. In this case, the sample may have the same meaning as the pixel in some cases, and this distinction may be easily made by those skilled in the art. Hereinafter, the sample generated by the upsampling process is referred to as interpolation sample.

An interpolation sample and / or a reconstruction sample corresponding to the interpolation sample at any position within the current picture (eg, (m, n), where m and n are integers greater than or equal to 0) includes the location of the interpolation sample. Can be generated based on a low resolution sample within a window of a predetermined size and / or of a predetermined shape.

As described above with reference to FIGS. 1 and 2, the upsampler may perform upsampling on the input residual block. That is, interpolation samples at any location in the residual block (eg, (m, n), where m and n are integers greater than or equal to 0) can be generated by upsampling. In one example, the up sampler may perform a zero-padding process on the position of the sample to be interpolated. Here, the zero padding process may mean a process of filling a value of 0 at the position of the sample to be interpolated. Also, for example, the up sampler may generate interpolated samples by performing copy or simple linear filtering based on the low resolution samples in the window.

In addition, as described above with reference to FIGS. 1 and 2, the reconstruction block may be generated by adding a prediction block and a residual block on which upsampling is performed. At this time, the filter unit may derive the final reconstructed sample of the (m, n) position by performing filtering on the reconstructed sample of the (m, n) position in the generated reconstruction block based on the low resolution sample in the window. have. In this case, the filtering may correspond to, for example, low-pass filtering.

Referring to FIG. 5, it is assumed that a position of a sample to be currently interpolated is (m, n), and 510 of FIG. 5 is a sample of the (m, n) position. In this case, the upsampler may determine or set a window N (m, n) of a predetermined size including the position of (m, n) for generating an interpolation sample of the position (m, n). 520 of FIG. 5 shows the window N (m, n). Here, the window N (m, n) may be a 2 × 2 square window, and the window may include 2 × 2 low resolution samples. However, the size, shape and / or position of the window is not limited to the embodiment of FIG. 5 and may be determined or set differently according to implementation and / or needs.

In this case, the upsampler may generate an interpolated sample at the position (m, n) by performing upsampling on the residual block. In one example, the up-sampler may perform a zero padding procedure on the location of the sample to be interpolated. In addition, the image encoding / decoding apparatus may generate a reconstruction block by adding a prediction block and a residual block on which upsampling is performed. In this case, the filter unit performs filtering (eg, low pass filtering) on the reconstructed sample 510 at the (m, n) position in the generated reconstructed block based on the low resolution sample in the window 520. , the final reconstructed sample of position (m, n) can be derived.

Meanwhile, the above upsampling process and the filtering process may be performed based on upsampling ratio information and filter type information, respectively. In this case, the down sampling rate and the up sampling rate may be the same. Embodiments described later are described based on upsampling, but an upsampling rate value applied to upsampling may also be used as a downsampling rate value.

As described above, the up / down sampling ratio may be determined differently depending on the degree of monotony of the image, the degree of importance of the detail of the image, and / or the degree of frequency of the image. For example, if the video signal being encoded / decoded is a signal corresponding to a low frequency region characterized by a continuous series of monotonous values, rather than a high frequency region where detail is of importance, then using a high sampling rate is a rate-distortion optimization (rate -distortion optimization (RDO) may be advantageous in terms of. On the contrary, when the image signal to be encoded / decoded is a signal corresponding to a high frequency region where detail is important, it may be advantageous to use a low sampling rate in view of rate-distortion optimization.

In addition, like the sampling rate, the filter type may be determined differently according to the characteristics of the image. For example, the filter unit may apply a filter that performs a weak smoothing on an edge region including an edge, and applies a strong smoothing on a non-edge region that does not include an edge. You can apply filters to perform. Application of this adaptive filter type can allow salient features such as edges in the video signal to be better maintained.

The image encoding apparatus may derive an optimal combination of upsampling (and / or downsampling) ratio and filter type in terms of rate-distortion optimization. Here, the combination of upsampling (and / or downsampling) ratio and filter type may be represented by a reduced-resolution update type (RRU type) and / or a reduced-resolution update mode. That is, one RRU type may indicate and / or include one upsampling (and / or downsampling) rate and one filter type. In this case, the upsampling process and the filtering process for one block may be performed based on one RRU type information and / or one RRU mode information. The above-described RRU type may be independently applied to each of the texture image and the depth image. Therefore, the filter type applied to the texture image and the filter type applied to the depth image may be different from each other, and the sampling rate applied to the texture image and the sampling rate applied to the depth image may also be different.

Table 1 below shows an embodiment of an RRU type according to the present invention. However, the embodiment of the RRU type is not limited to Table 1 described below, and a value, an upsampling rate value, and a filter type assigned to the RRU type may be determined differently from those of the embodiment of Table 1 as necessary.

TABLE 1

Referring to Table 1, one RRU type may indicate one upsampling rate and one filter type. For example, when the RRU type is 0, since the upsampling ratio is 1, upsampling may not be performed and filtering may not be performed. As another example, when the RRU type is 1, since the upsampling ratio is 2, twice the upsampling may be performed and the filter A may be applied. Specific examples of filter A and filter B shown in Table 1 will be described later.

When the RRU type information is used, the image encoding apparatus may determine an optimal RRU type and / or an RRU mode in view of rate-distortion optimization. In this case, the image encoding apparatus performs resolution reduction update based on the determined RRU type and / or RRU mode, thereby causing local characteristics of the image such as salient features (eg, edge regions), high frequency texture, etc. of the image. Can keep them better. That is, when the RRU type information is used, since the combination of the sampling rate and the filter type may be adaptively determined according to the characteristics of the image, the adaptive resolution reduction update may be enabled.

Hereinafter, an embodiment of a rate-distortion optimization based RRU type determination method is described. In the following embodiment, it is assumed that the cost function for determining the RRU type is applied based on macro block units. However, the present invention is not limited thereto, and the cost function for determining the RRU type may be applied based on a CU, a PU, and / or a TU.

When inter prediction is applied, a rate-distortion optimization based RRU type determination method may be represented by Equation 1 according to an embodiment.

[Equation 1]

Where J represents a cost function and λ represents a Lagrange multiplier. In addition, rru_type may indicate an RRU type, and mb_type may indicate a general macroblock type such as inter_16x16, skip, and the like. On the other hand, when the cost function is calculated in units of CUs, PUs, or TUs instead of macroblocks, the mb_type may be replaced with cu_type, pu_type, or tu_type, respectively. Here, cu_type, pu_type, and tu_type may indicate the type of CU, the type of PU, and the type of TU, respectively.

And R_MVDenotes a bit rate for encoding a motion vector (MV) and / or a reference picture index (eg, ref_idx), where R_{mb_type}May represent a bit rate for encoding mb_type. Where R_{mb_type}Is based on the unit in which the cost function is calculated_{cu_type}, R_{pu_type}Or R_{tu_type}Can be replaced with R in this case_{cu_type}, R_{pu_type}And R_{tu_type}May denote a bit rate for encoding a CU, a PU, and a TU, respectively. R_{rru_type}Denotes a bit rate for encoding the up sampling rate and the filter type.

In Equation 1, D may be calculated as Equation 2 below.

[Equation 2]

Here, MB denotes a macroblock, X denotes a position of a sample existing in the input image, and X _p denotes a sample position in the reference image corresponding to a sample of the X position. On the other hand, when the cost function is calculated in units of CUs, PUs, or TUs instead of macroblocks, the MBs may be replaced with CUs, PUs, or TUs. MV represents a motion vector, where X _p may refer to a position of a prediction sample in a reference picture indicated by MV. In addition, I may represent a pixel value in the input image, and I _p may represent a predicted pixel value in the reference image.

The image encoding apparatus may find an upsampling ratio and a filter type that minimize the aforementioned cost function. In this case, the apparatus for encoding an image may determine an RRU type that minimizes a cost function as an optimal RRU type for the current block.

6 is a flowchart schematically showing an embodiment of a decoding method according to the present invention. The decoding process according to the embodiment of FIG. 6 may be applied to both the texture image and the depth image.

After the encoder determines the RRU type based on rate-distortion optimization, the encoder may encode and transmit information about the RRU type to the decoder. The decoder may receive and decode information about the RRU type from the encoder. In this case, the decoder may perform upsampling and filtering based on the information about the decoded RRU type.

In the embodiment of FIG. 6, it is assumed that the RRU type is defined as in Table 1 above. In addition, since the residual block and prediction block generation processes in the decoder have been described above with reference to FIG. 2, a description thereof will be omitted.

Referring to FIG. 6, when the RRU type is 0, the decoder may not perform a resolution reduction update process (S610). When the RRU type is 0, the upsampling ratio is 1, so upsampling may not be performed and filtering may not be applied according to the filter type information.

If the RRU type is 1 or 2, the decoder may perform upsampling on the residual block (S620). When the RRU type is 1 or 2, since the upsampling ratio is 2, the decoder may perform upsampling at a rate of twice the width and height directions of the residual block, respectively. The decoder may generate a reconstructed block by adding the residual block on which upsampling has been performed with the prediction block (S630).

When the reconstruction block is generated, the decoder may perform filtering on each reconstruction sample in the generated reconstruction block. In this case, when the RRU type is 1, since the filter type corresponds to A, the decoder may apply filter A to the reconstructed samples in the reconstruction block (S640). In addition, when the RRU type is 2, since the filter type corresponds to B, the decoder may apply filter B to the reconstructed samples in the reconstruction block (S650). Here, specific examples of the filter A and the filter B will be described later.

Even when the RRU type is 3 or 4, the decoder may perform upsampling on the residual block (S660). When the RRU type is 3 or 4, since the upsampling ratio is 4, the decoder can perform upsampling at a ratio of four times in each of the width and height directions of the residual block. The decoder may generate a reconstructed block by adding the residual block on which upsampling has been performed with the prediction block (S670).

When the reconstruction block is generated, the decoder may perform filtering on each reconstruction sample in the generated reconstruction block. In this case, when the RRU type is 3, since the filter type corresponds to A, the decoder may apply filter A to the reconstructed samples in the reconstruction block (S680). In addition, when the RRU type is 4, since the filter type corresponds to B, the decoder may apply filter B to the reconstructed samples in the reconstruction block (S690). Here, specific examples of the filter A and the filter B will be described later.

Meanwhile, although the above-described embodiment is described with reference to the decoder, it may be applied to the encoder in a similar manner. As described above with reference to FIG. 1, the encoder may perform the upsampling process and the filtering process as in the decoder, and thus the above-described upsampling method and the filtering method may be applied to the encoder in the same manner.

Hereinafter, embodiments of the filter type according to the present invention are described. In the embodiments of Table 1 and FIG. 6 described above, filter A and filter B are used, and embodiments described below are described with reference to filter A and filter B. FIG. However, the filter type according to the present invention is not limited thereto, and the order in which the filter types are allocated according to the RRU type and / or the number and type of filter types used may be determined differently according to implementation and / or needs. .

In addition, the above-described embodiments may be applied not only to the texture video but also to the depth video, and that the filter applied to the texture image and the filter applied to the depth image may be different from each other. Therefore, in the embodiments described below, embodiments of the filter applied to the texture image and the filter applied to the depth image are described independently.

Example 1: Filter for Textured Image

1 to 6, the filter unit may apply a bi-laterally weighted filter to the reconstructed image (or reconstructed block) corresponding to the texture image. The bidirectional weight filter may perform a weighted average based on pixel values in a window corresponding to the pixel to be filtered. The bidirectional weight filter applied to the reconstructed image (or reconstructed block) corresponding to the texture image may be represented by Equation 3 below.

[Equation 3]

Here, (m, n) may indicate the position of the pixel to be filtered and N (m, n) may indicate a window corresponding to the pixel to be filtered. Since the embodiment of the window has been described above, it will be omitted here. In addition, I (i, j) may represent the input pixel values of the filter, (i, j) may represent the position of the input pixel present in the window N (m, n). I '(m, n) may indicate an output value of the filter for the pixel to be filtered at the position (m, n).

In Equation 3, w (i, j) represents a weighting kernel, and the geometric distance between (m, n) and (i, j) and the photometric between the two points It may correspond to a monotonically decreasing function based on distance. Here, the geometric distance may mean a physical distance between two points, and the luminance distance may indicate a difference in intensity and / or a pixel value between pixels existing at two points. That is, the weight kernel of the filter applied to the texture image is based on the difference in intensity (and / or luma pixel value difference) between the two pixels as well as the geometric distance between the pixel to be filtered and the surrounding pixels existing in the window. Can be determined. In this case, the filter unit may apply a greater weight to a pixel similar to the pixel to be filtered. Equation 4 below shows an embodiment of the above-described weight kernel.

[Equation 4]

Here, I _{(i, j)} may represent a pixel input value at position (i, j), and I _{(m, n)} may represent a pixel input value at position (m, n). In addition, sigma _s is a spreading parameter indicating a smoothing degree in a spatial domain, and sigma _I is a spreading parameter indicating a smoothing degree in a pixel-intensity region. Can be.

The bidirectional weight filter described above may have a non-linear characteristic and may perform smoothing on the image while preserving the edge of the image.

In the above-described embodiment of Table 1 and FIG. 6, filter A and filter B may correspond to the bidirectional weight filter described above. In this case, filter A may be a filter applied to a textureless region (or block) and / or a region (or block) in which no edge exists, and filter B may be a high-textured region ( Or block) and / or a filter applied to the edge region (or block) in which the edge is present.

In this case, a bidirectional weighting filter that performs strong smoothing may be applied to the textureless region (or block) and / or the region (or block) where no edge exists. Thus, the spreading parameters sigma _s and sigma _I applied to filter A may be diffusion parameters with relatively large values. In addition, for a highly-textured region (or block) and / or an edge region (or block) in which an edge exists, a bidirectional weighting filter that performs weak-smoothing may be applied. Thus, the spreading parameters sigma _s and sigma _I applied to filter B may be diffusion parameters with relatively small values.

Example 2: Filter 1 for Depth Image

As described above, the filter unit may perform filtering on the reconstructed image corresponding to the depth image. At this time, the texture image and the depth image may be different from each other, but may have a similar tendency. Therefore, when the texture image (or picture or block) corresponding to the depth image (or picture or block) to be filtered is valid, the filter unit performs filtering on the reconstructed image corresponding to the depth image based on the pixel value of the decoded texture image. Can be done. At this time, for example, the filter unit may perform filtering on the filtering target pixel based on the pixel in the region corresponding to the window of the filtering target pixel among the pixels in the texture picture. That is, the filter applied to the reconstructed image corresponding to the depth image may perform filtering using information related to the texture image. Hereinafter, in the embodiments described below, pixels in the depth image are referred to as depth pixels, and pixels in the texture image are referred to as texture pixels.

On the other hand, since the depth image has a feature in which details are not important as compared to the texture image in the remaining regions except the edge region, sharpness of the edge portion is important in the depth image, and the remaining portion may not be important. . Therefore, a filter considering the characteristics of the depth image needs to be provided.

In the above-described embodiment of Table 1 and FIG. 6, the filter A and the filter B applied to the reconstructed image (or block) corresponding to the depth image may be a bidirectional weighting filter. In this case, the filter A may be a bidirectional weighted Winner-Take-All (WTA) filter, and the filter B may be a bidirectional weighted median filter.

First, in the bidirectional weighted WTA filter corresponding to filter A, the pixel value of the depth pixel having the highest weight score (and / or weight frequency) in the window corresponding to the pixel to be filtered is the filter output value (filtered pixel value). Can be determined. In this case, the bidirectional weighted WTA filter applied to the reconstructed image corresponding to the depth image may be represented by Equation 5 below.

[Equation 5]

Here, (m, n) may represent the position of the depth pixel to be filtered, and N (m, n) may represent a window corresponding to the depth pixel to be filtered. Since the embodiment of the window has been described above, it will be omitted here. In addition, D (i, j) may represent input depth pixel values of the filter, and (i, j) may represent a position of an input pixel existing in the window N (m, n). D ′ (m, n) may indicate an output value of the filter for the depth pixel to be filtered at the position (m, n).

In equation (5)

Denotes a weighting kernel and may correspond to a monotonic reduction function based on the geometric distance between (m, n) and (i, j) and the luminance distance between the two points. Here, the geometric distance may mean a physical distance between two points, and the luminance distance _may correspond to an intensity difference and / or texture between the texture pixels I _{(m, n)} and I _{(i, j)} existing at the same position as the two points. This may indicate a pixel value difference. That is, the weighted kernel of the filter applied to the depth image is not only the geometric distance between the depth pixel to be filtered and the neighboring pixels present in the window, but also the intensity difference (and / or) between the texture pixels present at the same positions as the two pixels, respectively. Luma pixel value difference). In this case, the filter unit may apply a greater weight to a pixel similar to the pixel to be filtered.

In addition, sigma _s is a spreading parameter indicating a smoothing degree in a spatial domain, and sigma _I is a spreading parameter indicating a smoothing degree in a pixel-intensity region. Can be.

Referring to Equation 5, the bidirectional weighted WTA filter may derive a weight score for each depth pixel in the window based on the texture pixel. In Equation 5 described above

May correspond to a weighting score. In this case, the filter unit may determine a depth pixel value having the highest weight score among the depth pixels in the window as an output value for the depth pixel to be filtered. That is, according to the bidirectional weighted WTA filter described above, the filter unit selects a value having the highest weight score (and / or weight frequency) among the pixels in the window corresponding to the depth pixel to be filtered, and the filtered pixel for the depth pixel to be filtered. Can be derived by value. Thus, in this case the filter part does not introduce or derive new depth values that do not exist prior to filtering, so that the object boundary in the depth image and / or the characteristics of the boundary can be better maintained.

Next, in the bidirectional weighted median filter corresponding to filter B, the depth pixel value corresponding to the median value based on the weight score (and / or weight frequency) within the window corresponding to the pixel to be filtered is the filter output value (filtering). Pixel value). In this case, the bidirectional weighted median filter applied to the reconstructed image corresponding to the depth image may be represented by Equation 6 below.

[Equation 6]

Here, (m, n) may represent the position of the depth pixel to be filtered, and N (m, n) may represent a window corresponding to the depth pixel to be filtered. Since the embodiment of the window has been described above, it will be omitted here. In addition, D (i, j) may represent input depth pixel values of the filter, and (i, j) may represent a position of an input pixel existing in the window N (m, n). D ′ (m, n) may indicate an output value of the filter for the depth pixel to be filtered at the position (m, n).

In Equation 6,

Denotes a weighting kernel and may correspond to a monotonic reduction function based on the geometric distance between (m, n) and (i, j) and the luminance distance between the two points. Here, the geometric distance may mean a physical distance between two points, and the luminance distance may be an intensity difference between the 'texture pixels' (I _{(m, n)} and I _{(i, j)} ) existing at the same position as the two points and / or Alternatively, texture pixel values may be different from each other. That is, the weighted kernel of the filter applied to the depth image is not only the geometric distance between the depth pixel to be filtered and the surrounding pixels present in the window, but also the intensity difference (and / or) between the texture pixels present at the same positions as the two pixels, respectively. Luma pixel value difference). In this case, the filter unit may apply a greater weight to a pixel similar to the pixel to be filtered.

In addition, sigma _s is a spreading parameter indicating a smoothing degree in a spatial domain, and sigma _I is a spreading parameter indicating a smoothing degree in a pixel-intensity region. Can be.

In addition, 'Scale' is a constant, and represents a value that is multiplied to allow C _{k to} have a value within a predetermined appropriate range (for example, 1 to 100). Meanwhile, in Equation 6

Can represent the largest integer value less than or equal to x.

'median' means median,

It may be repeated D _i C _i is one, that means that D _i is listed as C _i dog. In Equation 6, D _i may have a larger value as i is larger, and D _tot may represent the largest depth pixel value in the window.

Referring to Equation 6, the bidirectional weighted median filter may derive a weight score for each depth pixel in the window based on the texture pixel. In Equation 6, the weight score may correspond to C _k . In this case, the filter unit may find a depth pixel corresponding to an intermediate value among depth pixels in the window based on the derived weight score. The pixel value of the depth pixel corresponding to the derived intermediate value may be determined as an output value for the pixel to be filtered. That is, according to the bi-directional weighted median filter described above, the filter unit may derive a median value based on a weight score (and / or weighted frequency) among pixels in a window corresponding to the depth pixel to be filtered, and the derived intermediate The value may be determined as a filter output value (filtered pixel value) for the depth pixel to be filtered. Thus, in this case the filter part does not introduce or derive a new depth value that does not exist prior to filtering, so that the object boundary in the depth image and / or the characteristics of the boundary can be better maintained.

Example 3: Filter 2 for Depth Image

As described above, the filter unit may perform filtering on the reconstructed image corresponding to the depth image. In this case, in the case of 'Example 2', the filter unit may perform filtering based on pixel values of the decoded texture image. However, when filtering is performed on the filtering target depth image (or the picture or the block), the texture image (or the picture or the block) corresponding to the filtering depth image (or the picture or the block) may not be valid.

For example, when the decoding process of the filtering depth image is performed before the corresponding texture image, the filter unit may not use the texture image related information when filtering the filtering depth image. As another example, when the decoding process of the filtering target depth image and the texture image corresponding thereto is performed in parallel, the decoding process of the texture image may not be completed when the filtering of the filtering target image is performed. Even in this case, the filter unit may not use the texture image related information when performing filtering of the filtering depth image.

Accordingly, as described above, when the texture image corresponding to the filtering depth image is not valid, the filter unit may perform filtering based on pixels in the filtering depth image, not the texture image.

On the other hand, since the depth image has a feature in which details are not important as compared to the texture image in the remaining regions except the edge region, sharpness of the edge portion is important in the depth image, and the remaining portion may not be important. . Therefore, a filter considering the characteristics of the depth image needs to be provided.

1 to 6, the filter unit may apply a bidirectional weighted median filter to the reconstructed image (or block) corresponding to the depth image. When the bidirectional weighted median filter is used, the filter unit may derive a median value based on a weight score (and / or weight frequency) among pixels in a window corresponding to the depth pixel to be filtered, and the derived median value Can be determined as the filter output value (filtered pixel value). In this case, the bidirectional weighted median filter applied to the reconstructed image corresponding to the depth image may be represented by Equation 7 below.

[Equation 7]

Here, (m, n) may represent the position of the depth pixel to be filtered, and N (m, n) may represent a window corresponding to the depth pixel to be filtered. Since the embodiment of the window has been described above, it will be omitted here. In addition, D (i, j) may represent input depth pixel values of the filter, and (i, j) may represent a position of an input pixel existing in the window N (m, n). D ′ (m, n) may indicate an output value of the filter for the depth pixel to be filtered at the position (m, n).

In Equation 7,

Denotes a weighting kernel and may correspond to a monotonic reduction function based on the geometric distance between (m, n) and (i, j) and the luminance distance between the two points. Here, the geometric distance may mean a physical distance between the two points, and the luminosity distance may be an intensity difference between the 'depth pixels' D _{(m, n)} and D _{(i, j)} present at the two points and / or Depth pixel values may represent differences. That is, the weight kernel of the filter applied to the depth image may be determined based on the difference in intensity (and / or luma pixel value difference) between the two pixels as well as the geometric distance between the depth pixel to be filtered and the surrounding pixels existing in the window. have. In this case, the filter unit may apply a greater weight to a pixel similar to the pixel to be filtered.

In addition, sigma _s is a spreading parameter indicating a smoothing degree in a spatial domain, and sigma _D is a spreading parameter indicating a smoothing degree in a pixel-intensity region. Can be.

In addition, 'Scale' is a constant, and represents a value that is multiplied to allow C _{k to} have a value within a predetermined appropriate range (for example, 1 to 100). On the other hand, in equation (7)

Can represent the largest integer value less than or equal to x.

'median' means median,

It may be repeated D _i C _i is one, that means that D _i is listed as C _i dog. In Equation 7, D _i may have a larger value as i is larger, and D _tot may represent the largest depth pixel value in the window.

Referring to Equation 7, the bi-directional weighted median filter may derive a weight score for each depth pixel in the window based on the depth pixel in the window. In Equation 7, the weight score may correspond to C _k . In this case, the filter unit may find a depth pixel corresponding to an intermediate value among depth pixels in the window based on the derived weight score. The pixel value of the depth pixel corresponding to the derived intermediate value may be determined as an output value for the pixel to be filtered. That is, according to the bi-directional weighted median filter described above, the filter unit may derive a median value based on a weight score (and / or weighted frequency) among pixels in a window corresponding to the depth pixel to be filtered, and the derived intermediate The value may be determined as a filter output value (filtered pixel value) for the depth pixel to be filtered. Thus, in this case the filter part does not introduce or derive a new depth value that does not exist prior to filtering, so that the object boundary in the depth image and / or the characteristics of the boundary can be better maintained.

Meanwhile, in the embodiments of Table 1 and FIG. 6 described above, filter A and filter B may correspond to the bidirectional weighted median filter described above. In this case, filter A may be a filter applied to a non-edge region (or block) in which no edge exists, and filter B may be a filter applied to an edge region (or block) in which the edge exists. Can be.

In this case, for a non-edge region (or block) where no edge exists, a bidirectional weighted median filter that performs strong smoothing may be applied. Therefore, the spreading parameters sigma _s and sigma _D applied to filter A may be spreading parameters having a relatively large value. Also, for an edge region (or block) in which an edge exists, a bidirectional weighted median filter that performs weak smoothing may be applied. Therefore, the spreading parameters sigma _s and sigma _D applied to the filter B may be spreading parameters having a relatively small value.

7 is a flowchart schematically showing another embodiment of a decoding method according to the present invention.

The decoding process according to the embodiment of FIG. 7 may be applied to both the texture image and the depth image. In addition, the above-described resolution reduction update (RRU) method may be applied to the embodiment of FIG. 7.

Referring to FIG. 7, the decoder may generate a prediction block (S710). When inter prediction is applied, the decoder may generate a predictive block based on previously decoded blocks and / or picture information provided in the reference picture buffer. When intra prediction is applied, the decoder may generate a prediction block by performing prediction based on pixel information in the current picture. In addition, the decoder may generate a residual block (S720). Since a specific embodiment of the residual block generation process has been described above with reference to FIG. 2, it will be omitted here.

Referring to FIG. 7 again, the decoder may receive and decode RRU type information from the encoder (S730).

As described above, the encoder may determine the RRU type based on the rate-distortion optimization and then encode and transmit the RRU type information to the decoder. The RRU type information may include sampling rate information and / or filter type information. Here, since the filter is a filter applied to a reconstruction block generated based on the residual block on which upsampling is performed, it may also be called an RRU filter or an upsampling filter.

In one embodiment, the encoder may encode and transmit sampling rate information and / or RRU filter information for each residual block to the decoder. Here, for example, the sampling rate information may be represented by the syntax element sampling_ratio, and the RRU filter type information may be represented by the syntax element rru_filter_type. In this case, the sampling rate information and / or the RRU filter information may be included in a residual block level and / or a residual block header (eg, residual ()) and transmitted from the encoder to the decoder through a bitstream. In this case, the decoder may receive and decode the sampling rate information and / or the RRU filter information. The sampling rate information and the RRU filter information transmitted to the decoder may be defined by syntax elements as included in Table 2 according to an embodiment.

TABLE 2

Here, sampling_ratio may represent up-sampling ratio information. In this case, sampling_ratio may represent an upsampling rate value itself, or may represent a value used to derive an actual upsampling ratio. In one example, the decoder may derive the actual upsampling ratio by the following Equation (8).

[Equation 8]

Up-sampling ratio = sampling_ratio << 1

Here, the up-sampling ratio may indicate an actual upsampling ratio applied to the upsampling process. For example, when the value assigned to sampling_ratio is 2, the actual upsampling rate value applied to the upsampling process may be 4.

In addition, rru_filter_type may indicate an upsampling filter type. That is, rru_filter_type may indicate the type of upsampling filter used for filtering the reconstructed block generated based on the residual block on which upsampling is performed.

In another embodiment, the encoder may transmit the RRU type information to the decoder through one syntax element indicating the RRU type, unlike in the above-described embodiment. In one example, the RRU type information may be represented by a syntax element rru_type .

In this case, the encoder and the decoder may store the same look-up table indicating the relationship between the RRU type, the upsampling rate, and the RRU filter type. As described above, the encoder can determine the optimal upsampling rate and the optimal RRU filter type based on the rate-distortion optimization. In this case, the encoder may determine the RRU type corresponding to the determined up-sampling rate and the RRU filter type based on the lookup table, and may transmit information about the determined RRU type to the decoder through one syntax element rru_type. The decoder may receive and decode the RRU type information transmitted from the encoder. In this case, the decoder may determine the RRU type for the current block based on the decoded RRU type information, and may derive an upsampling rate and an RRU filter type based on the determined RRU type and the stored lookup table.

Referring to FIG. 7 again, the decoder may perform upsampling on the generated residual block based on the decoded RRU type information (S740). Since the RRU type information includes upsampling rate information, the decoder may perform upsampling on the residual block according to the upsampling rate indicated by the upsampling rate information. However, when the upsampling ratio for the residual block is 1, upsampling may not be performed for the residual block.

In addition, the decoder may generate a reconstruction block based on the prediction block and the residual block on which upsampling is performed (S750). In this case, if upsampling is not performed on the residual block, the residual block used to generate the reconstructed block may be a residual block on which upsampling is not performed.

When the reconstruction block is generated, the decoder may perform filtering on each reconstruction sample in the generated reconstruction block (S760). Since the above-described RRU type information includes filter type information, the decoder may perform filtering based on the filter type information. Since specific embodiments of the filter applied to the texture image and the filter applied to the depth image have been described above, a description thereof will be omitted.

According to the embodiments described above, the present invention may perform resolution reduction update based on the RRU type and / or RRU mode, so that characteristics of an image such as an edge may be better maintained. That is, when the RRU type information is used, the sampling rate or the combination of the filter types may be changed according to the characteristics of the image, so that adaptive resolution reduction updating may be possible.

On the other hand, as described above, the depth image has a feature that the detail is less important than the texture image in most regions except the edge region. Thus, the upscaling method and / or resolution reduction update method described above will be particularly efficient when applied to depth images in terms of rate-distortion optimization.

In the above-described embodiment, the methods are described based on a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and any steps may occur in a different order or simultaneously from other steps as described above. have. In addition, those skilled in the art will appreciate that the steps shown in the flowcharts are not exclusive and that other steps may be included or one or more steps in the flowcharts may be deleted without affecting the scope of the present invention.

The above-described embodiments include examples of various aspects. While not all possible combinations may be described to represent the various aspects, one of ordinary skill in the art would recognize that other combinations are possible. Accordingly, the invention is intended to embrace all other replacements, modifications and variations that fall within the scope of the following claims.

Claims (20)

1. Receiving a bitstream;
   Generating a prediction block and a residual block for a current picture based on the bitstream;
   Performing up-sampling on the residual block based on RRU (Rrduced-Resolution Update) type information derived from the bitstream;
   Generating a reconstruction block based on the prediction block and the residual block on which the upsampling is performed; And
   Performing filtering on the reconstruction block based on the RRU type information,
   The RRU type information includes sampling ratio information and filter type information.
   In the upsampling step, upsampling is performed based on a sampling rate indicated by the sampling rate information.
   And performing the filtering in the filtering step based on a filter indicated by the filter type information.
2. The method of claim 1,
   The sampling rate information is,
   The lower the frequency of the region corresponding to the residual block in the current block indicates a higher sampling rate.
3. The method of claim 1,
   The filtering step may include:
   Determining a window of a predetermined size and shape for a pixel to be filtered in the residual block; And
   And performing the filtering on the filtering target pixel based on the pixel in the window.
4. The method of claim 3, wherein
   If the current picture is a picture in a texture video,
   And the filter indicated by the filter type information is a bi-laterally weighted filter that performs a weighted average based on pixels in the window.
5. The method of claim 4, wherein
   The bidirectional weight filter,
   When the filtering target pixel is a pixel in an edge region including an edge, weak smoothing is performed.
   And if the filtering target pixel is a pixel in a non-edge region that does not include an edge, strong smoothing.
6. The method of claim 3, wherein
   When the current picture is a depth picture in a depth video, and the texture picture corresponding to the current picture is available in a texture video corresponding to the depth video,
   In the filtering step,
   And filtering based on a texture pixel in an area corresponding to the window in the texture picture.
7. The method of claim 6,
   The filter indicated by the filter type information is a bi-laterally weighted Winner_Take_All filter,
   The bidirectional weighted WTA filter,
   A weighting score is derived for each depth pixel in the window based on the texture pixel, and a pixel value of a depth pixel having the highest weight score among the depth pixels is determined as an output value for the filtering target pixel. and,
   And the depth pixel is a pixel in the depth picture.
8. The method of claim 6,
   The filter indicated by the filter type information is a bi-laterally weighted median filter,
   The bidirectional weighted median filter,
   Deriving a weighting score for each depth pixel in the window based on the texture pixel, Deriving a middle value among the depth pixels based on the weighting score, Depth pixel corresponding to the derived intermediate value Determine a pixel value of as an output value for the pixel to be filtered,
   And the depth pixel is a pixel in the depth picture.
9. The method of claim 3, wherein
   If the current picture is a depth picture in a depth image, and the texture picture corresponding to the current picture is not valid in the texture image corresponding to the depth image,
   The filter indicated by the filter type information is a bidirectional weighted median filter,
   The bidirectional weighted median filter,
   A weighting score is derived for each of the depth pixels based on the depth pixel in the window, a median value is derived from the depth pixels based on the weight score, and a depth corresponding to the derived median value. Determine a pixel value of the pixel as an output value for the pixel to be filtered,
   And the depth pixel is a pixel in the depth picture.
10. The method of claim 9,
    The bidirectional weighted median filter,
    When the filtering target pixel is a pixel in an edge region including an edge, weak smoothing is performed.
    And if the filtering target pixel is a pixel in a non-edge region that does not include an edge, strong smoothing.
11. A receiver for receiving a bitstream;
    A prediction block and residual block generation unit generating a prediction block and a residual block for a current picture based on the bitstream;
    An up sampler for performing up-sampling on the residual block based on RRU (Rrduced-Resolution Update) type information derived from the bitstream;
    A reconstruction block generation unit configured to generate a reconstruction block based on the prediction block and the residual block on which the upsampling is performed; And
    On the basis of the RRU type information, including a filter for performing filtering on the recovery block,
    The RRU type information includes sampling ratio information and filter type information.
    The upsampler performs upsampling based on the sampling rate indicated by the sampling rate information.
    And the filter unit performs filtering based on a filter indicated by the filter type information.
12. The method of claim 11,
    The sampling rate information is,
    The lower the frequency of the region corresponding to the residual block in the current block indicates a higher sampling rate.
13. The method of claim 11,
    The filter unit,
    And determining a window having a predetermined size and shape with respect to the filtering target pixel in the residual block, and performing filtering on the filtering target pixel based on the pixel in the window.
14. The method of claim 13,
    If the current picture is a picture in a texture video,
    And a filter indicated by the filter type information is a bi-laterally weighted filter that performs a weighted average based on pixels in the window.
15. The method of claim 14,
    The bidirectional weight filter,
    When the filtering target pixel is a pixel in an edge region including an edge, weak smoothing is performed.
    And if the filtering target pixel is a pixel in a non-edge region that does not include an edge, strong smoothing.
16. The method of claim 13,
    When the current picture is a depth picture in a depth video, and the texture picture corresponding to the current picture is available in a texture video corresponding to the depth video,
    The filter unit,
    And filtering based on a texture pixel in an area corresponding to the window in the texture picture.
17. The method of claim 16,
    The filter indicated by the filter type information is a bi-laterally weighted Winner_Take_All filter,
    The bidirectional weighted WTA filter,
    A weighting score is derived for each depth pixel in the window based on the texture pixel, and the pixel value of the depth pixel having the highest weight score among the depth pixels is determined as an output value for the filtering target pixel. and,
    And the depth pixel is a pixel in the depth picture.
18. The method of claim 16,
    The filter indicated by the filter type information is a bi-laterally weighted median filter,
    The bidirectional weighted median filter,
    Deriving a weighting score for each depth pixel in the window based on the texture pixel, Deriving a middle value among the depth pixels based on the weighting score, Depth pixel corresponding to the derived intermediate value Determine a pixel value of as an output value for the pixel to be filtered,
    And the depth pixel is a pixel in the depth picture.
19. The method of claim 13,
    If the current picture is a depth picture in a depth image, and the texture picture corresponding to the current picture is not valid in the texture image corresponding to the depth image,
    The filter indicated by the filter type information is a bidirectional weighted median filter,
    The bidirectional weighted median filter,
    A weighting score is derived for each of the depth pixels based on the depth pixel in the window, a median value is derived from the depth pixels based on the weight score, and a depth corresponding to the derived median value. Determine a pixel value of the pixel as an output value for the pixel to be filtered,
    And the depth pixel is a pixel in the depth picture.
20. The method of claim 19,
    The bidirectional weighted median filter,
    When the filtering target pixel is a pixel in an edge region including an edge, weak smoothing is performed.
    And if the filtering target pixel is a pixel in a non-edge region that does not include an edge, strong smoothing.

Images