KR20140046371A - Method and apparatus of determining a filtering strength of deblocking filtering for depth image - Google Patents

Abstract

Provided are a method and an apparatus for determining a boundary strength of a deblocking filter for a depth image, by which the boundary strength of the deblocking filter for a boundary between macro blocks of the depth image is set based on information about at least one mode selected from a plurality of coding/decoding modes. [Reference numerals] (710) Determine at least one among a plurality of conding/decoding modes for a depth image; (720) Obtain information about at least one mode; (730) Set a filtering strength of a deblocking filter based on information about at least one mode; (740) Determine whether the depth image is deblocking-filtered; (750) Perform deblocking filtering; (AA) Start; (BB) End

Description

METHOD AND APPARATUS OF DETERMINING A FILTERING STRENGTH OF DEBLOCKING FILTERING FOR DEPTH IMAGE}

The following embodiments are directed to a method and apparatus for determining the filtering strength of a deblocking filter for a depth image.

The 3D image is composed of a texture image and a depth image corresponding to 3D data. Each image may be encoded independently or dependently on each other. In general, the depth image may be used as 3D data for generating a virtual image in a 3D image system. Therefore, although the depth information is expressed in a general image format, the aspect of the characteristics and the importance of the image is different from the general color image. In particular, when compressing an image, the characteristics of the depth image should be further considered.

In general, compression techniques for color images have been developed to compress in consideration of human cognitive visual characteristics. However, in the case of depth image, it is necessary to improve compression efficiency while preserving more important information in depth image than compression method based on visual characteristics.

According to an embodiment, a method of determining a filtering strength of a deblocking filter for a depth image may include determining at least one of a plurality of encoding / decoding modes for a depth image; Acquiring information about the determined at least one mode; And setting the filtering strength of the deblocking filter on the boundary between the macroblocks of the depth image based on the information on the at least one mode.

The method may further include determining whether to perform deblocking filtering on the depth image based on at least two thresholds of the difference between the first pixel and the second pixel which are consecutive to each other in the macro blocks of the adjacent depth image. have.

The at least two thresholds may be used to distinguish whether the boundary between the macroblocks is a boundary region in the original image or is caused by discontinuity between pixels due to blocking phenomenon.

Converting a difference signal in units of macro blocks generated based on the information on the at least one mode; And determining the at least two thresholds based on the quantization coefficients of the quantized transformed differential signal.

The plurality of encoding / decoding modes may include a first mode that does not perform deblocking filtering on a boundary between the macroblocks and a second mode that performs the deblocking based on the filtering strength.

The setting of the filtering strength may include inter-encoding / decoding of an intra-encoded / decoded block among macroblocks disposed on both sides of the macroblocks based on the information on the at least one mode. The method may further include setting the same filtering strength for the block.

The method may further include differently setting the filtering intensity in a horizontal direction or a vertical direction according to the arrangement of the camera capturing the depth image.

The method may further include setting the filtering strength based on a direction of an intra coded / decoded macroblock among the macroblocks.

The method may further include determining the filtering strength in consideration of the slice type of the depth image.

If there is no residual signal in the macro blocks positioned on both sides of the boundary between the macro blocks, and the directionality of consecutive intra coded / decoded macro blocks is the same, horizontal deblocking based on the directionality The method may further include setting a filtering strength for the filter or the vertical deblocking filter.

The determining of the at least one encoding mode may include determining at least one of a plurality of encoding / decoding modes for the depth image based on a bit rate-distortion optimization (RDO) model. It may include a step.

According to an embodiment, an apparatus for determining a filtering strength of a deblocking filter for a depth image may include: a mode determiner configured to determine at least one of a plurality of encoding / decoding modes for the depth image; An acquisition unit for obtaining information about the determined at least one mode; And a setting unit configured to set the filtering strength of the deblocking filter for the boundary between the macroblocks of the depth image based on the information on the at least one mode.

The apparatus may further include a determining unit configured to determine whether to perform deblocking filtering on the depth image based on at least two thresholds of the difference between the first pixel and the second pixel which are continuous to each other in the macro blocks of the adjacent depth image. have.

A conversion unit for converting a difference signal in units of macro blocks generated based on the information on the at least one mode; And a threshold determiner configured to determine the at least two thresholds based on the quantized coefficients of the quantized transformed differential signal.

The plurality of encoding / decoding modes may include a first mode that does not perform deblocking filtering on a boundary between the macroblocks and a second mode that performs the deblocking based on the filtering strength.

The setting unit may perform filtering strengths on intra-encoded / decoded blocks and inter-encoded / decoded blocks among macroblocks disposed on both sides of the macroblocks based on the information on the at least one mode. Can be set equally.

The setting unit may set the filtering intensity differently in the horizontal direction or the vertical direction according to the arrangement of the camera capturing the depth image.

The setting unit may set the filtering strength based on a direction of an intra coded / decoded macro block among the macro blocks.

If the setting unit has no residual signal in the macro blocks positioned on both sides of the boundary between the macro blocks, and the directivity of consecutive intra coded / decoded macro blocks is the same, Filtering strength may be set for the horizontal deblocking filter or the vertical deblocking filter.

1 is a diagram illustrating an encoding structure of a multiview image.
2 is a diagram illustrating a configuration of a depth image encoding apparatus to which a method for determining a filtering strength of a deblocking filter for a depth image is applied, according to an exemplary embodiment.
3 is a diagram illustrating a configuration of a depth image decoding apparatus to which a method for determining a filtering strength of a deblocking filter for a depth image is applied according to an exemplary embodiment.
4 is a diagram illustrating a region referred to when predicting directionality for the intra prediction mode.
FIG. 5 illustrates a boundary of a macroblock in which a method of determining a filtering strength of a deblocking filter for a depth image is performed.
6 is a flowchart illustrating a process of setting a filtering strength for deblocking a color image in the image encoding apparatus.
7 is a flowchart illustrating a method of determining a filtering strength of a deblocking filter for a depth image, according to an exemplary embodiment.
8 is a flowchart illustrating a method of determining a filtering strength of a deblocking filter for a depth image, according to another exemplary embodiment.
9 is a block diagram of an apparatus for determining a filtering strength of a deblocking filter for a depth image, according to an exemplary embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

The following embodiments relate to a method and an apparatus for determining a filtering strength of a deblocking filter for a depth image. Before the detailed description, a coding structure of a multiview image and a filtering strength of the deblocking filter for a depth image are described. First, an encoding / decoding apparatus to which the method of determining is applied will be described.

1 is a diagram illustrating an encoding structure of a multiview image.

Referring to FIG. 1, a depth image may be configured as a coding structure of a multi-view image. A coding structure of a multiview image generally includes an anchor frame that encodes using only a reference between adjacent viewpoints and a non-anchor frame that uses both a reference in the time direction and a view in the view direction. Can be. An anchor frame is required for random access and is encoded using only inter-view prediction, and thus shows the difference between the compression of a single view image and the largest characteristic.

The encoding structure of the multi-view image of FIG. 1 is a multi-view video coding (MVC) that encodes a GOP (Group of Picture) as '8' when images of three viewpoints (Left View, Center View, Right View) are input. The structure is shown. When compressing a multiview image, the concept of hierarchical B picture is applied to the temporal axis and the view axis basically, thereby reducing a lot of redundancy between the images.

When the three view images are compressed in the MVC structure of FIG. 1, the left image and the center image may be sequentially compressed after the left image is compressed. In this case, the left image may be encoded while removing a temporal redundancy by searching for a similar region from previous images by performing motion estimation as in the case of compressing a 2D image. However, when a right image is compressed, an already compressed left image may be used as a reference image. Therefore, the right image may be encoded while removing not only temporal redundancy through motion search but also view redundancy between view points through disparity estimation. In the case of the center image, since both the left and right images can be used as reference images, the redundancy between viewpoints can be eliminated by performing a shift search in both directions.

In FIG. 1, a view that is encoded while predicting a view that is encoded without a prediction from another view, such as a left image, is unidirectionally predicted from an image of another view such as an I-View (Intra View) or a right (Pight) view, and the P-View ( In MVC, a viewpoint that is encoded by bidirectionally predicting from left and right views, such as a predictive view and a center image, is generally defined in MVC as a B-view (interpolative view).

Each picture may be divided into an I-picture, a P-picture, and an B-picture according to a coding type. In this case, the I-picture encodes the image itself without inter prediction, and the P-picture encodes the difference from the prediction value by performing the prediction between images using the reference image only in the forward direction, and the B-picture in both the forward and reverse directions. The inter prediction is encoded by using the reference image.

When encoding each macroblock, H.264 / AVC uses encoding modes of various macroblocks. Among them, H.264 / AVC selects and encodes an optimal encoding mode, and encodes information about the encoding mode type for each macroblock. Can be. In this case, in order to minimize the bits used to represent the information about the type of the encoding mode, macro blocks may be encoded with priority from modes that are frequently generated and have fewer bits to be encoded.

<Table 1>, <Table 2> and <Table 3> below show the types of macro blocks used in frame encoding of I picture, P picture and B picture in H.264 / AVC, respectively. to be.

[Table 1]

[Table 2]

[Table 3]

However, the encoding modes of the macroblocks shown in Tables 1, 2, and 3 are designed from the perspective of the color image and are not suitable for the depth image. In addition, the SKIP mode, which can reduce bits most efficiently among encoding modes, has a lower probability of generating a depth image than a color image. In the SKIP mode, encoding mode information and a residual component of a macroblock are not encoded.

The SKIP mode (hereinafter referred to as an 'inter SKIP mode') predicts the median motion vector value from the motion vectors of adjacent macroblocks. However, in the depth image, the inter SKIP mode is not suitable as the encoding mode of the macroblock for the depth image because the main characteristic is a flat characteristic without much texture.

In addition, the depth image is currently commonly provided in the form of 4: 2: 0 yuv. In this case, the values of the color difference signals are all set to 128 so that residuals of the color difference components do not occur. Therefore, in H.264 / AVC, an encoding mode having a residual for a chrominance component is also not suitable as an encoding mode of a macroblock for a depth image.

As a result, in H.264 / AVC encoding mode, a coding method based on a motion vector is not appropriate except for a region in which some textures exist in a depth image. In addition, the case of Bi-prediction (bidirectional prediction) used for B picture frames of H.264 / AVC is not suitable for depth images. Therefore, macroblock modes designed in terms of color images are largely inefficient in depth images. This unnecessarily large number of macro block modes increases the bits used for encoding macro block information, and the prediction method and priority between macro blocks for each mode are also inefficient in terms of color images.

In general, in encoding of a digital image, a quality degradation phenomenon, that is, a blocking artifact, may occur due to an error in a quantization process of a block unit. The blocking phenomenon may result in discontinuity at the boundary of adjacent blocks when the compression ratio is high when each block is independently processed, which may cause deterioration of image quality. Therefore, as a countermeasure for the blocking phenomenon, for example, the H.264 / AVC standard may use an in-loop deblocking filter during the encoding / decoding process.

The deblocking filter can mitigate a distortion phenomenon in which the block boundary portion generated in the pixel encoding process is distorted, and the decoded image generated through this can be used as a reference image in the next image. Hereinafter, a depth image encoding apparatus and a depth image decoding apparatus including an in-loop deblocking filter will be described with reference to FIGS. 2 and 3.

2 is a diagram illustrating a configuration of a depth image encoding apparatus to which a method for determining a filtering strength of a deblocking filter for a depth image is applied, according to an exemplary embodiment.

Referring to FIG. 2, the depth image encoding apparatus includes an intra mode predictor 210, an intra encoder 212, a motion predictor 214, an inter encoder 216, an encoding mode determiner 218, and a prediction block. The generator 220, the difference unit 222, the transform and quantization unit 224, the entropy encoder 226, the inverse transform and inverse quantizer 228, the combiner 230, and the deblocking filter unit 232. It may include.

The intra mode predictor 210 may predict the direction of the current macro block by using the encoded macro blocks of the neighboring picture. In this case, the predicted directionality is a vertical direction extending vertically from the upper boundary pixel of the current macro block to generate the prediction block, and horizontal extending horizontally from the left boundary pixel of the current macro block to generate the prediction block. The DC (Direct Current) direction in which the prediction block is generated by the direction and the average of the upper boundary pixel and the left boundary pixel of the current macro block, and the plane that generates the prediction block by considering the upper boundary pixel and the left boundary pixel of the current macro block together ( Can be one of the plane directions.

The intra encoder 212 may intra encode the current macro block according to intra candidate encoding modes.

The intra candidate encoding mode may include an intra prediction mode for intra encoding using the predicted directionality. The intra prediction mode may include at least one of an intra skip mode, an intra direct mode, and an intra direction mode. The intra skip mode is a mode in which the predicted directional information and the residual signal are not transmitted to the decoding apparatus, the intra direct mode is a mode in which the predicted directional information is not transmitted to the decoding apparatus, and the intra direction mode transmits the residual signal to the decoding apparatus. The mode may not be.

In addition, the intra skip mode may be the highest priority or the next highest priority among the priorities of the respective encoding modes included in the intra candidate encoding modes and the inter candidate encoding modes.

The types of intra candidate encoding modes may be classified according to at least one of a kind of a size of a macro block, whether or not the directional information is encoded, and whether there is a residual signal for a brightness component of the macro block.

The motion predictor 214 may estimate the motion vector using the current macroblock to be encoded and the previously input macroblock.

The inter encoder 216 may inter encode the current macro block for each inter candidate encoding mode based on the motion vector estimated by the motion predictor 214.

In this case, the inter candidate encoding modes may be encoded by referring to one motion vector per macroblock. In addition, the types of the inter candidate encoding modes may be classified in consideration of the type of the size of the macro block and whether the residual signal is encoded. At this time, the macro block means the largest block to be encoded and may be larger than 16 × 16.

The encoding mode determiner 218 considers the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes, and has the highest rate-distortion optimization (RDO) cost among the encoding modes. The small encoding mode may be determined as the encoding mode of the current macroblock. Meanwhile, information about the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be preset or defined when encoding a depth image, and may be transmitted to the decoding apparatus.

The encoding mode determiner 218 may change the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to the frequency at which the encoding modes are finally determined.

The prediction block generator 220 may generate the prediction block by using the encoding mode determined by the encoding mode determiner 218.

The difference unit 222 may generate a difference signal (or residual signal) in units of macro blocks by subtracting the prediction block from the current macro block.

The transform and quantization unit 224 may transform the difference signal generated by the difference unit 222 using a transform scheme, and quantize the transformed differential signal to generate quantized transform coefficients. An example of the transformation method is Discrete cosine transform (DCT).

The entropy encoder 226 may generate a bit stream by entropy encoding encoding information such as quantized transform coefficients and motion vectors. The generated bit stream may be a compressed depth image.

The inverse transform and inverse quantizer 228 dequantizes the difference signal quantized by the transform and quantization unit 224 for use in prediction of a frame to be encoded next, inverse discrete cosine transform, and reconstructs the difference signal before encoding. Can be.

The combiner 230 may reconstruct the current macroblock before encoding by adding the reconstructed difference signal and the predictive block generated by the predictive block generator 220.

The deblocking filter unit 232 may remove the blocking phenomenon by filtering the reconstructed block or the reconstructed depth image. In this case, the deblocking filter unit 232 may set the filtering strength of the deblocking filter by using a method such as FIGS. 7 to 8 to be described later.

3 is a diagram illustrating a configuration of a depth image decoding apparatus.

Referring to FIG. 3, an apparatus for decoding a depth image may include an entropy decoder 310, an encoding mode restoring unit 312, a motion restoring unit 314, an inter decoding unit 316, an intra mode restoring unit 318, and an intra decoding unit. The unit 320 may include an inverse transform and inverse quantization unit 322, a coupling unit 324, and a deblocking filter unit 326.

The entropy decoder 310 may entropy decode the input compressed bit stream to extract encoding information such as a quantized coefficient value and a motion vector of the difference image.

The encoding mode reconstructor 312 may identify the encoding mode of the encoded macroblock using priority information of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes.

The encoding mode reconstruction unit 312 may change the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes according to the frequency of the identified encoding modes. In addition, the information about the priority of each encoding mode included in the intra candidate encoding modes and the inter candidate encoding modes may be preset or defined when decoding the depth image and may be received by the encoding apparatus.

The motion reconstruction unit 314 may check the motion vector from the encoding information in the entropy decoder 310.

The inter decoder 316 may generate a prediction block by inter decoding using the motion vector identified by the motion reconstructor 314.

The intra mode reconstructor 318 may predict the direction of the encoded macro block using the decoded neighboring macro blocks when the encoding mode of the encoded macro block is the intra prediction mode.

If the encoding mode of the encoded macroblock is one of intra candidate encoding modes, the intra mode reconstructor 318 may check directional information included in the received bit stream.

The intra decoder 320 may generate a prediction block by using the directional information predicted by the intra mode reconstructor 318 or the confirmed directional information.

The inverse transform and inverse quantizer 322 may inverse quantize the extracted quantization coefficient value in macroblock units to generate coefficient values of the differential signal corresponding to the inverse transform. The inverse transform and inverse quantization unit 322 may obtain a differential signal by performing discrete cosine transform (IDCT) on the coefficient values of the generated difference image. As a result, the compressed bit stream may be decoded and temporarily restored to the depth image.

The combiner 324 may reconstruct the depth image by adding the difference signal and the prediction block. The differential signal may be input from the inverse transform and inverse quantizer 322 to the combiner 324, and the prediction block may be input from the inter decoder 316 or the intra decoder 320 to the combiner 324.

The deblocking filter 326 may remove the blocking phenomenon by filtering the depth image reconstructed by the combiner 324. The deblocking filter 326 may set the filtering strength of the deblocking filter by using a method as illustrated in FIGS. 7 to 8 to be described later.

The intra mode prediction unit 210 of the encoding apparatus and the intra mode reconstruction unit 318 of the decoding apparatus may predict the directional prediction by the method shown in FIG. 4 below.

4 is a diagram illustrating a region referred to when predicting directionality for the intra prediction mode.

Referring to FIG. 4, the intra mode predictor 210 and the intra mode reconstructor 318 may include a boundary value of the upper block 420 adjacent to the current macro block 410 and a left block as shown in Equation 1 below. The complexity of the boundary values of 430 may be compared to follow a more complex boundary direction.

Here, SSE (Sum of Squared Errors) is one of the complexity measurement techniques, A_SSE is the complexity of the boundary value of the upper block, L_SSE is the complexity of the boundary value of the left block.

[Equation 1] is an embodiment for predicting the directionality of the current macroblock. In addition to this method, the directionality may also be predicted by using a previously encoded depth image of an adjacent view or using color information. In addition, the directionality can be predicted through analysis of mode information, motion information, or edge components of the neighboring block, not boundary pixels of the neighboring block. The equation for obtaining the directional information may use various differences such as a simple sum or a variance, not a sum of squares of differences.

Various methods other than Equation 1 may be used for directional prediction.

Hereinafter, a boundary of a macroblock in which a blocking phenomenon occurs will be described with reference to FIG. 5, and a general method of setting a filtering strength for deblocking in the deblocking filtering unit will be described with reference to FIG. 6.

FIG. 5 illustrates a boundary of a macroblock in which a method of determining a filtering strength of a deblocking filter for a depth image is performed.

Referring to FIG. 5, a deblocking filter is generally a one-dimensional filter, and may be performed in a horizontal direction and a vertical direction after encoding of a slice is completed. In the case of H.264, the deblocking filter may classify the blocks into p pixel groups and q pixel groups based on the boundary 510 of the macro block as shown in FIG. .

First, the deblocking filter may determine the boundary strength (BS) according to the characteristics of the macroblock to which the p pixel group and the q pixel group belong, as shown in FIG. 6. Here, the characteristics of the macro block include whether the macro block is intra coded or inter coded, whether the macro block is an edge, whether there is a residual component in the macro block, and the motion Conditions such as whether there is a vector may be considered.

The filtering strength of the deblocking filter on the boundary between the macroblocks, that is, the boundary strength, may have a value from 0 to 4 according to this condition. Since the value of the boundary strength is large, the blocking phenomenon is large. If the value of the boundary strength is large, filtering may be performed more strongly when the deblocking is performed later. On the other hand, the boundary strength of 0 means that the blocking phenomenon between the two macro blocks is insufficient, so that the deblocking filter does not need to be performed.

After the boundary strength is determined, the deblocking filter may determine whether to perform deblocking filtering once again in consideration of pixel values of the p pixel group and the q pixel group participating in the deblocking.

For example, in the case of H.264, deblocking filtering may be performed when the boundary strength is greater than zero and satisfies Equation 2 below.

Here, α and β are thresholds for the difference between two consecutive pixels, and the boundary 510 between the p pixel group and the q pixel group is a blocking phenomenon in which the discontinuity between successive pixels is generated or the boundary in the original image. Can be used to distinguish between areas.

α and β vary in size according to the quantization parameter QP. For example, α and β may have a larger value as the quantization parameter QP increases. This means that the larger the quantization parameter QP, the larger the threshold for the boundary 210 of the macroblock, so that more deblocking filtering should be performed.

However, in this case, since the deblocking filter is designed in consideration of the characteristics of the color image and the human visual characteristics, the deblocking filter does not show great efficiency in encoding the depth image, and in some cases, may degrade performance.

6 is a flowchart illustrating a process of setting a filtering strength for deblocking a color image in the image encoding apparatus.

Referring to FIG. 6, the image encoding apparatus determines whether there is an intra-encoded macroblock among the two blocks based on the boundary of the macroblock (610). In this case, the intra encoding includes an intra prediction mode, and the intra prediction mode includes at least one of an intra skip mode, an intra direct mode, and an inter direction mode.

As a result of checking 610, if any one of both blocks is intra coded based on the boundary 510, the image encoding apparatus may check whether the boundary between the two blocks is an edge of the macro block ( 620).

As a result of checking 620, if the boundary is not an edge of the macro block, the image encoding apparatus may set the filtering strength of the deblocking filter for the boundary between the macro blocks, that is, the boundary strength (BS) to 3 ( 324). In this case, the boundary strength BS may be set to a value from 0 to 4, wherein the boundary strength 0 means no filtering, and the boundary strength 4 indicates the strongest filtering strength.

If the boundary between the two blocks is an edge of the macro block, the image encoding apparatus may check whether both macroblocks are encoded in the intra mode without the residual signal based on the boundary (650). ). In this case, the intra mode without the difference signal may include an intra skip mode, an intra direct mode, an intra direction mode, and the like.

As a result of checking 650, if both blocks are encoded in the intra mode without the difference signal based on the boundary, the image encoding apparatus may set the boundary strength to 3 (624). On the other hand, if it is confirmed in 650 that both blocks are not encoded in the intra mode without the difference signal based on the boundary, the image encoding apparatus may set the boundary strength to 4 (622).

In operation 630, when both blocks are inter-coded based on the boundary, the image encoding apparatus may check whether there is a difference signal in at least one of both macroblocks based on the boundary (630).

As a result of checking 630, if there is a macroblock with a difference signal, the image encoding apparatus may set the boundary strength to 2 (632). On the contrary, if the macroblock with the difference signal does not exist as a result of checking 630, the image encoding apparatus may determine whether the motion difference of both macroblocks is greater than or equal to a reference value or whether both macroblocks refer to different frames (640). .

In operation 640, when both macroblocks have a motion difference greater than or equal to a reference value or both macroblocks refer to different frames, the image encoding apparatus may set the boundary strength to 1 (642). On the other hand, when the verification result of 640 indicates that both macroblocks have a motion difference smaller than the reference value and both blocks refer to the same frame, the image encoding apparatus may set the boundary strength to 0 (644).

Referring to FIG. 6, in the case of intra coding, the filtering strength of the deblocking filter for the boundary between macroblocks of the depth image is determined to be 4 or 3, and in the case of inter coding, the filtering strength is determined to be one of 2, 1, and 0. Can be.

Comparing the set values of the filtering strengths determined in this way, it can be seen that the case of intra coding is relatively stronger than the case of inter coding. As described above, the deblocking filter used for image compression is designed in consideration of the characteristics of the color image. However. The depth image is characterized in that it is flat unlike the color image. Therefore, an embodiment will be described with reference to a deblocking filtering method suitable for a depth image having different characteristics from a color image.

7 is a flowchart illustrating a method of determining a filtering strength of a deblocking filter for a depth image, according to an exemplary embodiment.

Determining the boundary intensity for the depth image has additional considerations in addition to the conditions to be considered when determining the boundary intensity of the color image. For example, a coding method of performing bit rate distortion optimization on a depth image in view of synthesis of a virtual image among depth image encoding methods will be described.

In the encoding method of performing bit rate distortion optimization on a depth image, an image before deblocking filtering is already optimized for synthesis of a virtual image. Therefore, even if a blocking phenomenon exists, the boundary strength should be set small so that the pixel value of the coded depth image does not change as much as possible. Therefore, in determining the boundary strength of the depth image, information about which bit rate-distortion optimization model is used in encoding the depth image may be considered.

Referring to FIG. 7, an apparatus for determining a filtering strength of a deblocking filter for a depth image (hereinafter, a filtering device) may include at least one mode among a plurality of encoding / decoding modes for a depth image. Can be determined (710). In operation 710, the filtering device may determine at least one mode based on a bit rate-distortion optimization (RDO) model.

The plurality of encoding / decoding modes may include a first mode that does not perform deblocking filtering on a boundary between macroblocks and a second mode that performs deblocking based on the filtering strength.

The filtering device may acquire information on at least one mode determined at 710 (720).

The filtering device may set the filtering strength of the deblocking filter for the boundary between the macroblocks of the depth image based on the information about the at least one mode determined in 720 (730). Here, the filtering strength of the deblocking filter for the boundary between the macroblocks of the depth image may be referred to as 'boundary strength'.

In operation 730, the filtering apparatus sets the same filtering strength for the intra-coded block and the inter-coded block among the macroblocks disposed on both sides of the macroblock based on the information about the at least one mode. Can be. In this case, as the deblocking filter for the depth image, a filter for preserving edges such as a median filter or a bilateral filter, rather than an averaging filter may be used. Can be.

As described above with reference to FIG. 6, in the deblocking filtering of the color image, the boundary strength is set to be large for the macro coded / decoded macroblock. This means that the pixels used for the prediction of the intra-encoded / decoded macroblock are neighboring pixels that have not undergone deblocking, and the efficiency of the prediction for the intra-encoded / decoded macroblock is higher than the prediction for the inter-encoded / decoded macroblock. Because it is not good. However, since the depth image has a large spatial correlation, the prediction of the intra coded / decoded macroblock in the depth image is as effective as the prediction of the inter coded / decoded macroblock. Therefore, unlike the color image, the depth image may set the same filtering strength for the intra-encoded / decoded macroblock and the inter-encoded / decoded macroblock among macroblocks disposed on both sides based on the boundary between the macroblocks. have.

In the case of an I slice using only prediction for an intra coded / decoded macroblock, the condition of the boundary strength described in FIG. 6 needs to be maintained. However, in the case of the intra coded / decoded macroblock in the P slice or the B slice, the same boundary strength condition may be used as the inter coded / decoded macroblock. Here, a slice refers to a set of macro blocks composed of several macro blocks, and can be understood as a connection structure consisting of only macro blocks on the same horizontal line.

In addition, when the coded bit pattern (CBP) of the block is 0 so that there is no residual signal and the direction of successive intra macro blocks is the same, the filtering device sets the boundary strength for the vertical or horizontal deblocking filter to '0' according to the direction. Can be.

The filtering device may set the filtering intensity differently in the horizontal direction or the vertical direction according to the arrangement of the camera for capturing the depth image. In other words, the filtering device may omit horizontal or vertical deblocking filtering depending on the arrangement of the cameras.

The filtering device may set the filtering strength based on the direction of the intra coded / decoded macro block among the macro blocks. In this case, the directionality of the macro block is a vertical direction extending vertically from the upper boundary pixel of the macro block, a horizontal direction extending horizontally from the left boundary pixel of the macro block, and an upper boundary pixel and the left boundary of the macro block. One of a direct current (DC) direction extending in the average direction of the pixels and a plane direction extending in the direction in which both the upper boundary pixel and the left boundary pixel of the macro block are considered may be included. In addition, the filtering device may determine the filtering strength in consideration of the slice type of the depth image.

The filtering device may determine whether to perform deblocking filtering on the depth image (740). In this case, the filtering device may determine whether to perform deblocking filtering on the depth image based on at least two thresholds (for example, α ′ and β ′). The at least two thresholds may be for a difference between the first pixel and the second pixel that are continuous to each other in the macro blocks of the adjacent depth image among the macro blocks. For example, the first pixel and the second pixel that are continuous to each other in the macroblocks of the depth image may include pixel p0 and pixel q0, pixel p1 and pixel p0, and pixel q1 and pixel q0 in FIG. 2.

At least two thresholds may be used to distinguish whether the boundary between macro blocks is a boundary area in the original image or is due to discontinuity between pixels due to blocking phenomenon.

In an embodiment, at least two thresholds α ′ and β ′ for deblocking filtering on the depth image may be determined to satisfy Equation 3 below.

Here, m is a value for reflecting the characteristics of the depth image according to the change of the quantization parameter QP, n is a value that can be determined in consideration of the characteristics of the image to be encoded (for example, the depth image). As such, m and n may be obtained according to characteristics of the depth image. For example, when m is '0', the thresholds α 'and β' may not be affected by the change in the quantization parameter QP. . [alpha] and [beta] will refer to the description of Equation 2 described above.

The filtering device may perform deblocking filtering on boundaries between macroblocks of the depth image according to the determination result of operation 740 (750).

8 is a flowchart illustrating a method of determining a filtering strength of a deblocking filter for a depth image, according to another exemplary embodiment.

Referring to FIG. 8, in operation 810, the filtering apparatus may determine at least one mode among a plurality of encoding / decoding modes for a depth image. In operation 510, the filtering apparatus may determine at least one mode based on a bit rate-distortion optimization (RDO) model.

The filtering device may acquire information on at least one mode determined at 810 (820).

The filtering device may set the filtering strength of the deblocking filter for the boundary between the macroblocks of the depth image, based on the information about the at least one mode acquired in 820 (830). Here, the filtering strength of the deblocking filter for the boundary between the macroblocks of the depth image may be referred to as 'boundary strength'.

The filtering device generates a difference signal in units of macroblocks based on the information on at least one mode acquired in 820, and then converts the difference signal in operation 850. At 850, the filtering device may transform the differential signal using, for example, a Discrete cosine transform (DCT) method.

The filtering device may determine at least two thresholds, based on the quantization coefficients (QPs) quantized the transformed differential signal at 850 (operation 860). In operation 860, the filtering device may determine at least two thresholds using Equation 3 described above. In this case, the at least two thresholds may represent a difference between the first pixel and the second pixel that are continuous to each other in the macroblocks of the adjacent depth image.

The filtering device may determine whether to perform deblocking filtering on the depth image based on at least two thresholds (870).

The filtering device may perform deblocking filtering on boundaries between macroblocks of the depth image according to the determination result of 870 (880).

9 is a block diagram of an apparatus for determining a filtering strength of a deblocking filter for a depth image, according to an exemplary embodiment.

Referring to FIG. 9, the filtering apparatus according to an embodiment may include a mode determiner 910, an acquirer 920, a setter 930, a determiner 940, a converter 950, and a threshold determiner 960. ) May be included.

The mode determiner 910 may determine at least one of a plurality of encoding / decoding modes for the depth image. In addition, the mode determiner 910 may determine at least one mode among a plurality of encoding / decoding modes for the depth image based on a bit rate-distortion optimization (RDO) model. The plurality of encoding / decoding modes may include a first mode that does not perform deblocking filtering on a boundary between macroblocks and a second mode that performs deblocking based on the filtering strength.

The acquirer 920 may obtain information about at least one mode determined by the mode determiner 910.

The setting unit 930 may set the filtering strength of the deblocking filter for the boundary between the macroblocks of the depth image based on the information about the at least one mode acquired by the obtaining unit 920. have. At this time, the setting unit 930 is based on the information on the at least one mode, the intra coded / decoded block and the inter coded / decoded block among the macro blocks disposed on both sides based on the boundary between the macro blocks; It is possible to set the filtering intensity for the same.

In addition, the setting unit 930 sets the filtering intensity based on the directionality of the intra coded / decoded macroblock among the macroblocks, or varies the filtering intensity in the horizontal direction or the vertical direction according to the arrangement of the camera that captures the depth image. Can be set. In addition, the setting unit 930 does not have a residual signal in the macroblocks positioned on both sides of the boundary between the macroblocks, and when the directionalities of consecutive intra-encoded / decoded macroblocks are the same, The filtering intensity for the horizontal deblocking filter or the vertical deblocking filter may be set based on the directionality.

The determination unit 940 may determine whether to perform deblocking filtering on the depth image based on at least two thresholds of the difference between the first pixel and the second pixel which are continuous to each other in the macro blocks of the adjacent depth image. have.

The converter 950 may convert a residual signal in units of macro blocks, which is generated based on information on at least one encoding mode. In this case, the transform unit 950 may transform the differential signal using, for example, a Discrete cosine transform (DCT) method.

The threshold determiner 960 may determine the at least two thresholds based on a quantization coefficient obtained by quantizing the difference signal converted by the transformer 950.

The apparatus for determining the filtering strength of the deblocking filter for the depth image according to an embodiment may be implemented by the above-described deblocking filter units 232 and 326 of the depth image encoding apparatus and the depth image decoding apparatus, or as separate apparatuses. It may be implemented.

The method according to an embodiment of the present invention can be implemented in the form of a program command which can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be construed as being limited to the embodiments described, but should be determined by equivalents to the appended claims, as well as the appended claims.

910: mode determination unit
920:
930: setting unit
940: judgment
950: conversion unit
960: threshold determination unit

Claims (20)

Determining at least one of a plurality of encoding / decoding modes for the depth image;
Acquiring information about the determined at least one mode; And
Setting a filtering strength of a deblocking filter on a boundary between macroblocks of the depth image based on the information on the at least one mode.
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
Determining whether to perform deblocking filtering on the depth image based on at least two thresholds of the difference between the first pixel and the second pixel which are consecutive to each other in the macroblocks of the adjacent depth image;
And determining a filtering strength of the deblocking filter for the depth image. 3. The method of claim 2,
The at least two thresholds
And determining the filtering intensity of the deblocking filter for the depth image, which is used to distinguish whether the boundary between the macroblocks is a boundary region in the original image or due to discontinuity between pixels due to a blocking phenomenon. 3. The method of claim 2,
Converting a difference signal in units of macro blocks generated based on the information on the at least one mode; And
Determining the at least two thresholds based on the quantized coefficients of the quantized transformed differential signal
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
The plurality of encoding / decoding modes
A deblocking filter for a depth image including a first mode that does not perform deblocking filtering on a boundary between the macroblocks and a second mode that performs the deblocking based on the filtering strength. How to determine the filtering strength. The method of claim 1,
The setting of the filtering strength
Based on the information on the at least one mode, the filtering strength of the intra-encoded / decoded block and the inter-encoded / decoded block among the macroblocks disposed on both sides based on the boundary between the macroblocks is equal to each other. Steps to set up
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
Setting the filtering intensity differently in a horizontal direction or a vertical direction according to an arrangement of a camera for capturing the depth image;
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
Setting the filtering strength based on a directionality of an intra coded / decoded macroblock among the macroblocks
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
Determining the filtering intensity in consideration of the slice type of the depth image
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
If there is no residual signal in the macro blocks positioned on both sides of the boundary between the macro blocks, and the directionality of consecutive intra coded / decoded macro blocks is the same, horizontal deblocking based on the directionality Setting the filtering strength for the filter or vertical deblocking filter
And determining a filtering strength of the deblocking filter for the depth image. The method of claim 1,
Determining the at least one encoding mode
Determining at least one of a plurality of encoding / decoding modes for the depth image based on a bit rate-distortion optimization (RDO) model
And determining a filtering strength of the deblocking filter for the depth image. A computer-readable recording medium for recording a program for performing the method of any one of claims 1 to 11. A mode determination unit to determine at least one mode among a plurality of encoding / decoding modes for the depth image;
An acquisition unit for obtaining information about the determined at least one mode; And
A setting unit configured to set a filtering strength of a deblocking filter on a boundary between macroblocks of the depth image based on the at least one mode information;
Device for determining the filtering strength of the deblocking filter for a depth image comprising a. 14. The method of claim 13,
Determination unit for determining whether or not to perform the deblocking filtering on the depth image based on at least two thresholds for the difference between the first pixel and the second pixel continuous in the macroblocks of the adjacent depth image
Apparatus for determining the filtering strength of the deblocking filter for a depth image further comprising. 15. The method of claim 14,
A conversion unit for converting a difference signal in units of macro blocks generated based on the information on the at least one mode; And
A threshold determination unit that determines the at least two thresholds based on the quantized coefficients of the quantized transformed differential signal
Apparatus for determining the filtering strength of the deblocking filter for a depth image further comprising. 14. The method of claim 13,
The plurality of encoding / decoding modes
A deblocking filter for a depth image including a first mode that does not perform deblocking filtering on a boundary between the macroblocks and a second mode that performs the deblocking based on the filtering strength. Device for determining filtering strength. 14. The method of claim 13,
The setting unit
On the basis of the information on the at least one mode, to set the same filtering strength for the intra coded block and the inter coded / decoded block among the macro blocks arranged on both sides based on the boundary between the macro blocks; A device for determining the filtering strength of a deblocking filter for depth images. 15. The method of claim 14,
The setting unit
And an apparatus for determining a filtering strength of a deblocking filter for a depth image in which the filtering intensity is set differently in a horizontal direction or a vertical direction according to the arrangement of the camera that captures the depth image. 15. The method of claim 14,
The setting unit
And an apparatus for determining a filtering strength of a deblocking filter for a depth image that sets the filtering strength based on a direction of an intra coded / decoded macroblock among the macroblocks. 15. The method of claim 14,
The setting unit
If there is no residual signal in the macro blocks positioned on both sides of the boundary between the macro blocks, and the directionality of consecutive intra coded / decoded macro blocks is the same, horizontal deblocking based on the directionality A device for determining the filtering strength of a deblocking filter for a depth image that sets the filtering strength for a filter or a vertical deblocking filter.

Images