KR101598857B1 - Image/video coding and decoding system and method using graph based pixel prediction and depth map coding system and method - Google Patents

Abstract

A system and method for image encoding / decoding using graph-based pixel prediction and a depth map encoding system and method are disclosed. The encoding system generates an edge map based on a pixel value difference between neighboring pixels in an input image, connects the pixels through the generated edge map to generate a graph, and based on the generated graph, And the image is encoded based on the selected prediction value.

Description

TECHNICAL FIELD [0001] The present invention relates to an image encoding / decoding system and method using a graph-based pixel prediction, a depth map encoding system, and a depth map encoding system and method.

Embodiments of the present invention relate to a system and method for image encoding / decoding using graph-based pixel prediction and a depth map encoding system and method.

There has been used a method of predicting a picture using a neighboring pixel of a current picture to be encoded when an image (including a moving picture) is encoded. For example, in H.264, which is a digital video codec standard, when an image of one frame is divided into blocks of 4x4 pixels, when encoding one block, the pixel values of the current block are calculated using the pixel values of the neighboring blocks After the prediction, the compression efficiency is improved by encoding the difference between the predicted value and the pixel value of the current block.

It is difficult to precisely predict the pixel value of the current block with only surrounding pixel values, so that it is possible to perform more accurate prediction by additionally providing the shape information of the current block. For example, when a depth map having three-dimensional depth information of an image is encoded, a method of dividing the current block into one straight line, dividing the current block into two regions, and encoding a representative value of each region has been used.

In this specification, a system and a method for effectively encoding and decoding an image are proposed.

An edge map generating unit for generating an edge map based on pixel value differences between neighboring pixels in the input image, a graph generating unit for generating a graph by connecting pixels through the generated edge map, There is provided an encoding system including a predictive value selection unit for selecting a predictive value for one pixel, and a predictive encoding unit for predicting at least one pixel and at least one pixel based on the predictive value.

According to one aspect, when the pixel value difference is equal to or greater than a predetermined threshold value, the edge map generating unit can determine that an edge exists between neighboring pixels and display the edge map on the edge map, thereby generating an edge map. At this time, the threshold value can be adjusted at the time of edge generation so that the bit amount of the encoded edge map can be adjusted by adjusting the edge generation amount according to the target bit rate of the input image.

In another aspect, the encoding system may further include an edge map encoding unit encoding the generated edge map, and the edge map encoding unit may compress the edge map by compressing the edge map through binary arithmetic encoding. At this time, the encoded edge map may be included in the bit string in which the input image is encoded.

In another aspect, the graph generating unit can generate a graph by connecting a plurality of pixels that are not separated from each other by edges of an edge map.

Generating an edge map based on a pixel value difference between neighboring pixels in an input image, generating a graph by connecting pixels through the generated edge map, generating a graph based on the generated graph, Selecting a predicted value, and predicting the at least one pixel based on the at least one pixel and the predicted value.

An edge map decoding unit for decoding the edge map in the inputted bit stream, a graph generating unit for connecting the pixels through the decoded edge map to generate a graph, a predictive value selecting unit for selecting a predictive value for at least one pixel And a prediction compensating unit for restoring at least one pixel based on the selection unit and the predictive value and the pixel information of the input bitstream.

Generating a graph by connecting pixels through a decoded edge map; selecting a prediction value for at least one pixel based on the generated graph; And reconstructing at least one pixel based on the pixel information of the bit stream.

There is provided a depth map encoding method including a step of predicting an image quality of an image to be synthesized through a depth map based on a depth map and a reference image, and a step of selecting an encoding mode based on the predicted image quality.

There is provided a depth map encoding system including an image quality predicting unit for predicting an image quality of an image to be synthesized through a depth map based on a depth map and a reference image, and an encoding mode selecting unit for selecting an encoding mode based on the predicted image quality.

By performing pixel prediction effectively according to the local information of the image using the graph, the compression efficiency can be improved to improve the quality of the reconstructed image and reduce the amount of compressed bits.

When a 3D image is encoded through image synthesis using a depth map, the optimized encoding mode can be determined by predicting the effect of the distortion value generated by loss coding the depth map on the synthesized image.

1 is a block diagram for explaining an internal configuration of a coding system according to an embodiment of the present invention.
2 is a block diagram for explaining an internal configuration of a decoding unit in an embodiment of the present invention.
FIG. 3 is an example of pixels of an image according to an embodiment of the present invention.
4 is a flowchart illustrating an encoding method according to an embodiment of the present invention.
5 is a block diagram for explaining an internal configuration of a decoding system according to an embodiment of the present invention.
6 is a flowchart showing a decoding method in an embodiment of the present invention.
7 is a block diagram for explaining an internal configuration of a depth map encoding system according to an embodiment of the present invention.
8 is a flowchart illustrating a depth map encoding method according to an embodiment of the present invention.
9 is an example for explaining a method of estimating a distortion occurring in a composite image using distortion of a depth map in an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram for explaining an internal configuration of a coding system according to an embodiment of the present invention. 1, the coding system according to the present embodiment includes an edge map generating unit 101, a graph generating unit 102, a predicted value selecting unit 130, a predictive encoding unit 104, a transforming and quantizing unit 105, an entropy encoding unit 106, a decoding unit 107, and an edge map encoding unit 109.

The edge map generation unit 101 generates an edge map based on pixel value differences between neighboring pixels in the input image. At this time, when the pixel value difference is equal to or larger than a preset threshold value, the edge map generation unit 101 may determine that an edge exists between neighboring pixels and display the edge map on the edge map, thereby generating an edge map. In this case, when the pixel value difference is not equal to or greater than the threshold value, it is determined that there is no edge between neighboring pixels, and it can be displayed on the edge map. For example, when it is determined that an edge exists, the value of '1' is assigned to the corresponding position of the edge map, and when it is determined that no edge exists, the value of '0' And can be displayed on the edge map. Here, the threshold value can be adjusted at the time of edge generation so that the bit amount of the encoded edge map can be adjusted by adjusting the edge generation amount according to the target bit rate of the input image.

The graph generating unit 102 generates a graph by connecting pixels through the generated edge map. At this time, the graph generating unit 102 may generate a graph by connecting a plurality of pixels that are not separated from each other by the edge of the edge map. That is, in order to generate a graph, the graph generating unit 102 refers to the edge map to connect the two pixels when there is an edge between neighboring two pixels, and connects the two pixels when there is no edge, can do.

At this time, for example, the graph can be expressed by the following equation (1).

Here, 'G' may denote a graph, 'V' may denote a pixel of an image, and 'E' may denote a connection between pixels. In this case, when the pixel at the position (i, j) and the pixel at the position (i ', j') are connected to each other, the connection between the pixels can be expressed by the following equation (2).

At this time, the directionality between connected pixels may not be displayed.

When an input image is categorized and coded into blocks of a predetermined pixel size, each block can be divided into a plurality of regions by the corresponding graph. That is, the pixels connected to each other can form one region by creating one path.

The predictive value selection unit 130 selects a predictive value for at least one pixel based on the generated graph. At this time, the predictive value selection unit 130 may select a pixel value of a pixel closest to a block including at least one pixel and closest to at least one pixel among at least one pixel and a graph-connected pixel as a predicted value.

Alternatively, the predictive value selector 130 may assign different weights to the pixel values of neighboring pixels that are adjacent to the block including at least one pixel and are graphically connected to the at least one pixel, Can be selected as the predicted value.

Such prediction value selection will be described later in more detail with reference to FIG.

At this time, the weights can be determined based on the spectral distribution of the graph. At this time, the predictive value selecting unit 130 may calculate the Eigen vector of the Laplacian matrix of the graph to use the spectral distribution, and determine the weight based on the eigenvector. For example, a laplacian matrix can be calculated from a graph and a eigenvector can be calculated to represent a spectral distribution. At this time, the eigenvalues are expressed in a descending order from a small value to a high frequency in a low frequency. In this case, the pixel value x to be predicted can be obtained by setting the pixel value of the pixel to be predicted to x, generating a vector expressed by the following Equation (3), and then using Equation (4) below.

Here, 'e' may denote a eigenvector, and 'a ₂ a ₃ ... a _N ' may denote a pixel value of a block including neighboring pixels including a pixel to be predicted. 'T' can be a transpose, and the operator '·' can mean a vector inner operator.

In another embodiment, the predictive value selector 130 may calculate a predictive value of a pixel value of a block including at least one pixel and a pixel value of neighboring, graphically connected pixels or pixel values assigned different weights to a plane equation As a predicted value.

At this time, the pixel values of the decoded pixels may be used as the pixel values of the block and neighboring pixels in order to find an appropriate predicted value for each area. This is for the purpose of restoring the encoded pixels using the same information in the decoding system to be described later. A method of using the pixel values of the pixels around the decoded block will be described in detail later.

The predictive encoding unit 104 predictively encodes at least one pixel based on at least one pixel and a predictive value. At this time, the predictive encoding unit 104 may encode the difference value between the pixel value of the at least one pixel and the predictive value.

The transform and quantization unit 105 transforms and quantizes the predictively encoded pixels. That is, the predictive-coded difference values of the block can be loss-compressed through transform and quantization such as Discrete Cosine Transform (DCT).

The entropy encoding unit 106 entropy-codes the transformed and quantized pixels to generate a bit stream. At this time, the entropy encoding unit 106 may compress the pixel values of the transformed and quantized pixels, that is, the transformed and quantized difference values, into bit streams through entropy encoding such as Huffman encoding or arithmetic encoding.

The decoding unit 107 decodes the converted and quantized pixels based on the predicted values. At this time, the reconstructed neighboring pixels 108 may refer to the decoded pixels mentioned above for a while. That is, the pixel value of the neighboring pixels of the block including at least one pixel in the predictive value selecting unit 103 can use the pixel values of the corresponding restored neighboring pixels. As described above, using the reconstructed neighboring pixels 108 for prediction value selection is intended to enable reconstruction of encoded pixels using the same information in a decoding system.

The edge map encoding unit 109 encodes the generated edge map. At this time, the edge map encoding unit 109 can compress the edge map by binary arithmetic coding and encode the edge map. In this way, the encoded edge map can be included in the bit string obtained by coding the inputted image, which is the bit stream described above.

2 is a block diagram for explaining an internal configuration of a decoding unit in an embodiment of the present invention. 2, the decoding unit 107 may include an inverse transform and inverse quantization unit 201 and a prediction compensation unit 202. [

The inverse transform and inverse quantization unit 201 may inversely transform and dequantize the transformed and quantized values through the transform and quantization unit 105. [

In addition, the prediction compensator 202 may restore the pixels by applying the predictive value selected by the predictive value selector 103 to the inverse-transformed and inverse-quantized values. The reason for reconstructing the pixels again is that the predicted value is selected using the pixel value of the reconstructed pixel in the predicted value selection unit 103 so that the reconstructed pixel can be reconstructed using the same information in the decoding system.

FIG. 3 is an example of pixels of an image according to an embodiment of the present invention. The figure 300 shows a plurality of pixels, and upper and lower case alphabets may refer to respective pixels. In the example of FIG. 3, the pixels of the alphabets a to p constitute one block of 4x4 pixel size. The bold line in figure (300) shows that the pixels are divided into two areas by edges. That is, the pixels a, b, c, d, e, f, g and i and the pixels h, j, k, l, m, n, o and p are separated from each other by the edge. At this time, the pixel value of the pixel L, which is the closest pixel among the pixels neighboring the corresponding block and connected by the graph, is selected as the prediction value for each of the pixels h, j, k, l, m, have. The pixel values of the closest pixels neighboring the corresponding block and connected to the graph by the predicted values of the pixels a, b, c, d, e, f, . For example, a pixel value of the pixel B may be used as a predicted value for the pixel b, and a pixel value of the pixel C may be used as a predicted value for the pixel c and g. When there are a plurality of closest pixels such as pixels a and f, priority can be given to pixels existing in a specific direction. In the example of FIG. 3, the pixel value of the uppermost pixel is selected as the predicted value. In this case, the pixel value of the selected pixel may be directly used as the predicted value. However, as described with reference to FIG. 1, if the pixel value of the selected pixel is connected to a plurality of pixels, different weights are given to the pixel values of the pixels, May be used as a predicted value. For example, pixels a, b, c, d, e, f, g and i in the 4x4 block are included in one path of the graph, and pixels A, B, C, D, . In this case, weights are obtained for the pixels A, B, C, D, I, J, and K and the weights w_A, w_B, w_C, w_D, w_I, w_J, The predicted value can be calculated as shown in Equation (5) below.

Here, 'a_pred' may mean the predicted value of the pixel a.

4 is a flowchart illustrating an encoding method according to an embodiment of the present invention. The encoding method according to the present embodiment can be performed by the encoding system described with reference to FIG. In FIG. 4, a description will be given of a coding method by explaining a process of performing each step by the coding system.

In step 410, the encoding system generates an edge map based on pixel value differences between neighboring pixels in the input image. At this time, if the pixel value difference is equal to or greater than a predetermined threshold value, the encoding system determines that an edge exists between neighboring pixels and displays the edge map on the edge map, thereby generating an edge map. In this case, when the pixel value difference is not equal to or greater than the threshold value, it is determined that there is no edge between neighboring pixels, and it can be displayed on the edge map. For example, when it is determined that an edge exists, the value of '1' is assigned to the corresponding position of the edge map, and when it is determined that no edge exists, the value of '0' And can be displayed on the edge map. Here, the threshold value can be adjusted at the time of edge generation so that the bit amount of the encoded edge map can be adjusted by adjusting the edge generation amount according to the target bit rate of the input image.

In operation 420, the encoding system generates a graph by connecting pixels through the generated edge map. At this time, the encoding system can generate a graph by connecting a plurality of pixels that are not separated from each other by the edge of the edge map. That is, in order to generate the graph, the encoding system can generate the graph by connecting the two pixels when there is an edge between neighboring two pixels by referring to the edge map, and by connecting the two pixels when there is no edge.

At this time, for example, the graph can be expressed as Equation (1).

When an input image is categorized and coded into blocks of a predetermined pixel size, each block can be divided into a plurality of regions by the corresponding graph. That is, the pixels connected to each other can form one region by creating one path.

In step 430, the encoding system selects a predicted value for at least one pixel based on the generated graph. At this time, the encoding system can select a pixel value of a pixel closest to a block including at least one pixel and adjacent to at least one pixel among at least one pixel and a graph-connected pixel as a predicted value.

Alternatively, the encoding system may assign different weights to the pixel values of neighboring pixels that are adjacent to the block including at least one pixel and are graphically connected to the at least one pixel, The calculation result between the values can be selected as the predicted value.

At this time, the weights can be determined based on the spectral distribution of the graph. At this time, in order to use the spectral distribution, the encoding system can calculate the eigenvectors of the Laplacian matrix of the graph and determine the weights based on the eigenvectors. For example, a laplacian matrix can be calculated from a graph and a eigenvector can be calculated to represent a spectral distribution. At this time, the eigenvalues are expressed in a descending order from a small value to a high frequency in a low frequency. At this time, the pixel value x to be predicted can be obtained by setting the pixel value of the pixel to be predicted to x, generating the vector expressed by the above-described Equation 3, and then using Equation 4 below.

In yet another embodiment, the encoding system may select a plane value calculated by substituting pixel values of neighboring, graphically connected pixels or pixel values assigned different weights into a plane equation with a block including at least one pixel as a predicted value It is possible.

At this time, the pixel values of the decoded pixels may be used as the pixel values of the block and neighboring pixels in order to find an appropriate predicted value for each area. This is for the purpose of restoring the encoded pixels using the same information in the decoding system to be described later. A method of using the pixel values of the pixels around the decoded block will be described in detail later.

In step 440, the encoding system predictively encodes at least one pixel based on the at least one pixel and the predicted value. At this time, the encoding system can encode the difference value between the pixel value of the at least one pixel and the predicted value, for example.

In step 450, the encoding system transforms and quantizes the predictively encoded pixels. That is, the predictive-coded difference values of the block can be loss-compressed through conversion and quantization such as discrete cosine transform.

In step 460, the encoding system entropy-codes the transformed and quantized pixels to generate a bit stream. At this time, the encoding system can compress the pixel values of the transformed and quantized pixels, that is, the transformed and quantized difference values, into a bit stream through entropy encoding such as Huffman encoding or arithmetic encoding.

In step 470, the encoding system encodes the generated edge map. At this time, the encoding system can encode the edge map by compressing the edge map through binary arithmetic coding. In this way, the encoded edge map can be included in the bit string obtained by coding the inputted image, which is the bit stream described above.

The encoding system may also decode the transformed and quantized pixels based on the predicted values. That is, in step 430, pixel values of neighboring pixels of a block including at least one pixel may use pixel values of corresponding decoded neighboring pixels. As described above, the use of such decoded neighboring pixels for prediction value selection is intended to enable the decoding method to recover the encoded pixels using the same information.

5 is a block diagram for explaining an internal configuration of a decoding system according to an embodiment of the present invention. 5, the decoding system according to the present embodiment includes an edge map decoding unit 501, a graph generating unit 502, a predicted value selecting unit 503, an entropy decoding unit 504, an inverse transforming and inverse quantizing unit 505, and a prediction compensation unit 507. [

The edge map decoding unit 501 decodes the edge map in the inputted bit stream. At this time, the edge map decoding unit 501 can decode the edge map from the input bit stream through the binary arithmetic decoding.

The graph generating unit 502 generates a graph by connecting pixels through the decoded edge map. At this time, the graph generating unit 502 may generate a graph by connecting a plurality of pixels that are not separated from each other by the edge of the edge map. That is, the method of generating the graph is the same as the method described with reference to FIG. 1 and FIG.

The predictive value selection unit 503 selects a predictive value for at least one pixel based on the generated graph. In this case, the image corresponding to the inputted bit stream can be classified into blocks of a predetermined pixel size, the predictive value selector 503 is adjacent to a block including at least one pixel, and the at least one pixel and the graph A pixel value of a pixel closest to the at least one pixel is selected as the predicted value or a block neighboring a block including the at least one pixel, And the calculation result between the pixel values to which different weights are assigned can be selected as the predicted value. That is, the method of selecting the predicted value is also the same as the method described with reference to FIG. 1 and FIG.

The entropy decoding unit 504 entropy-decodes the input bitstream to decode pixels, and the inverse-transform and inverse-quantization unit 505 inversely transforms and dequantizes entropy-decoded pixels. That is, the decoding system can decode the bit stream generated through the entropy encoding, the transform and the inverse quantization in FIG. 1 and FIG. 4 into the original value through entropy decoding, inverse transform and inverse quantization.

At this time, the predictive value selecting unit 503 may select a predictive value for the at least one pixel based on the pixels neighboring the block including the at least one pixel among the inversely transformed and inverse-quantized pixels and the graph. In other words, corresponding pixels among the inversely transformed and dequantized pixels may be used as neighboring pixels of the above-described block. Here, the reconstructed neighboring pixels 506 shown in FIG. 5 may refer to a pixel neighboring a block including the at least one pixel among the inversely-transformed and inverse-quantized pixels.

The prediction compensating unit 507 restores at least one pixel based on the predictive value and the pixel information of the input bit stream. That is, the pixel information of the bit string includes a difference value between the original pixel value and the predicted value, and the predictive compensation unit 507 can restore at least one pixel by adding a predictive value to the difference value.

6 is a flowchart showing a decoding method in an embodiment of the present invention. The decoding method according to the present embodiment can be performed by the decoding system described with reference to FIG. FIG. 6 illustrates a decoding method by explaining a process of performing each step by the decoding system.

In step 610, the decoding system entropy-decodes the input bit stream to decode the pixel.

In step 620, the decoding system decodes the edge map in the input bitstream. At this time, the decoding system can decode the edge map from the input bitstream through binary arithmetic decoding.

In operation 630, the decoding system inversely transforms and dequantizes entropy-decoded pixels. That is, the decoding system reconstructs the bitstream generated through the entropy encoding, the transform, and the inverse quantization in FIGS. 1 and 4 through the steps 610 and 630 to the original value through entropy decoding, inverse transform, and inverse quantization It can be decoded.

In step 640, the decoding system generates a graph by connecting the pixels through the decoded edge map. At this time, the decoding system can generate a graph by connecting a plurality of pixels which are not separated from each other by the edge of the edge map. That is, the method of generating the graph is the same as the method described with reference to FIG. 1 and FIG.

At step 650, the decoding system selects a prediction value for at least one pixel based on the generated graph. In this case, the image corresponding to the input bit stream may be classified into blocks of a predetermined pixel size, the decoding system may be adjacent to a block including at least one pixel, and the pixels connected to the at least one pixel A pixel value of a pixel closest to the at least one pixel is selected as the predicted value or a pixel value of a pixel neighboring a block including the at least one pixel and connected to the graph is weighted with different weights And the calculation result between the pixel values to which different weights are assigned can be selected as the predicted value. That is, the method of selecting the predicted value is also the same as the method described with reference to FIG. 1 and FIG.

The decoding system may also select a prediction value for the at least one pixel based on the neighboring pixels and the graph including the at least one pixel of the inversely transformed and dequantized pixels. That is, corresponding pixels among the inversely transformed and dequantized pixels may be used as neighboring pixels of the above-described block.

In step 650, the decoding system restores the at least one pixel based on the predicted value and the pixel information of the input bit stream. That is, the pixel information of the bit stream is a value including the difference value between the original pixel value and the predicted value, and the decoding system can restore at least one pixel by adding the predicted value to the difference value.

The description omitted in FIGS. 5 and 6 can be referred to the description of FIG. 1 to FIG.

Hereinafter, a method of synthesizing an image using a depth map to encode a three-dimensional image will be described. In this case, the encoding / decoding system and methods according to the embodiments of the present invention can be effectively used for compressing the depth map. In this case, when the graph-based prediction method and other conventional prediction methods (eg, 4 × 4 spatial prediction method, 16 × 16 spatial prediction method, or various temporal prediction methods used in H.264) are used, an optimal prediction method is determined and used There is a need. In order to determine an optimal prediction method, a prediction method having the least cost can be selected by calculating the rate distortion cost considering the amount of bits generated and distortion caused thereby. To calculate this cost, a Lagrange optimization method, which can be expressed as Equation (6) below, can be used.

Here, 'D' may denote a distortion value, 'R' may denote a bit rate, and 'λ' may denote a Lagrangian coefficient. At this time, in the embodiments of the present invention, in order to calculate the distortion value, the distortion value generated by the loss-coding of the depth map is not directly used but the influence of the distortion value on the synthesized image is predicted, . This is because the depth map itself is not displayed to the user when the depth map is applied to a three-dimensional image, but is used only to synthesize an image. Therefore, it is desirable to perform the optimization using the distortion value generated in the synthesized image appearing to the user. However, in order to perform the optimization, calculating the distortion value by synthesizing the images for each prediction method requires too much complexity and is practically limited. Therefore, it is possible to perform more accurate optimization with the predicted value by predicting the distortion occurring in the synthesized image using the distortion generated in the depth map.

At this time, when coding the depth map and the corresponding image, the displacement value generated by multiplying the pixel value of the image at the same position as the pixel position of the depth map to be coded and the distortion value generated in the depth map pixel value at that position The distortion value generated in the synthesized image can be predicted using the difference between the pixel values of the pixels at the shifted positions. In this case, the weight value used for calculating the displacement includes an internal coefficient such as a focal distance of the camera used to generate the three-dimensional image, an external coefficient indicating the camera position and angle, and a maximum depth and a minimum Depth, and so on.

In addition, when one of the prediction methods other than the graph-based prediction method is used to compress the depth map, the optimum prediction method can be determined based on the predicted value of the distortion value generated in the synthesized image. That is, the rate distortion cost is calculated using the estimated distortion value when calculating the rate distortion cost for each prediction method according to the Lagrangian optimization method, and the prediction method having the least rate distortion cost is calculated as the optimal prediction method You can choose to use it.

7 is a block diagram for explaining an internal configuration of a depth map encoding system according to an embodiment of the present invention. 7, the depth map encoding system according to the present embodiment includes an image quality predicting unit 701, an encoding mode selecting unit 702, a graph-based predicting unit 703, a spatial predicting unit 704, A prediction unit 705, a predictive encoding unit 706, a transform and quantization unit 707, and an entropy encoding unit 708.

The picture quality predicting unit 701 predicts the picture quality of the picture to be synthesized through the depth map based on the depth map and the reference picture. At this time, the picture quality predicting unit 701 assigns a weight to the first pixel value of the reference picture at the same position as the pixel position of the depth map and the distortion value generated in the pixel value of the depth map at the position, The distortion value generated in the synthesized image can be calculated using the difference between the second pixel values of the positions shifted as much as possible.

In this case, as described above, the weights include information such as an internal coefficient such as a focal length of a camera used for generating a three-dimensional image, an external coefficient indicating a camera position and an angle, and a maximum depth and a minimum depth of the acquired image Lt; / RTI >

The encoding mode selection unit 702 selects an encoding mode based on the predicted image quality. At this time, the encoding mode selection unit 702 may calculate a rate-distortion cost based on the distortion value, and select an encoding mode having the calculated rate-distortion cost as a minimum value.

Here, the encoding mode may include two or more of a graph-based prediction mode, a spatial prediction mode, and a temporal prediction mode. That is, when the graph-based prediction mode is selected in the coding mode selecting unit 702, the graph-based predicting unit 703 selects the spatial prediction mode when the spatial prediction mode is selected, The time predicting unit 705 can operate.

In the graph-based prediction mode, an edge map is generated based on pixel value differences between neighboring pixels in an image to be synthesized, a pixel is connected through the generated edge map to generate a graph, And selecting a prediction value for at least one pixel. That is, the graph-based predicting unit 103 may include the edge map generating unit 101, the graph generating unit 102, and the predictor selecting unit 103 described with reference to FIG. Alternatively, the edge map encoding unit 104 may be further included. In this case, the predictive encoding unit 706, the transforming and quantizing unit 707, and the entropy encoding unit 708 are similar to the predictive encoding unit 104, the transform and quantization unit 105 and the entropy encoding unit 106, respectively.

8 is a flowchart illustrating a depth map encoding method according to an embodiment of the present invention. The depth map encoding method according to the present embodiment can be performed by the depth map encoding system described with reference to FIG. FIG. 8 illustrates a depth map encoding method by explaining a process in which each step is performed by the depth map encoding system.

In step 810, the depth map encoding system predicts the image quality of the image to be synthesized through the depth map based on the depth map and the reference image. At this time, the depth map encoding system assigns a weight to the first pixel value of the reference image at the same position as the pixel position of the depth map and the distortion value generated in the pixel value of the depth map at the position, The distortion value generated in the synthesized image can be calculated using the difference between the second pixel values at one position.

In this case, as described above, the weights include information such as an internal coefficient such as a focal length of a camera used for generating a three-dimensional image, an external coefficient indicating a camera position and an angle, and a maximum depth and a minimum depth of the acquired image Lt; / RTI >

In step 820, the depth map encoding system selects an encoding mode based on the predicted image quality. At this time, the depth map encoding system may calculate a rate-distortion cost based on the distortion value, and select an encoding mode in which the calculated rate distortion cost has a minimum value. Here, the encoding mode may include two or more of a graph-based prediction mode, a spatial prediction mode, and a temporal prediction mode.

In step 830, the depth map encoding system checks whether or not it is a graph based prediction mode. That is, the depth map encoding system may perform step 840 if the encoding mode selected in step 830 is a graph-based prediction mode, and step 850 if the encoding mode selected in step 830 is not a graph-based prediction mode.

In step 840, the depth map encoding system performs graph-based predictive encoding, and in step 850, the depth map encoding system performs spatial prediction encoding or temporal prediction encoding. At this time, step 840 for graph-based predictive encoding may include steps 401 to 403 described with reference to FIG.

In step 860, the depth map encoding system transforms and quantizes predictively encoded pixels, and in step 870, the depth map encoding system entropy encodes the transformed and quantized pixels to generate a bit stream. These steps 860 and 870 may correspond to steps 450 and 460 described in FIG. 4, respectively.

The contents omitted in Figs. 7 and 8 can be referred to the description of Fig. 1 to Fig.

9 is an example for explaining a method of estimating a distortion occurring in a composite image using distortion of a depth map in an embodiment of the present invention. When the image 902 acquired at the position of the camera p 901 is synthesized with the image 904 acquired at the position of the camera p '903, the pixel at the first position (x _im , y _im ) If no error has occurred in the depth map, it is mapped to the second position (x ' _im , y' _im ) 906. However, if an error occurs in the depth map, it is mapped to the third position (x ' _{im + tx} , y' _{im + ty} ) 907 by the displacement (tx, ty) Therefore, the pixel value of the first position ( _xim , _yim ) 905 and the pixel position of the third position ( _{x'im + tx} , _{y'im + ty} ) 907 in the image obtained at the position of the camera p (901) The distortion value can be calculated using the difference between the pixel values of the pixels.

As described above, according to the embodiments of the present invention, the pixel prediction is performed effectively according to the local information of the image using the graph, thereby improving the compression efficiency and improving the quality of the reconstructed image, , When the three-dimensional image is encoded through the image synthesis using the depth map, the optimized encoding mode can be determined by predicting the effect of the distortion value generated by loss-coding the depth map on the synthesized image.

The methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the present invention or may be available to those skilled in the art of computer software.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

101: Edge map generating unit
102:
103:
109: edge map coding unit

Claims (27)

An edge map generation unit for generating an edge map based on a pixel value difference between neighboring pixels in an input image;
A graph generating unit for generating a graph by connecting pixels through the generated edge map;
A predicted value selection unit for selecting a predicted value for at least one pixel based on the generated graph; And
A predictive encoding unit for predicting the at least one pixel based on the at least one pixel and the predictive value,
Lt; / RTI >
Wherein the edge map generator comprises:
When the pixel value difference is equal to or greater than a preset threshold value, determining that an edge exists between the neighboring pixels and displaying the edge map on the edge map,
The threshold value,
And adjusts the bit amount of the encoded edge map by adjusting an edge generation amount according to a target bit rate of the input image. delete delete An edge map generation unit for generating an edge map based on a pixel value difference between neighboring pixels in an input image;
A graph generating unit for generating a graph by connecting pixels through the generated edge map;
A predicted value selection unit for selecting a predicted value for at least one pixel based on the generated graph;
A predictive encoding unit that predictively encodes the at least one pixel based on the at least one pixel and the predicted value; And
An edge map encoding unit for encoding the generated edge map,
Lt; / RTI >
Wherein the edge map encoding unit comprises:
Compresses and encodes the edge map through binary arithmetic coding,
Wherein the encoded edge map is included in a bit string obtained by encoding the input image. The method according to claim 1,
Wherein the graph generating unit comprises:
And generates a graph by connecting a plurality of pixels that are not separated from each other by an edge of the edge map. The method according to claim 1,
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
And selects a pixel value of a pixel neighboring the block including the at least one pixel and closest to the at least one pixel among the pixels connected to the at least one pixel by the graph as the predicted value. The method according to claim 1,
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
Wherein the weighting unit assigns different weights to the pixel values of pixels adjacent to the block including at least one pixel and connected to the graph by using the at least one pixel, . An edge map generation unit for generating an edge map based on a pixel value difference between neighboring pixels in an input image;
A graph generating unit for generating a graph by connecting pixels through the generated edge map;
A predicted value selection unit for selecting a predicted value for at least one pixel based on the generated graph; And
A predictive encoding unit for predicting the at least one pixel based on the at least one pixel and the predictive value,
Lt; / RTI >
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
Wherein the weighting unit assigns different weights to the pixel values of pixels adjacent to the block including at least one pixel and connected to the graph by using the at least one pixel, , ≪ / RTI &
Wherein the weights are determined based on a spectral distribution of the graph. An edge map generation unit for generating an edge map based on a pixel value difference between neighboring pixels in an input image;
A graph generating unit for generating a graph by connecting pixels through the generated edge map;
A predicted value selection unit for selecting a predicted value for at least one pixel based on the generated graph; And
A predictive encoding unit for predicting the at least one pixel based on the at least one pixel and the predictive value,
Lt; / RTI >
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
Wherein the weighting unit assigns different weights to the pixel values of pixels adjacent to the block including at least one pixel and connected to the graph by using the at least one pixel, , ≪ / RTI &
Wherein the weights are determined using an Eigen vector of a Laplace matrix of the graph. The method according to claim 1,
Wherein the predictive-
Wherein the difference value between the pixel value of the at least one pixel and the predicted value is encoded. The method according to claim 1,
A transform and quantization unit for transforming and quantizing the predictively encoded pixel; And
An entropy encoding unit that entropy-codes the transformed and quantized pixels to generate a bitstream,
Further comprising: 12. The method of claim 11,
And a decoding unit for decoding the transformed and quantized pixels based on the predicted value,
Further comprising:
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
And selects a prediction value for the at least one pixel based on the pixels neighboring the block including the at least one pixel among the decoded pixels and the graph. An edge map decoding unit decoding the edge map in the inputted bit stream;
A graph generating unit for generating a graph by connecting pixels through the decoded edge map;
A predicted value selection unit for selecting a predicted value for at least one pixel based on the generated graph; And
A prediction compensating unit for restoring the at least one pixel based on the prediction value and the pixel information of the input bitstream,
Lt; / RTI >
The edge map includes:
When the difference between the pixel values is equal to or greater than a preset threshold value, it is determined that an edge exists between the pixels and is displayed on the edge map,
The threshold value,
Is adjusted at the time of edge generation so that the bit amount of the encoded edge map can be adjusted by adjusting the edge generation amount according to a desired bit amount of the input image. 14. The method of claim 13,
Wherein the edge map decoding unit comprises:
And decodes the edge map from the input bitstream through binary arithmetic decoding. 14. The method of claim 13,
Wherein the graph generating unit comprises:
And connecting the plurality of pixels that are not separated from each other by edges of the edge map to generate the graph. 14. The method of claim 13,
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
And selects the pixel value of a pixel neighboring the block including the at least one pixel and closest to the at least one pixel among the pixels connected to the at least one pixel by the graph as the predicted value. 14. The method of claim 13,
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
Wherein the weighting unit assigns different weights to the pixel values of pixels adjacent to the block including at least one pixel and connected to the graph by using the at least one pixel, . 14. The method of claim 13,
An entropy decoding unit that entropy-decodes the input bit stream to decode the pixel; And
An inverse transform and inverse quantization unit for inversely transforming and dequantizing the entropy-
Further comprising: 19. The method of claim 18,
Wherein the input image is classified into blocks of a predetermined pixel size,
Wherein the predicted value selector comprises:
And selects a predicted value for the at least one pixel based on the pixels neighboring the block including the at least one pixel among the inversely transformed and dequantized pixels and the graph. An image quality predicting unit for predicting an image quality of an image to be synthesized through the depth map based on the depth map and the reference image; And
And an encoding mode selection unit for selecting an encoding mode based on the predicted image quality,
Lt; / RTI >
Wherein the picture quality predicting unit,
A first pixel value of the reference image at the same position as a pixel position of the depth map and a second pixel at a position shifted by a displacement generated by weighting a distortion value generated in a pixel value of the depth map at the position, And calculates a distortion value occurring in the image to be synthesized using the difference between the values. 21. The method of claim 20,
Wherein the coding mode includes at least two of a graph-based prediction mode, a spatial prediction mode, and a temporal prediction mode. 22. The method of claim 21,
Wherein the graph-based predictive encoding mode comprises:
Generating an edge map based on a pixel value difference between neighboring pixels in the image to be synthesized, connecting the pixels through the generated edge map to generate a graph, and outputting the generated edge map to at least one pixel And a mode for selecting a predicted value for the depth map. delete 21. The method of claim 20,
Wherein the encoding mode selection unit comprises:
And calculates a rate-distortion cost based on the distortion value to select an encoding mode with the calculated rate distortion cost having a minimum value. Generating an edge map based on pixel value differences between neighboring pixels in an input image;
Generating a graph by connecting pixels through the generated edge map;
Selecting a prediction value for at least one pixel based on the generated graph; And
Predicting the at least one pixel based on the at least one pixel and the predicted value
Lt; / RTI >
Wherein the step of generating the edge map comprises:
When the pixel value difference is equal to or greater than a preset threshold value, determining that an edge exists between the neighboring pixels and displaying the edge map on the edge map,
The threshold value,
And adjusting the bit amount of the encoded edge map by adjusting an edge generation amount according to a desired bit amount of the input image. Decoding an edge map in an input bitstream;
Generating a graph by connecting pixels through the decoded edge map;
Selecting a prediction value for at least one pixel based on the generated graph; And
Restoring the at least one pixel based on the predictive value and the pixel information of the input bitstream
Lt; / RTI >
The edge map includes:
When the difference between the pixel values is equal to or greater than a preset threshold value, it is determined that an edge exists between the pixels and is displayed on the edge map,
The threshold value,
And adjusting the bit amount of the encoded edge map by adjusting an edge generation amount according to a desired bit amount of the input image. Estimating an image quality of an image to be synthesized through the depth map based on a depth map and a reference image; And
Selecting an encoding mode based on the predicted picture quality
Lt; / RTI >
Wherein the step of predicting the image quality comprises:
A first pixel value of the reference image at the same position as a pixel position of the depth map and a second pixel at a position shifted by a displacement generated by weighting a distortion value generated in a pixel value of the depth map at the position, And calculates a distortion value generated in the synthesized image using the difference between the values.

Images