KR20110071483A - Apparatus and method for processing depth image using bit depth reduction - Google Patents

Abstract

PURPOSE: An image processing apparatus and method using bit depth reduction are provided to reduce unnecessary bit used in the expression of a depth image. CONSTITUTION: A histogram analysis unit(130) analyzes the histogram of a depth image. According to the analysis result of the histogram of the depth image, a bit depth reduction unit(140) reduces the bit depth of the depth image. An image composition unit(180) restores the depth image in which the bit depth is reduced. The image composition unit composes the restored image and a color image.

Description

Depth image processing apparatus and method using bit depth reduction {APPARATUS AND METHOD FOR PROCESSING DEPTH IMAGE USING BIT DEPTH REDUCTION}

The following embodiments are related to a depth image processing apparatus and method using bit depth reduction.

In general, a stereoscopic image is a three-dimensional image that simultaneously provides shape information about depth and space. Unlike stereo, which provides images of different viewpoints to the left and right eyes, images taken from different viewpoints are required to provide the same image as seen from different directions whenever the viewer views different views. However, since images taken from various viewpoints have a large amount of data, it is almost impossible to compress and transmit them in consideration of network infrastructure and terrestrial bandwidth.

Instead of compressing and transmitting all the videos from multiple viewpoints, a depth image can be created and compressed together with the images of some viewpoints from multiple viewpoints to reduce the amount of data generated during compression. The depth image is an image representing a three-dimensional distance difference between objects in the image and is expressed as a value between 0 and 255 for each pixel in an 8-bit image. The larger the pixel value of the depth image, the closer the viewer is. Rather than compressing and transmitting all the images taken by multiple cameras, if you obtain a depth image for some color images and send the selected color image and the obtained depth image, the image can be synthesized at the desired time point after decoding at the receiving end. The amount of data generated at the time can be reduced.

Unlike the color image, since the depth image has many flat portions, the amount of generated bits is only about half of the color image even when compressed. In order to make a depth image, you can shoot using a depth camera.However, to create a depth image at multiple points of view, several depth cameras are needed, and depth cameras are more expensive than cameras for shooting a normal two-dimensional video. Instead of using a camera, a method of estimating a depth image from a given multiview image is often used.

Depth image processing apparatus according to an embodiment of the present invention is a histogram analysis unit for analyzing the histogram of the depth image, and a bit depth reduction unit for reducing the bit depth of the depth image according to the analysis result of the histogram of the depth image And an image synthesizer configured to reconstruct a depth image having the reduced bit depth and synthesize the color image.

According to an aspect of the present invention, the histogram analyzer may determine the amount of bit depth to be reduced according to the histogram distribution of the histogram by analyzing the histogram for the depth image.

In addition, according to an aspect of the present invention, the bit depth reducer is a depth image encoder which reduces the bit depth of the depth image according to the determined amount of the bit depth to be reduced and encodes the depth image having the reduced bit depth. It may further include.

Also, according to an aspect of the present invention, a bit for increasing a depth depth of the decoded depth image by an amount of bit depth reduced by the depth image decoder and the bit depth reducer to decode the coded depth image. It may further include a depth increasing portion.

In addition, according to an aspect of the present invention, after the bit depth is increased for the depth image of which the bit depth is reduced, a quantization error compensator for compensating a quantization error of an input depth image and a depth image having a reduced / increased bit depth is further included. It may include.

The depth image processing method may further include analyzing a histogram of a depth image, reducing a bit depth of the depth image according to an analysis result of the histogram of the depth image, and the bit. Reconstructing a depth image having a reduced depth and synthesizing it with a color image.

In addition, according to an aspect of the present disclosure, analyzing the histogram of the depth image may determine an amount of bit depth to be reduced according to a histogram distribution of the histogram by analyzing the histogram of the depth image.

Further, according to an aspect of the present invention, the step of reducing the bit depth is to reduce the bit depth for the depth image according to the determined amount of bit depth to be reduced, and to encode the depth image with the reduced bit depth It may further comprise a step.

According to an aspect of the present invention, the method may further include decoding the encoded depth image and increasing the depth depth of the decoded depth image by the amount of the reduced bit depth.

According to an aspect of the present invention, the method may further include compensating a quantization error of an input depth image and a depth image of which the bit depth is reduced / increased after increasing the bit depth of the depth image of which the bit depth is reduced. Can be.

The embodiment of the present invention can increase the compression efficiency of the depth image by reducing unnecessary bits used for representing the depth image.

In addition, the embodiment of the present invention can greatly reduce the bit depth of the depth image to compensate for the quantization error generated during the quantization process.

Hereinafter, with reference to the contents described in the accompanying drawings will be described in detail an embodiment according to the present invention. However, the present invention is not limited to or limited by the embodiments. Like reference numerals in the drawings denote like elements.

1 is a diagram illustrating a configuration of a depth image processing apparatus using bit depth reduction according to an embodiment of the present invention.

Referring to FIG. 1, the depth image processing apparatus 100 may include a color image encoder 110, a color image decoder 120, a histogram analyzer 130, a bit depth reducer 140, and a depth image encoder ( 150, a depth image decoder 160, a bit depth increaser 170, and an image synthesizer 180.

The color image encoder 110 encodes the input color image. For example, the color image encoder 110 may compress the input color image by encoding the input color image according to an encoding method such as H.264.

The color image decoder 120 decodes the encoded color image. For example, the color image decoder 120 may reconstruct the color image by decoding the compressed color image by a decoding method such as H.264.

The histogram analyzer 130 analyzes the histogram of the input depth image. That is, the histogram analyzer 130 determines the amount of bit depth to be reduced according to the histogram distribution of the histogram as a result of analyzing the histogram of the input depth image.

5 is a diagram illustrating an example of a histogram of a color image.

Referring to FIG. 5, the histogram for the color image is dense so that there are almost no spaces on the x-axis, indicating that almost all pixel values of 0 to 255 are present in one image. As such, as the bit depth in the color image increases, a greater number of colors are represented, so that the bit depth of the color image is not reduced.

6 is a diagram illustrating an example of a histogram of a depth image.

Referring to FIG. 6, a histogram for a depth image is a sparse sparse bar graph, which means that there are only pixels having some specific values among values 0 to 255 that can be represented by an 8-bit image in one image. When analyzing a histogram of a depth image generated by estimating a depth image from a multiview image, an 8-bit depth image may have a value of 0 to 255, but only a specific pixel brightness value. The 8-bit depth image has a value between 0 and 255, but the depth image obtained through the depth estimation has only about 50 pixel brightness values among the 0 to 255 pixel brightness values. Since only about 50 pixel brightness values exist in the bit depth image, unnecessary bits are used to represent the depth image. If only 50 pixel brightnesses among 0 to 255 pixel brightnesses exist in an image, only 20% (50 × 100/256) of the pixel brightnesses that an 8-bit image can represent is used. Since the depth image has low compression efficiency because the pixels in the image are inferior in correlation, the bit depth reduction may be used to make the histogram of the histogram more denser in order to increase the correlation of the histogram.

As such, the histogram analyzer 130 may determine that only some pixel values are used as a result of analyzing the histogram of the input depth image, and thus may determine the amount of bit depth to be reduced.

The histogram analyzer 130 compares the result of analyzing the histogram of the input depth image with a specific threshold to determine how much to reduce the bit depth of the input depth image. For example, as a result of analyzing the histogram of the depth image, the histogram analyzer 130 may determine a bit depth to be reduced as one bit when the pixel brightness number is greater than or equal to the first threshold and smaller than the second threshold. As another example, the histogram analyzer 130 may determine a bit depth to be reduced to 2 bits when the pixel brightness number is greater than or equal to the second threshold and smaller than the third threshold as a result of analyzing the histogram of the depth image. . As another example, the histogram analyzer 130 may determine, as a result of analyzing the histogram of the depth image, the bit depth to be reduced to 3 bits when the pixel brightness number is greater than or equal to the third threshold and smaller than the fourth threshold. have. As another example, the histogram analyzer 130 may determine a bit depth to be reduced to 4 bits when it is larger than or equal to the fourth threshold and smaller than the fifth threshold.

The bit depth reducer 140 reduces the bit depth of the depth image according to the analysis result of the histogram of the depth image. That is, the bit depth reduction unit 170 reduces the bit depth of the decoded depth image according to the determined amount of bit depth to be reduced.

7 is a diagram illustrating an example of a histogram after a bit depth is reduced to 7 bits of a depth image.

Referring to FIG. 7, when the determined bit depth to be reduced is 1 bit, the bit depth reducer 140 may reduce the bit depth to an 8-bit depth image and a 7-bit depth image. That is, when the bit depth to be reduced determined by the histogram analyzer 130 is 1 bit and the bit depth of the input depth image is 8 bits, the bit depth reducer 140 determines the bit depth of the input depth image. By reducing 1 bit, a depth image having a bit depth of 7 bits can be output. When the depth of the bit is reduced to a 7-bit depth image, since there are 50 brightnesses from 0 to 127, a pixel brightness value of 39% (50 × 100/128) is used to represent the 7-bit image, thereby improving bit efficiency. Increases.

8 is a diagram illustrating an example of a histogram after reducing the bit depth to 6 bits of a depth image.

Referring to FIG. 8, when the determined bit depth to be reduced is 2 bits, the bit depth reducer 140 may reduce the bit depth from the 8-bit depth image to the 6-bit depth image. That is, when the determined bit depth to be reduced is 2 bits and the bit depth of the input depth image is 8 bits, the bit depth reduction unit 140 reduces the bit depth of the input depth image by 2 bits to 6 bits. The depth image of the bit depth may be output.

9 is a diagram illustrating an example of a histogram after reducing a bit depth of a depth image to 5 bits.

Referring to FIG. 9, when the determined bit depth to be reduced is 3 bits, the bit depth reducer 140 may reduce the bit depth to an 8-bit depth image into a 5-bit depth image. That is, when the determined bit depth to be reduced is 3 bits, and the depth depth of the input depth image is 8 bits, the bit depth reduction unit 140 decreases the bit depth of the input image by 3 bits to 5 bits. The depth image of the bit depth may be output.

FIG. 10 is a diagram illustrating an example of a histogram after reducing a bit depth of a depth image to 4 bits.

Referring to FIG. 10, when the determined bit depth to be reduced is 4 bits, the bit depth reducer 140 may reduce the bit depth from the 8-bit depth image to the 4-bit depth image. That is, when the determined bit depth to be reduced is 4 bits, and the depth depth of the input depth image is 8 bits, the bit depth reduction unit 140 reduces the bit depth of the input depth image by 4 bits to 4 bits. An in-depth depth image may be output.

As the bit depth reduction unit 140 decreases the bit depth as shown in FIGS. 7 to 10, the distribution of the histogram histogram becomes more dense, which means that the bit efficiency for image representation is gradually increased.

The depth image encoder 150 receives a depth image of which the bit depth is reduced from the bit depth reducer 140 and encodes the depth image of which the bit depth is reduced.

The depth image decoder 160 decodes the encoded depth image.

The bit depth increasing unit 170 increases the bit depth of the decoded depth image. That is, the bit depth increasing unit 170 increases the bit depth of the decoded depth image by the amount of the bit depth reduced by the bit depth reduction unit 140. For example, when the bit depth increasing unit 170 decreases the bit depth reduced by the bit depth reduction unit 140 and the bit depth of the depth image output from the bit depth reduction unit 140 is 7 bits, By increasing the bit depth of the decoded depth image by one bit, a depth image having a bit depth of 8 bits may be output. As another example, when the bit depth increase unit 170 has a bit depth reduced by the bit depth reduction unit 140 and the bit depth of the depth image output from the bit depth reduction unit 140 is 6 bits, By increasing the bit depth of the decoded depth image by 2 bits, a depth image having a bit depth of 8 bits may be output. As another example, the bit depth increase unit 170 has a bit depth reduced by the bit depth reducer 140 and 3 bits, and a bit depth of the depth image output from the bit depth reducer 140 is 5 bits. The depth image having a bit depth of 8 bits may be output by increasing the bit depth of the decoded depth image by 3 bits. As another example, the bit depth increasing unit 170 may have 4 bits of the bit depth reduced by the bit depth reduction unit 140 and 4 bits of the depth image output from the bit depth reduction unit 140. The depth image having a bit depth of 8 bits may be output by increasing the bit depth of the decoded depth image by 4 bits. The depth image having the increased bit depth is a depth image reconstructed with the same bit depth as that of the input depth image.

The image synthesizer 180 synthesizes the decoded color image and the depth image of which the bit depth is increased. That is, the image synthesizing unit 180 outputs a stereoscopic image by synthesizing the decoded color image output from the color image decoding unit 120 and the depth image of which the bit depth is increased from the bit depth increasing unit 170.

As described above, the depth image processing apparatus 100 according to an embodiment of the present invention analyzes a histogram of a depth image to determine how much bit depth is to be reduced according to a specific threshold, thereby compressing the bit depth by reducing the quantization. The efficiency can be increased.

2 is a diagram illustrating a configuration of a depth image processing apparatus using bit depth reduction according to another exemplary embodiment of the present invention.

2, a depth image processing apparatus 200 using bit depth reduction according to another exemplary embodiment of the present invention may include a color image encoder 110, a color image decoder 120, a histogram analyzer 130, Bit depth reducer 140, first depth image encoder 150, first depth image decoder 160, first bit depth increaser 170, image synthesizer 180, and second bit depth increase The unit 210 includes a quantization error calculator 220, a second depth image encoder 230, a second depth image decoder 240, and a quantization error compensator 250. Here, the color image encoder 110, the color image decoder 120, the histogram analyzer 130, the bit depth reducer 140, the first depth image encoder 150, and the first depth image decoder ( 160, the first bit depth increasing unit 170 will not be described for the same operation as that shown in FIG. 1.

For example, when the size of the input depth image is '99' and the bit depth is reduced to 1/2, the bit depth reduction unit 140 reduces the depth of the input depth image to 1/2. The depth image having the reduced depth depth may be '49'.

The second bit depth increasing unit 210 increases the depth depth of the depth image in which the bit depth is reduced by the bit depth reducing unit 140. That is, the second bit depth increasing unit 210 may provide a depth depth for the depth image of which the bit depth output from the bit depth reduction unit 140 is reduced by the amount of bit reduction reduced by the bit depth reduction unit 140. To increase. For example, in the second bit depth increasing unit 210, the amount of bit reduction reduced by the bit depth reduction unit 140 is 1/2, and the size of the depth image output from the bit depth reduction unit 140 is increased. If the value is '49', the depth depth of the depth image having the reduced bit depth reduction output from the bit depth reduction unit 140 may be doubled to output a depth image having a size of '98'.

The quantization error calculator 220 calculates a quantization error by comparing the input depth image with the depth image output by the second bit depth increasing unit 210. For example, when the size of the input depth image is '99' and the size of the depth image output by the second bit depth increasing unit 210 is '98', the quantization error calculator 220 may determine the quantization error. Can be calculated as '1'.

The second depth image encoder 230 encodes the quantization error of the depth image calculated by the quantization error calculator 220.

The second depth image decoder 230 decodes a quantization error of the depth image encoded by the second depth image encoder 230.

The quantization error compensator 250 compensates the decoded quantization error output from the second depth image decoder 240 with respect to the depth image having the increased bit depth output from the first bit depth increaser 170. That is, the quantization error compensator 250 reconstructs the depth image more accurately by compensating for the quantization error generated by compressing the input image according to the bit sensitivity reduced by the bit depth reducer 140.

The image synthesizer 180 synthesizes the decoded color image output from the color image decoder 120 and the depth image compensated for from the quantization error compensator 250 and outputs an input image.

As described above, the depth image processing apparatus 200 according to another exemplary embodiment may greatly reduce the bit depth of the depth image to compensate for the quantization error generated during the quantization process.

3 is a diagram illustrating an operation flow of a depth image processing method using bit depth reduction according to an embodiment of the present invention.

1 and 3, in step 310, the histogram analyzer 130 analyzes a histogram of a depth image. That is, in step 310, the histogram analyzer 130 analyzes the histogram of the depth image to determine the amount of bit depth to be reduced according to the histogram distribution of the histogram. For example, in operation 310, the histogram analyzer 130 may analyze the histogram of the depth image, compare the pixel brightness number and the threshold value according to the histogram analysis result, and determine a bit depth to be reduced for the depth image. have.

In operation 320, the bit depth reducer 140 reduces the bit depth of the depth image according to the determined bit depth to be reduced.

In operation 330, the depth image encoder 150 encodes the depth image having the reduced bit depth.

In operation 340, the depth image decoder 160 decodes the encoded depth image.

In operation 350, the bit depth increasing unit 170 increases the bit depth of the decoded depth image. That is, in step 350, the bit depth increasing unit 170 increases the bit depth of the decoded depth image by the bit depth reduced by the bit depth reducing unit 140.

In operation 360, the color image encoder 110 encodes the input color image.

In operation 370, the color image decoder 120 decodes the encoded color image.

In operation 380, the image synthesizer 180 synthesizes the depth image having the increased bit depth and the decoded color image. That is, in operation 380, the image synthesizer 180 reconstructs the depth image restored to the original bit depth and the color restored by the color image decoder 120 as the bit depth is increased by the bit depth increase unit 170. Stereoscopic images are output by synthesizing the images.

As described above, the depth image processing method according to an exemplary embodiment of the present invention analyzes a histogram of a depth image to determine how much bit depth is reduced according to a specific threshold, and decreases the bit depth to quantize to increase compression efficiency. You can.

4 is a diagram illustrating an operation flow of a depth image processing method using bit depth reduction according to another exemplary embodiment of the present invention.

2 and 4, in step 411, the histogram analyzer 130 analyzes a histogram of a depth image. That is, in step 411, the histogram analyzer 130 analyzes the histogram of the depth image to determine the amount of bit depth to be reduced according to the histogram distribution of the histogram. For example, in step 411, the histogram analyzer 130 analyzes the histogram for the depth image, compares the pixel brightness number and the threshold value according to the histogram analysis result, and determines the amount of bit depth to be reduced for the depth image. You can decide.

In operation 412, the bit depth reducer 140 reduces the bit depth of the depth image according to the determined amount of bit depth to be reduced.

In operation 413, the first depth image encoder 150 encodes the depth image having the reduced bit depth. That is, in operation 413, the first depth image encoder 150 may further compress the depth image by performing a quantization process on the depth image according to the reduced bit depth.

In operation 414, the first depth image decoder 160 decodes the encoded depth image.

In operation 415, the first bit depth increasing unit 170 increases the bit depth of the decoded depth image. That is, in step 415, the first bit depth increasing unit 170 increases the bit depth of the decoded depth image by the amount of the bit depth reduced in step 412.

In operation 416, the second bit depth increasing unit 210 increases the bit depth of the depth image having the reduced bit depth. That is, in operation 416, the second bit depth increasing unit 210 increases the bit depth of the depth image having the bit depth reduced by the amount of the bit depth reduced in operation 412.

In operation 417, the quantization error calculator 220 calculates a quantization error by comparing the input depth image with the depth image having the increased bit depth. That is, in step 417, the quantization error calculator 220 compares the input depth image with the depth image in which the reduced bit depth is increased and calculates a quantization error based on the difference.

In operation 418, the second depth image encoder 230 encodes the depth image of the calculated quantization error.

In operation 419, the second depth image decoder 230 decodes the depth image of the encoded quantization error.

In operation 420, the quantization error compensator 250 compensates the quantization error by reflecting the depth image of the decoded quantization error in the depth image having the increased bit depth.

In operation 421, the color image encoder 110 encodes the color image.

In operation 422, the color image decoder 120 decodes the encoded color image.

In operation 423, the image synthesizer 180 synthesizes the depth image compensated for the quantization error and the decoded color image. That is, in operation 423, the image synthesis unit 180 outputs a stereoscopic image by synthesizing the depth image in which the quantization error is compensated by the operation 420 and the decoded color image in operation 422.

As described above, the depth image processing method according to another embodiment of the present invention can greatly reduce the bit depth of the depth image to compensate for the quantization error generated in the quantization process.

Depth image processing methods according to the present invention can be implemented in the form of program instructions that can be executed by various computer means can be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

1 is a diagram illustrating a configuration of a depth image processing apparatus using bit depth reduction according to an embodiment of the present invention.

2 is a diagram illustrating a configuration of a depth image processing apparatus using bit depth reduction according to another exemplary embodiment of the present invention.

3 is a diagram illustrating an operation flow of a depth image processing method using bit depth reduction according to an embodiment of the present invention.

4 is a diagram illustrating an operation flow of a depth image processing method using bit depth reduction according to another exemplary embodiment of the present invention.

5 is a diagram illustrating an example of a histogram of a color image.

6 is a diagram illustrating an example of a histogram of a depth image.

7 is a diagram illustrating an example of a histogram after a bit depth is reduced to 7 bits of a depth image.

8 is a diagram illustrating an example of a histogram after reducing the bit depth to 6 bits of a depth image.

9 is a diagram illustrating an example of a histogram after reducing a bit depth of a depth image to 5 bits.

FIG. 10 is a diagram illustrating an example of a histogram after reducing a bit depth of a depth image to 4 bits.

Claims (11)

A histogram analyzer analyzing the histogram of the depth image; A bit depth reduction unit that reduces the bit depth of the depth image according to an analysis result of the histogram of the depth image; And An image synthesizer which reconstructs the depth image having the reduced bit depth and synthesizes the color image Including, a depth image processing device. The method of claim 1, The histogram analysis unit, And analyzing the histogram for the depth image to determine the amount of bit depth to decrease according to the histogram distribution of the histogram. The method of claim 2, The bit depth reduction unit, Reduce the bit depth for the depth image according to the determined amount of bit depth to decrease, And a depth image encoder configured to encode the depth image having the reduced bit depth. The method of claim 3, wherein A depth image decoder to decode the encoded depth image; And And a bit depth increasing unit configured to increase the depth depth of the decoded depth image by an amount of the bit depth reduced by the bit depth reducing unit. The method of claim 1, And a quantization error compensator configured to compensate for the quantization error of the input depth image and the depth image of which the bit depth is reduced / increased after increasing the bit depth with respect to the depth image having the reduced bit depth. Analyzing the histogram of the depth image; Reducing the bit depth for the depth image according to an analysis result of the histogram of the depth image; And Restoring the depth image having the reduced bit depth and synthesizing it with a color image; Including, a depth image processing method. The method of claim 6, Analyzing the histogram of the depth image, And analyzing the histogram for the depth image to determine the amount of bit depth to decrease according to the histogram distribution of the histogram. The method of claim 7, wherein Reducing the bit depth, Reduce the bit depth for the depth image according to the determined amount of bit depth to decrease, And encoding a depth image having the reduced bit depth. The method of claim 8, Decoding the encoded depth image; And Increasing the depth depth for the decoded depth image by the amount of the reduced bit depth. The method of claim 6, And compensating for the quantization error of the input depth image and the depth image of which the bit depth is reduced / increased after increasing the bit depth for the depth image having the bit depth reduced. A computer readable recording medium having recorded thereon a program for performing the method of any one of claims 6 to 10.

Images