KR101642965B1 - Apparatus and method for processing depth image using bit depth reduction - Google Patents

Abstract

A depth image processing apparatus and method using bit depth reduction are disclosed. The depth image processing apparatus compresses the depth image by reducing the bit depth by analyzing the histogram of the depth image, restores the compressed depth image, increases the bit depth for the reduced depth image, and synthesizes the restored color image .

Bit depth, decrease, increase, depth image, quantization, error, compensation

Description

[0001] APPARATUS AND METHOD FOR PROCESSING DEPTH IMAGE USING BIT DEPTH REDUCTION [0002]

The following embodiments relate to a depth image processing apparatus and method using bit depth reduction.

In general, stereoscopic images are three-dimensional images that simultaneously provide shape information about depth and space. Unlike stereos, which provide images of different viewpoints on the left and right eyes, images taken at different viewpoints are needed to provide the same images viewed from different directions each time the observer views them. However, because the amount of data taken at various points of view is vast, compression and transmission are almost impossible to achieve when considering network infrastructure and terrestrial bandwidth.

Instead of compressing and transmitting all the video data at various points in time, a depth image is created and compressed and transmitted together with an image at a certain point in the images at a plurality of points of view. The depth image is a three-dimensional distance difference between objects in an image. The depth image is expressed as a value between 0 and 255 for each pixel in an 8-bit image. The larger the pixel value of the depth image, the closer it is to the viewer. Since depth images of some color images are obtained by compressing and transmitting all the images shot by various cameras, and the selected color image and the obtained depth image are transmitted, it is possible to synthesize images at a desired point after decoding by the receiving end, It is possible to reduce the amount of data generated.

Since the depth image has a lot of flat parts unlike the color image, the amount of generated bits is only half of the color image even if compressed. In order to create a depth image, it is possible to shoot using a depth camera. However, in order to create a depth image at a plurality of viewpoints, a plurality of depth cameras are required. Since a depth camera is expensive compared to a camera that shoots a normal 2D video, Instead of using a camera, a method of estimating a depth image in a given multi-view image is often used.

The depth image processing apparatus according to an embodiment of the present invention includes a histogram analyzing unit for analyzing a histogram of a depth image, a bit depth reducing unit for decreasing a bit depth for the depth image according to an analysis result of the histogram of the depth image, And an image synthesis unit for reconstructing the depth image with the reduced bit depth and synthesizing the restored depth image with a color image.

According to an aspect of the present invention, the histogram analyzing unit may analyze the histogram of the depth image to determine the amount of bit depth to be reduced according to the bar graph distribution of the histogram.

According to an aspect of the present invention, the bit depth reduction unit may further include a depth image encoding unit that reduces a bit depth for the depth image and a depth image with a reduced bit depth according to the determined amount of bit depth to be reduced .

According to an aspect of the present invention, there is provided a depth image decoding apparatus comprising: a depth image decoding unit for decoding the encoded depth image; and a bit depth increasing unit for increasing a depth depth of the decoded depth image by an amount of bit depth reduced by the bit depth reducing unit. And may further include a depth increasing portion.

According to an aspect of the present invention, there is provided a quantization error compensator that compensates a quantization error of an input depth image and a depth image whose bit depth is decreased / increased after increasing the bit depth of the depth image with the bit depth reduced .

According to another aspect of the present invention, there is provided a depth image processing method comprising the steps of: analyzing a histogram of a depth image; decreasing a bit depth of the depth image according to an analysis result of a histogram of the depth image; And reconstructing the depth image with the reduced depth to synthesize the restored depth image with the color image.

According to an aspect of the present invention, the step of analyzing the histogram of the depth image may analyze the histogram of the depth image to determine the amount of bit depth to be reduced according to the histogram distribution of the histogram.

According to an aspect of the present invention, the step of decreasing the bit depth may include decreasing the bit depth for the depth image according to the determined amount of bit depth to be reduced, and encoding the depth image with the reduced bit depth Step < / RTI >

According to an aspect of the present invention, the method may further include decoding the encoded depth image and increasing depth depth of the decoded depth image by the amount of the reduced bit depth.

According to an aspect of the present invention, there is further provided a method of compensating a quantization error of an input image and a depth image with a reduced / increased bit depth after increasing the bit depth of the depth image with the reduced bit depth .

Embodiments of the present invention can reduce the unnecessary bits used for depth image representation, thereby increasing the compression efficiency of the depth image.

In addition, the embodiment of the present invention can significantly reduce the bit depth of the depth image and compensate the quantization error generated in the quantization process.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. Like reference symbols in the drawings denote like elements.

1 is a block diagram of a depth image processing apparatus using a bit depth reduction according to an embodiment of the present invention.

1, the depth image processing apparatus 100 includes a color image encoding unit 110, a color image decoding unit 120, a histogram analyzing unit 130, a bit depth reducing unit 140, a depth image encoding unit 150, a depth image decoding unit 160, a bit depth increasing unit 170, and an image synthesizing unit 180.

The color image encoding unit 110 encodes the input color image. For example, the color image coding unit 110 may compress the input color image by encoding the input color image according to a coding scheme such as H.264.

The color image decoding unit 120 decodes the encoded color image. For example, the color image decoding unit 120 may recover a color image by decoding a color image compressed by a decoding method such as H.264 or the like.

The histogram analyzer 130 analyzes the histogram of the input depth image. That is, the histogram analyzer 130 determines the amount of bit depth to be reduced according to the bar graph distribution of the histogram as a result of analyzing the histogram of the input depth image.

5 is a diagram showing an example of a histogram for a color image.

Referring to FIG. 5, the histogram of the color image is so dense that the bar graph is so thick that there is almost no space in the x-axis, indicating that almost all the pixel values of 0 to 255 exist in one image. As described above, since the larger the bit depth in the color image, the larger the number of colors is represented, the bit depth for the color image is not reduced.

6 is a diagram showing an example of a histogram for a depth image.

Referring to FIG. 6, the histogram of the depth image is roughened, which means that there is only a pixel having a specific value from 0 to 255 that an 8-bit image can express in one image. When analyzing the histogram of the depth image generated by the method of estimating the depth image in the multi-view image, the 8-bit depth image has a value of 0 to 255, but often has only a specific pixel brightness value. The 8-bit depth image has a value between 0 and 255. However, the depth image value obtained through depth estimation has only about 50 pixel brightness values from 0 to 255 pixel brightness values. Since only about 50 pixel brightness values actually exist in the bit depth image, unnecessary bits are used for depth image representation. If only 50 pixel brightnesses among 0 to 255 pixel brightnesses exist in one image, only 20% (50 × 100/256) of the pixel brightnesses that 8-bit image can express is used. Since the depth image has a low compression efficiency because the correlation between the pixels in the image is low, the bit depth reduction can be used in order to increase the correlation of the histogram by making the histogram of the histogram more densely.

As such, the histogram analyzer 130 can determine that only some pixel values are used as a result of analyzing the histogram of the input depth image, and thus can determine the amount of bit depth to be reduced.

The histogram analyzer 130 compares the result of analyzing the histogram of the input depth image with a specific threshold to determine how much the bit depth for the input depth image is to be reduced. For example, the histogram analyzer 130 may determine the bit depth to be reduced to 1 bit when the pixel brightness is greater than or equal to the first threshold value and smaller than the second threshold value as a result of analyzing the histogram of the depth image. As another example, the histogram analyzing unit 130 may determine the bit depth to be reduced to 2 bits when the pixel brightness is greater than or equal to the second threshold value and smaller than the third threshold value as a result of analyzing the histogram of the depth image . As another example, the histogram analyzer 130 may determine the bit depth to be reduced to 3 bits when the pixel brightness is greater than or equal to the third threshold value and less than the fourth threshold value as a result of analyzing the histogram of the depth image have. As another example, the histogram analyzing unit 130 may determine the bit depth to be reduced to 4 bits if it is greater than or equal to the fourth threshold value and smaller than the fifth threshold value.

The bit depth reduction unit 140 reduces the bit depth for the depth image according to the analysis result of the histogram of the depth image. That is, the bit depth reduction unit 170 reduces the bit depth for the decoded depth image according to the determined amount of bit depth to be reduced.

7 is a diagram showing an example of a histogram after decreasing the bit depth to 7 bits in the depth image.

Referring to FIG. 7, when the determined bit depth to be reduced is 1 bit, the bit depth reduction unit 140 can reduce the bit depth by using a depth image of 8 bits and a depth image of 7 bits. That is, when the bit depth to be reduced determined by the histogram analyzer 130 is 1 bit and the bit depth of the input depth image is 8 bits, the bit depth reduction unit 140 calculates the bit depth for the input depth image By decrementing one bit, it is possible to output a 7-bit depth depth image. If the bit depth is reduced to 7-bit depth image, the pixel brightness of 50% is used to represent a 7-bit image because the brightness is 50 out of 0 to 127. Thus, .

8 is a diagram showing an example of a histogram after decreasing the bit depth by 6 bits in the depth image.

Referring to FIG. 8, when the determined bit depth to be reduced is 2 bits, the bit depth reduction unit 140 can reduce the bit depth by using a depth image of 8 bits and a depth image of 6 bits. That is, when the bit depth to be decreased is 2 bits and the bit depth of the input depth image is 8 bits, the bit depth reduction unit 140 reduces the bit depth for the input depth image by 2 bits, The depth image of the bit depth can be outputted.

9 is a diagram showing an example of a histogram after decreasing the bit depth by 5 bits in the depth image.

Referring to FIG. 9, when the determined bit depth to be reduced is 3 bits, the bit depth reduction unit 140 may reduce the bit depth by using a depth image of 8 bits and a depth image of 5 bits. That is, when the bit depth to be reduced is 3 bits and the depth depth of the input depth image is 8 bits, the bit depth reduction unit 140 reduces the bit depth for the input image by 3 bits, The depth image of the bit depth can be outputted.

10 is a diagram showing an example of a histogram after decreasing the bit depth by 4 bits in the depth image.

Referring to FIG. 10, when the determined bit depth to be reduced is 4 bits, the bit depth reduction unit 140 can reduce the bit depth to a depth image of 8 bits and a depth image of 4 bits. That is, the bit depth reduction unit 140 reduces the bit depth for the input depth image by 4 bits when the determined bit depth is 4 bits and the depth depth for the input depth image is 8 bits, A depth image of in-bit depth can be output.

As the bit depth reduction unit 140 reduces the bit depth as shown in FIGS. 7 to 10, the distribution of the histogram histogram becomes increasingly dense, which means that the bit efficiency for the image representation increases gradually.

The depth image encoding unit 150 receives the depth image whose bit depth is reduced from the bit depth reduction unit 140 and encodes the depth image whose bit depth is reduced.

The depth image decoding unit 160 decodes the depth image.

The bit depth increasing unit 170 increases the bit depth of the decoded depth image. That is, the bit depth increasing unit 170 increases the bit depth of the decoded depth image by the amount of the bit depth reduced by the bit depth reducing unit 140. For example, when the bit depth of the depth image output from the bit depth decreasing unit 140 is 7 bits, the bit depth increasing unit 170 determines that the bit depth of the depth image is 1 bit reduced by the bit depth reducing unit 140, By increasing the bit depth of the decoded depth image by 1 bit, a depth image having a bit depth of 8 bits can be output. The bit depth increasing unit 170 may be configured such that when the bit depth reduced by the bit depth reducing unit 140 is 2 bits and the bit depth of the depth image output from the bit depth reducing unit 140 is 6 bits, By increasing the bit depth of the decoded depth image by 2 bits, it is possible to output a depth image having a bit depth of 8 bits. As another example, the bit depth increasing unit 170 may be configured such that the bit depth reduced by the bit depth reducing unit 140 is 3 bits and the bit depth of the depth image output from the bit depth reducing unit 140 is 5 bits , And a depth image having a bit depth of 8 bits can be output by increasing the bit depth of the decoded depth image by 3 bits. As another example, the bit depth increasing unit 170 may be configured such that the bit depth reduced by the bit depth reducing unit 140 is 4 bits and the bit depth of the depth image output from the bit depth reducing unit 140 is 4 bits , And a depth image having a bit depth of 8 bits can be output by increasing the bit depth of the decoded depth image by 4 bits. The depth image with the increased bit depth is a depth image restored to the same bit depth as the bit depth of the input depth image.

The image synthesis unit 180 synthesizes the decoded color image and the depth image with the increased bit depth. That is, the image synthesizing unit 180 synthesizes the decoded color image output from the color image decoding unit 120 and the depth image with the increased bit depth from the bit depth increasing unit 170, thereby outputting the stereoscopic image.

As described above, the apparatus 100 for processing a depth image according to an exemplary embodiment of the present invention analyzes a histogram of a depth image to determine how much bit depth is to be reduced according to a specific threshold, thereby reducing bit depth and quantizing The efficiency can be increased.

2 is a block diagram of a depth image processing apparatus using bit depth reduction according to another embodiment of the present invention.

2, a depth image processing apparatus 200 using a bit depth reduction according to another exemplary embodiment of the present invention includes a color image encoding unit 110, a color image decoding unit 120, a histogram analyzing unit 130, The first depth image decoding unit 160, the first bit depth increasing unit 170, the image synthesizing unit 180, the second bit depth increasing unit 140, the first depth image encoding unit 150, the first depth image decoding unit 160, A second depth image decoding unit 240 and a quantization error compensating unit 250. The quantization error compensating unit 250 includes a quantization error calculating unit 220, a second depth image encoding unit 230, a second depth image decoding unit 240, Here, the color image encoding unit 110, the color image decoding unit 120, the histogram analyzing unit 130, the bit depth reducing unit 140, the first depth image encoding unit 150, the first depth image decoding unit 160, and the first bit depth increasing unit 170 will not be described in detail.

For example, the bit depth reduction unit 140 may reduce the depth depth of the input depth image by 1/2 when the input depth image size is '99' and the bit depth is reduced to 1/2, The size of the depth image with depth depth reduced to '49'.

The second bit depth increasing unit 210 increases the depth depth of the depth image with the bit depth reduced by the bit depth reducing unit 140. That is, the second bit depth increasing unit 210 determines the bit depth of the reduced depth image output from the bit depth reducing unit 140 by the amount of the reduced bit amount by the bit depth reducing unit 140, . For example, the second bit depth increasing unit 210 determines that the amount of bit reduction reduced by the bit depth reducing unit 140 is 1/2 and the size of the depth image output from the bit depth reducing unit 140 is In case of '49', the depth depth of the depth image with the reduced bit depth reduction output from the bit depth reduction unit 140 is doubled, and the depth image having the size of '98' can be output.

The quantization error calculator 220 compares the input depth image with a depth image output by the second bit depth increasing unit 210 to calculate a quantization error. For example, when the input depth image has a size of '99' and the depth image output by the second bit depth increasing unit 210 is '98', the quantization error calculator 220 calculates a quantization error Can be calculated as " 1 ".

The second depth image encoding unit 230 encodes the quantization error of the depth image calculated by the quantization error calculation unit 220. [

The second depth image decoding unit 230 decodes the quantization error of the depth image encoded by the second depth image encoding unit 230.

The quantization error compensating unit 250 compensates the decoded quantization error output from the second depth image decoding unit 240 for the depth image whose bit depth is increased from the first bit depth increasing unit 170. That is, the quantization error compensating unit 250 compensates the quantization error caused by compressing the input image according to the bit sensitivity reduced by the bit depth reducing unit 140, thereby more accurately restoring the depth image.

The image synthesizer 180 synthesizes the decoded color image output from the color image decoder 120 and the depth image compensated for the quantization error output from the quantization error compensator 250 to output an input image.

As described above, the depth image processing apparatus 200 according to another embodiment of the present invention can greatly reduce the bit depth of the depth image and compensate the quantization error generated in the quantization process.

3 is a flowchart illustrating a depth image processing method using bit depth reduction according to an embodiment of the present invention.

Referring to FIGS. 1 and 3, in step 310, the histogram analyzer 130 analyzes a histogram of a depth image. That is, in step 310, the histogram analyzing unit 130 analyzes the histogram of the depth image and determines the amount of bit depth to be reduced according to the histogram distribution of the histogram. For example, in step 310, the histogram analyzer 130 analyzes a histogram of the depth image, compares the number of pixels according to the histogram analysis result with a threshold value, and determines a bit depth to be reduced for the depth image have.

In step 320, the bit depth reduction unit 140 reduces the bit depth for the depth image according to the determined bit depth to be reduced.

In operation 330, the depth image encoding unit 150 encodes the reduced depth image.

In operation 340, the depth image decoding unit 160 decodes the encoded depth image.

In step 350, the bit depth increasing unit 170 increases the bit depth for the decoded depth image. That is, in step 350, the bit depth increasing unit 170 increases the bit depth for the decoded depth image by the bit depth decreasing unit 140.

In operation 360, the color image encoding unit 110 encodes the input color image.

In operation 370, the color image decoding unit 120 decodes the encoded color image.

In step 380, the image combining unit 180 synthesizes the depth image with the increased bit depth and the decoded color image. That is, in step 380, the image combining unit 180 extracts a depth image reconstructed at the original bit depth as the bit depth is increased by the bit depth increasing unit 170 and a depth image reconstructed by the color image decoding unit 120 And outputs stereoscopic images by synthesizing images.

As described above, in the depth image processing method according to an embodiment of the present invention, the histogram of the depth image is analyzed to determine how much bit depth is to be decreased according to a specific threshold, thereby decreasing the bit depth and quantizing, .

4 is a flowchart illustrating a depth image processing method using bit depth reduction according to another embodiment of the present invention.

Referring to FIGS. 2 and 4, in step 411, the histogram analyzer 130 analyzes the histogram of the depth image. That is, in step 411, the histogram analyzer 130 analyzes the histogram of the depth image and determines the amount of bit depth to be reduced according to the histogram distribution of the histogram. For example, in step 411, the histogram analyzer 130 analyzes the histogram of the depth image, compares the number of pixels according to the histogram analysis result with a threshold value, and calculates the amount of bit depth to be reduced for the depth image You can decide.

In step 412, the bit depth reduction unit 140 reduces the bit depth for the depth image according to the determined amount of bit depth to be reduced.

In step 413, the first depth image encoding unit 150 encodes the deep image with the reduced bit depth. That is, in step 413, the first depth image encoding unit 150 may compress the depth image by performing a quantization process on the depth image according to the reduced bit depth.

In step 414, the first depth image decoding unit 160 decodes the encoded depth image.

In step 415, the first bit depth increasing unit 170 increases the bit depth for the decoded depth image. That is, in step 415, the first bit depth increasing unit 170 increases the bit depth for the decoded depth image by the amount of the reduced bit depth in step 412.

In step 416, the second bit depth increasing unit 210 increases the bit depth for the depth image with the bit depth reduced. That is, in step 416, the second bit depth increasing unit 210 increases the bit depth for the depth image in which the bit depth is decreased by the amount of the bit depth reduced in step 412.

In step 417, the quantization error calculator 220 compares the input depth image with the depth image with the increased bit depth to calculate a quantization error. That is, in step 417, the quantization error calculator 220 compares the input depth image with the depth image which is increased again by the reduced bit depth, and calculates the quantization error by the difference.

In step 418, the second depth image encoding unit 230 encodes the depth image for the calculated quantization error.

In operation 419, the second depth image decoding unit 230 decodes the depth image of the encoded quantization error.

In operation 420, the quantization error compensator 250 compensates the quantization error by reflecting the depth image of the decoded quantization error on the depth image with the increased bit depth.

In step 421, the color image encoding unit 110 encodes the color image.

In step 422, the color image decoding unit 120 decodes the encoded color image.

In step 423, the image combining unit 180 combines the depth image with the quantization error compensated and the decoded color image. That is, in step 423, the image combining unit 180 outputs the stereoscopic image by synthesizing the depth image compensated for the quantization error by the step 420 and the decoded color image by the step 422.

As described above, the depth image processing method according to another embodiment of the present invention can greatly reduce the bit depth of the depth image and compensate the quantization error generated in the quantization process.

The depth image processing methods according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

1 is a block diagram of a depth image processing apparatus using a bit depth reduction according to an embodiment of the present invention.

2 is a block diagram of a depth image processing apparatus using bit depth reduction according to another embodiment of the present invention.

3 is a flowchart illustrating a depth image processing method using bit depth reduction according to an embodiment of the present invention.

4 is a flowchart illustrating a depth image processing method using bit depth reduction according to another embodiment of the present invention.

5 is a diagram showing an example of a histogram for a color image.

6 is a diagram showing an example of a histogram for a depth image.

7 is a diagram showing an example of a histogram after decreasing the bit depth to 7 bits in the depth image.

8 is a diagram showing an example of a histogram after decreasing the bit depth by 6 bits in the depth image.

9 is a diagram showing an example of a histogram after decreasing the bit depth by 5 bits in the depth image.

10 is a diagram showing an example of a histogram after decreasing the bit depth by 4 bits in the depth image.

Claims (11)

A histogram analyzer for analyzing a histogram of a depth image to determine an amount of bit depth to be reduced by comparing a pixel brightness number according to the histogram analysis result with a predetermined threshold value; A bit depth reduction unit for decreasing a bit depth for the depth image according to the determined amount of bit depth to be reduced; A depth image encoding unit for encoding the depth image with the reduced bit depth; And And an image synthesis unit for reconstructing the coded depth image and synthesizing the coded depth image with a color image, And a depth image processing unit. delete delete The method according to claim 1, A depth image decoding unit decoding the encoded depth image; And And a bit depth increasing unit for increasing the depth depth of the decoded depth image by an amount of the bit depth reduced by the bit depth reducing unit. The method according to claim 1, Further comprising a quantization error compensating unit for compensating for an input depth image and a quantization error of a depth image whose bit depth is decreased / increased after increasing the bit depth for the depth image of which the bit depth is decreased. Analyzing a histogram of the depth image and comparing the pixel brightness number according to the histogram analysis result with a predetermined threshold to determine an amount of bit depth to be reduced; Decreasing a bit depth for the depth image according to the determined amount of bit depth to be reduced; Encoding the depth image with the reduced bit depth; And And restoring the encoded depth image and synthesizing the restored depth image with a color image The depth image processing method comprising: delete delete The method according to claim 6, Decoding the encoded depth image; And Further comprising increasing the depth depth for the decoded depth image by the amount of the reduced bit depth. The method according to claim 6, Further comprising compensating for an input depth image and a quantization error of the depth image with the bit depth reduced / increased after increasing the bit depth for the depth image with the reduced bit depth. A computer-readable recording medium having recorded thereon a program for performing the method according to any one of claims 6, 9 and 10.

Images