KR20100102493A - Depth map coding method and apparatus using block-based adaptive bitplane coding - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a method and apparatus for improving image quality and coding efficiency of depth information when coding depth maps.

2. The technical problem to be solved by the invention

The present invention proposes a method of improving the quality and coding efficiency of a depth information map by maximizing the characteristics of the depth information map when coding the depth information map.

3. Summary of Solution to Invention

According to the present invention, a method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to be coded by dividing a depth information map into bitplane units Method of improving the quality and coding efficiency of the depth information map by using a method and apparatus.

4. Important uses of the invention

The present invention is used to improve coding efficiency and quality of a depth information map.

Description

Depth map coding method and apparatus using block-based adaptive bitplane coding

 The present invention relates to depth information map coding. More particularly, the present invention relates to a method and an apparatus for improving the quality and coding efficiency of depth information when coding a depth map.

As the interest of the sensational media has increased significantly, research on it is being actively conducted. As a related study, JVT (Joint Video Team of ISO / IEC JTC1 / SC29 / WG11 MPEG and ITU-T SG16 Q.6 VCEG) has been able to efficiently use multiple views from multiple cameras in July 2008. We have completed the multi-view video coding (H.264 / AVC Amendment 4) standard for coding and delivering to users. Also, the standard for 3D video is currently in progress in MPEG. 3D Video transmits N view images and N depth maps together and performs view interpolation using the depth map and images of more views than the view images received. This is how to create a. 3D video can support a three-dimensional display system, and a representative system is a free view-point TV (FTV).

Both multi-view video coding and the 3D Video standard provide a variety of viewpoints to bring a sense of realism to the user. Multi-view video coding, however, can only show a fixed number of views received from a fixed number of cameras to the user. In multi-view video coding, more cameras must be used to support more viewpoints, and more viewpoint data must be transmitted. However, there is a limit to the transmission bandwidth and storage capacity of existing transmission media (broadcast, communication, storage media), so there is a limit to delivering view data of many views.

The 3D video standard has emerged as a way to solve this problem. Unlike multi-view video coding, 3D video transmits a depth map in addition to the view point received from the camera, so that various points of view other than the view point can be deepened. By using the information map, the user can create the desired point of time to infinity. Therefore, 3D video does not need to transmit a large number of image viewpoints like multi-view video coding, and it has the advantage of saving bandwidth and storage space because only a few viewpoint images and depth information maps need to be transmitted.

In addition to encoding / decoding the received view image, 3D Video requires a process of generating a depth map, encoding / decoding a depth map, and generating a virtual view image using the depth map. . Therefore, the current 3D video standard is mainly studying how to create a depth information map and how to create a virtual viewpoint image using the depth information map. However, the method of coding the depth map is described in MPEG-C Part 3 in order to render the 3D image using the depth information in the plane space of the object closest to the real world in the picture and the distance of the object farthest from the real world. Only the coding method of parameters for transmitting the position of the plane space is defined, but the coding method of the depth information map itself is not yet defined. Therefore, it is necessary to study the depth information map coding method.

The depth information map expresses depth information corresponding to each pixel in a constant number of bits at the same resolution as the viewpoint image, using the distance between the camera and the actual object at the current viewpoint. As an example of the depth information map, FIG. 1 illustrates a depth information map (the right image of FIG. 1) of a "Breakdancers" image sequence (the left image of FIG. 1), which is a test sequence of multiview video coding of MPEG. In fact, the depth information map of FIG. 1 expresses depth information corresponding to each pixel in 8 bits and is represented as a larger value (brighter value) as it is closer to the camera.

An example of a method of obtaining the distance Z in the real world from each pixel using the depth information map of FIG. 1 is shown in Equation 1 below.

v is the depth information actually expressed in the depth information map of FIG. 1, and Z _far and Z _near are the parameters defined in the actual MPEG-C Part 3 and are the farthest parts (Z _far ) in the real world shown in the image. It represents the actual position of the Z _near . Therefore, the depth information represented in the depth information map is 2 ⁿ (n: the number of bits representing the depth information map, the depth information map of FIG. 1 and n in the above equation). = 8) equal expression.

Currently, it is common to use existing standard video coding methods such as H.264 / AVC or multi-view video coding as the coding method of depth information map, and the encoder structure of H.264 / AVC which is a standard video coding method is used. An example is shown in FIG. 2.

In the structure diagram of the H.264 / AVC encoder of FIG. 2, a unit for processing data is a macroblock having a size of 16 × 16 pixels, and an image is encoded in an intra mode or an inter mode to output a bitstream. In the Intra mode, the switch switches to Intra, and in the Inter mode, the switch switches to Inter. The main flow of the coding process is to first generate a prediction block for the input block image, then obtain the difference between the input block and the prediction block and code the difference. First, the generation of the prediction block is performed according to the intra mode and the inter mode. First, in intra mode, the prediction block is generated by performing spatial prediction using the neighboring coded pixels of the current block in the "intra prediction" process. In the inter mode, the reference image buffer is performed in the "motion prediction" process. A motion vector is found from a reference image stored in the reference image that is best matched with the currently input block, and then a prediction block is generated by performing "motion compensation" using the obtained motion vector. As described above, a residual block is generated by obtaining a difference between a currently input block and a prediction block, and then coding is performed on the residual block. Coding for the remaining blocks is performed in the order of "transformation", "quantization", and "entropy coding". First, in the "transform" process, the received residual block is input to perform transform and output a transform coefficient. In the "quantization" process, a quantized coefficient obtained by quantizing the input transform coefficient according to the quantization parameter is output. In the "entropy coding" process, the input quantized coefficients are output as a bitstream by performing entropy coding according to a probability distribution. Since H.264 / AVC performs inter-frame predictive coding, it is necessary to decode and store the current coded picture for use as a reference picture of a later input picture. Therefore, by performing "inverse quantization" and "inverse transformation" on the quantized coefficients, a reconstructed block is generated through a predictive image and an adder, and then a blocking artifact generated during the coding process is removed using a "deblocking filter". In the "reference video buffer".

An example of a decoder structure diagram of an existing standard video coding method, H.264 / AVC, is shown in FIG. 3.

In the H.264 / AVC decoder structure diagram of FIG. 3, a unit for processing data is a macroblock having a width of 16x16 pixels and a bitstream is input to decode and output a reconstructed image in intra mode or inter mode. do. In the Intra mode, the switch switches to Intra, and in the Inter mode, the switch switches to Inter. The main flow of the decoding process is to first generate a prediction block, and then decode the input bitstream to add a result block and a prediction block to generate a reconstructed block. First, the generation of the prediction block is performed according to the intra mode and the inter mode. First, in intra mode, a prediction block is generated by performing spatial prediction using neighboring coded neighboring pixel values of the current block in the "intra prediction" process, and stored in the reference image buffer using a motion vector in the inter mode. The prediction block is generated by searching for an area in the reference image and performing "motion compensation". In the "entropy decoding" process, the input bitstream is subjected to entropy decoding according to a probability distribution to output quantized coefficients. "Dequantize" the quantized coefficients and perform "inverse transform" to generate a reconstructed block through the predictive image and the adder, and then remove the blocking artifact through the "deblocking filter", and then remove the "reference image buffer." To save.

Although the high compression ratio can be obtained by using the video standard codec, this is a coding method designed to fit the characteristics of an actual general image, and has a disadvantage in that it cannot use the characteristics of the depth information map to the maximum. In general, an apparatus for synthesizing an image using a depth information map greatly affects the image quality according to the accuracy of the depth information map. Therefore, a coding method for obtaining an optimal efficiency using characteristics of the depth information map is required.

The present invention proposes a method of improving the quality and coding efficiency of a depth information map by maximizing the characteristics of the depth information map when coding the depth information map.

According to the present invention, a method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to be coded by dividing a depth information map into bitplane units A method and apparatus for improving the quality and coding efficiency of a depth information map are included.

The method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and a method of adaptively selecting a bitplane to code by separating a depth information map into bitplane units It has the effect of improving the image quality and coding efficiency.

Hereinafter, with reference to the accompanying drawings will be described embodiments of the present invention; In the following description of the embodiments of the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, terms to be described below are terms defined in consideration of functions in the embodiments of the present invention, which may vary according to intention or custom of a user or an operator. Therefore, the definition should be made based on the contents throughout the specification.

The depth information map has a rather gentle characteristic unlike the general real image, and it can be easily seen through FIG. 4. FIG. 4 is a three-dimensional graph representing the level of each pixel of the actual image and the depth information map of FIG. 1 (ie, the luminance component level in the real image, that is, the brightness level, and the depth level in the depth information map). In the graph of the actual image (left graph of Fig. 4), it can be seen that there is a severe change between the pixels, while the depth map (right graph of Fig. 4) shows a fairly gentle shape. can confirm.

In addition, the depth information map can be expressed in units of bit planes. The bit plane of the depth information map shows a relatively monotonous form compared to the bit plane of the real image. FIG. 5 shows a representation of a real image and a depth information map of FIG. 1 in units of bit planes. The bitplane of the real image (left of FIG. 5) has the most significant bit-plane (MSBP), but has a monotonous form information, but becomes more complicated toward the lower bitplane, and has the least significant bitplane (LSBP). From the least significant bit-plane, we can see that the three bitplanes are quite complex, almost in the form of white noise. However, although the bitplane of the depth information map (in FIG. 5) becomes more complicated from the highest bitplane to the lowest bitplane, it can be seen that the overall bitplane is considerably more monotonous than the actual bitplane. have. In addition, it can be seen that each bit plane of the depth information map maintains a certain shape. In general, when gray codes are applied to express bit planes, the similarity of neighboring bit values increases in each bit plane unit. The bit plane in which gray codes are applied to each level value of the depth information map is shown. 5, the same as the right side, it can be easily seen that the more monotonous form.

For example, FIG. 6 is a result of analyzing a bitplane image of a macroblock of an object boundary in a depth information map of a ballet image. The result of separating the actual macroblock and the actual macroblock into bitplanes, After applying the gray code, the result is divided into bit planes.

Looking at binary images obtained by dividing the macroblocks of the actual depth information map of FIG. 6 into bitplane units, it can be seen that binary images from the MSB bitplane to the LSB bitplane are monotonous. In particular, the MSB bitplane, the MSB-1 bitplane, and the MSB-3 bitplane show that the binary image matches perfectly, and the MSB-2 bitplane and the MSB-4 bitplane also show that the binary image matches perfectly. have. Comparing the binary image of the MSB, MSB-1, and MSB-3 bitplanes with the binary image of the MSB-2 and MSB-4 bitplanes, it can be seen that the binary images of the MSB-2, MSB-1, and MSB-4 bitplanes are completely inverted. Comparing the binary image of phosphorus with the binary image of MSB-6 bitplane, we can see that it is completely inverted. And it can be seen that the binary image of the LSB bitplane does not match the binary image of any other bitplane and has a completely different shape.

Therefore, when the gray code is applied to the macroblock of the actual depth information map of FIG. 6 and separated into bit plane units, the binary image of the current bit plane and the binary image of the bit plane higher than the current bit plane are displayed. In case of a perfect match, the binary image of the current bitplane is all '0 (0)', and the binary image of the current bitplane and the binary image of the bitplane one level higher than the current bitplane are completely inverted. In this case, all binary images of the current bitplane are '255 (1)'. As a result, when the gray codes are applied to the macroblocks of the actual depth information map of FIG. 6 and the binary images separated by bit plane units, the binary images of the MSB-1 bit plane are all '0 (0)'. The binary images of the MSB-2 bitplane, the MSB-3 bitplane, the MSB-4 bitplane, and the MSB-6 bitplane all become '255 (1)'.

Using this characteristic, gray code may be applied to the depth information map to be coded in bit plane units. When applying gray code to the macroblock of the actual depth information map of FIG. 6, when coding bit plane, MSB bit plane In the case of binary image of MSB-5 bitplane and LSB bitplane, the binary image compression method is used. In the case of other binary image of bitplane, all current binary image is '0 (0)'. You only need to code information about whether it is '255 (1)' or not. This bitplane coding method may have considerably superior coding efficiency than the DCT-based video compression method in which coding efficiency is poor at an object boundary part.

If loss of information occurs when the depth map is converted to gray code, the correct value cannot be restored when the gray code is restored to the existing depth map. Therefore, in order to allow loss of information for bit rate control in the "bitplane coding" process, coding should be performed while increasing the number of bitplanes to be coded in order from MSB bitplane to LSB bitplane. .

As another example, FIG. 7 illustrates the analysis of a bitplane image of a background partial macroblock in a depth information map of a ballet image. The result of separating the actual macroblock from the actual macroblock into a bitplane, and the actual macroblock Shows the result of applying a gray code to the bit plane and separating it into bit planes.

Looking at the binary images obtained by dividing the macroblocks of the actual depth information map of FIG. 7 in bitplane units, the binary images are monotonous but generally have a more complicated form than the binary images separated in bitplane units in FIG. 6. It can be seen that. The MSB bitplane and the MSB-1 bitplane are both '0 (0)' and the MSB-2 bitplane is '255 (1)', indicating that values exist from the MSB-2 bitplane. Able to know. Comparing the binary image of the MSB-4 bitplane with the binary image of the LSB bitplane, it can be seen that the binary image of the MSB-4 and LSB bitplanes is perfectly matched. Comparing with, we can see that the correspondence is completely reversed.

Accordingly, in the bitplanes in which gray coding is performed on the macroblocks of the actual depth information map of FIG. 7, the MSB, MSB-1, MSB-2, and MSB-4 bitplanes are all zeros (0). Since all are '255 (1)', only bitmap coding needs to send information on whether the binary image of the current bitplane is '0 (0)' or '255 (1)'. In the gray coded bitplane, binary images of the MSB-3, MSB-5, MSB-6, and LSB bitplanes should be coded using a binary image compression algorithm. However, since the macroblock of the actual depth information map of FIG. 7 is far from the camera, the pixel value is small and the correlation between the pixel values in the block is high. It may have higher coding efficiency than image coding.

Through this fact, there is a need for a method for properly utilizing the advantages of the bitplane unitary binary image coding method and the DCT-based video compression method, and a method for obtaining a desired bit rate in bitplane coding.

[Method of Invention]

In the present invention, in order to properly utilize the advantages of the bitplane unitary binary image coding method and the DCT-based video compression method, and to obtain a desired bitrate in bitplane coding, the block unit bitplane coding method and the conventional block unit video coding method The present invention proposes a method of adaptively selecting and coding and a method of adaptively selecting a bitplane to be coded by dividing the depth information map into bitplane units.

[Method] A method of adaptively selecting and coding a block unit bitplane coding method and an existing block unit video coding method, and separately coding a depth information map into bit plane units Bitplane  How to choose adaptively

<Encoder>

The method of adaptively selecting and coding a block unit bitplane coding method and a conventional block unit video coding method, and a method of adaptively selecting a bit plane to be coded by dividing a depth information map into bit plane units An example of an encoder structure diagram is shown in FIG. 8.

The encoder structure diagram of FIG. 8 is coded by dividing the input depth information map in MxN block units, and the depth information map block input in MxN block units is adaptively selected in a bitplane coding mode or an existing coding mode to perform coding. Becomes When the existing coding mode is selected, the portion marked as 'A' in FIG. 8 is performed, and 'A' receives the depth information map block and receives the existing video coding method (MPEG-1, MPEG-2, MPEG). -4 Part 2 Encode with Visual, H.264 / AVC, SVC, MVC, VC-1, AVS, KTA, etc. to output the bitstream. When the bitplane coding mode is selected, the part marked with 'B' is performed, and 'B' is a bitplane unit by receiving a depth information map block represented by n-bit, a reference image, and a quantization parameter. It encodes using a coding method and outputs a bitstream.

8 shows an encoder structure diagram of H.264 / AVC as an example of the existing coding mode 'A'. In the encoder structure diagram of 'A', a unit for processing data is a macroblock having a width of 16 × 16 pixels and encoding is performed in an intra mode or an inter mode. In the Intra mode, the switch inside 'A' is switched to Intra, and in the Inter mode, the switch is switched to Inter. The main flow of the coding process is to first generate a prediction block for the input macroblock, then obtain the difference between the input macroblock and the prediction block and code the difference. First, the generation of the prediction block is performed according to the intra mode and the inter mode. First, in intra mode, a prediction block is generated by performing spatial prediction using neighboring coded neighboring pixel values of the current macroblock in the "intra prediction" process, and in the inter mode, a reference picture stored in a reference picture buffer. Performs “motion prediction”, which is a process of finding a motion vector that finds the best match with the currently input block, and uses the obtained motion vector to predict the prediction block in the reference picture stored in the reference picture buffer. A prediction block is generated by performing a "motion compensation" process. As described above, a residual block is generated by obtaining a difference from a prediction block that generates the currently input block, and then coding is performed on the residual block. Coding for the remaining blocks is performed in the order of "transformation", "quantization", and "entropy coding". First, in the "transform" process, the received residual block is input to perform transform and output a transform coefficient. In the "quantization" process, a quantized coefficient obtained by quantizing the input transform coefficient according to the quantization parameter is output. In the "entropy coding" process, entropy coding is performed on the input quantized coefficients according to probability distribution and the result is output. The result is input to "MUX" and output as a bitstream. Since H.264 / AVC performs inter-frame predictive coding, it is necessary to decode and store the current coded picture for use as a reference picture of a later input picture. Therefore, by performing "inverse quantization" and "inverse transformation" on the quantized coefficients, a reconstructed block is generated through a predictive image and an adder, and then a blocking artifact generated during the coding process is removed using a "deblocking filter". In the "reconstructed video buffer".

In FIG. 8, the bitplane coding mode 'B' shows an encoder structure diagram of the bitplane unit coding method. In the encoder structure diagram of 'B', a unit for processing data is a block having a size of horizontal and vertical MxN pixels.

Here, when coding the depth information map, various units for processing data may be applied.

(1) As one embodiment, a block of MxN pixel size is divided into small blocks to form a sub block, and the configured sub blocks may be coded in different methods or different block modes.

(2) As another example, MxN pixel-sized blocks may be coded by combining them into large blocks or larger areas.

(3) As another embodiment, the combination of the above methods (1) and (2) may be coded into blocks of various sizes.

In addition, when the depth information map is coded as described above, a unit for processing data may be changed in various ways.

In the “gray code conversion” process of FIG. 8, the depth information map block is input and each pixel is converted into a gray code. Gray code is a code for changing only one bit when expressing successive values. In general, the depth information map has characteristics very similar to the surrounding pixels. When the depth information map is divided into bit planes, the pixels of each bit plane may have different values regardless of the similarity of the pixels. For example, assuming that two consecutive pixels in the depth information map represented by 8-bit have values of '127' and '128', '127' is represented by binary (01111111) ₂ and '128' is represented by It is represented as (10000000) ₂ in binary. In this case, the depth information itself has a similar value, but when the comparison is performed for each bit plane, it can be seen that all bits have different values. However, when the depth information map is changed to gray code, only 1-bit is different when the actual value is '1', and thus the similarity of neighboring bit values in each bit plane can be increased. The method of changing the binary values (a _m-1 … a ₂ a ₁ a ₀ ) ₂ of the actual m-bit depth information pixels to gray codes (g _m-1 … g ₂ g ₁ g ₀ ) ₂ is shown in Equation 2 below. Same as

는 XOR(eXclusive OR) 연산을 의미한다. 실제 '127'과 '128'의 값을 그레이 코드로 변환을 수행하면 '127'은 이진수로 (11000000)₂ 로 표현이 되며, '128'은 이진수로 (01000000)₂ 로 표현이 된다. 그레이 코드로 변환을 수행하였을 때 각 비트의 유사도가 증가한 것을 쉽게 확인할 수 있다. "그레이 코드 변환" 과정은 수행될 수도 있고 수행되지 않을 수도 있다.From here

Means an XOR (eXclusive OR) operation. If you convert the values of '127' and '128' to gray code, '127' is represented by binary (11000000) ₂ and '128' is represented by binary (01000000) ₂ . When the conversion to the gray code is performed, it can be easily seen that the similarity of each bit is increased. The "gray code conversion" process may or may not be performed.

In the “bitplane separation” process of FIG. 8, a depth information map block represented by n-bit is received and divided into n bitplane blocks. In the “bit rate adjustment” process of FIG. 8, a bit rate adjustment function may be selectively used when an input depth information map block is needed. The bit rate to be coded is adjusted to obtain a desired bit rate in actual coding. The description of "bit rate adjustment" will be described later in detail. Here, after the "bit plane separation" process is performed, the "bit rate adjustment" process may be performed, or after the "bit rate adjustment" process, the "bit plane separation" process may be performed. The separated bitplane blocks that have undergone the "bitplane separation" process and the "bitrate adjustment" process are inputted repeatedly by the "bitplane coding" process to output the bitstream as a result of performing "bitplane coding". do.

The reference picture stored in the reference picture buffer of FIG. 8 and the reconstructed picture stored in the reconstructed picture buffer are input through a process of "bitplane coding." In the “bitplane coding” process of FIG. 8, each bitplane is encoded to output a bitstream. How to perform "bitplane coding" is described in detail later.

In FIG. 8, bitplane coding is selected to perform "bitplane coding", and the coded data performs "bitplane decoding" for coding of a depth information map input later. The reference picture stored in the reference picture buffer of FIG. 8 and the reconstructed picture stored in the reconstructed picture buffer are input through a process of "bitplane decoding." In the "bitplane decoding" process, the bitstream is input and repeatedly performed as many as the number of bitplanes. The bitstream is decoded for each bitplane to output n bitplane images. How to perform "bitplane decoding" is described in detail later. In the "bitplane combining" process, each of the output bitplane images is combined to output an image represented by n-bit. In the "inverse gray code conversion" process, a gray code (g _m-1 … g ₂ g ₁ g ₀ ) ₂ of an actual m-bit depth information pixel is restored by restoring a depth information map expressed in gray code to its original form. The method of changing the value (a _m-1 ... a ₂ a ₁ a ₀ ) ₂ is shown in Equation 3 below.

XOR(eXclusive OR) 연산을 의미한다.From here

This means an XOR (eXclusive OR) operation.

In the "depth information map construction" process, an image represented by n-bit is configured according to the number of bits of the actual depth information map, and the depth information map is output.

Here, the method of constructing the depth information map may be variously applied.

(1) As one embodiment, in order to configure the current block according to the number of bits of the actual depth information map, all values of the uncoded bitplane block may be set to '0 (0)'.

(2) As another embodiment, in order to configure the current block according to the number of bits of the actual depth information map, all values of the uncoded bitplane block may be set to '255 (1)'.

(3) As another example, in order to configure the current block according to the number of bits of the actual depth information map, whether all uncoded bitplane blocks are configured as '0 (0)' or all as '255 (1)'. By coding information on whether or not to configure a non-coded bitplane block. Information on how to construct an uncoded bitplane block may be included in the bitstream.

(4) As another example, in order to configure the current block according to the number of bits of the actual depth information map, padding using the pixel values of the neighboring bitplane blocks adjacent to the block to the uncoded bitplane block Can be configured through the Here, padding using pixel values of the neighboring bitplane blocks may have various padding directions, and information on the corresponding padding direction is included in the bitstream.

In addition, the method of constructing the depth information map as described above may be changed in various ways.

The constructed depth information map is used for intra prediction in the existing coding mode 'A' or stored in the reconstructed image buffer through a deblocking filter. In this case, information about whether the deblocking filter is performed on the current block may be coded, and the deblocking filter may be performed based on the information. The depth map configured here may be stored in the reconstructed image buffer without passing through the deblocking filter. In addition, the configured depth information map may be used for intra prediction after passing through a deblocking filter.

As an example of a method of performing "bitplane coding", the method used in binary shape coding of the international video standard MPEG-4 Part 2 Visual (ISO / IEC 14496-2) can be used. have. An example of the proposed "bitplane coding" is shown in FIG. 9.

The current bitplane block input in FIG. 9 means a bitplane block to be coded in a depth information map block to be currently coded. The reference image stored in the reference image buffer of FIG. 8 and the reconstructed image stored in the reconstructed image buffer of FIG. 8 are each applied with gray codes through a "gray code conversion" process and a "bitplane separation" process. A reference bitplane image of the same bitplane, which is separated into a bitplane of and is identical to the bitplane to be currently coded, is input in a "motion prediction" and a "CAE encoding" process.

In the "motion prediction" process of FIG. 9, the portion most similar to the current bitplane block is searched in the reference bitplane image to calculate and output a motion displacement, that is, a motion vector with a region that matches best. In the "motion compensation" process, a motion compensated bitplane block is output from the reference bitplane image using the motion vector generated in the "motion prediction" process.

In the "mode decision" process, the mode of the current bitplane block is determined through the current bitplane block and the motion-compensated bitplane block, and the coding mode of the current depth information map frame is used to determine the time direction prediction. Intra picture mode determination method used when not performed, and inter picture mode determination method used when the coding mode of the current depth information map frame performs time direction prediction is performed. The detailed description will be described later in detail.

The mode of the bitplane block determined here can be coded in various ways.

(1) As one example, the mode of the current bitplane block may be coded through fixed length encoding.

(2) As another example, the mode of the current bitplane block may be coded through variable length encoding using a probability according to the frequency of appearance of the bitplane block mode. In this case, coding may be performed using a pre-calculated probability table.

(3) As another example, the mode of the current bitplane block is coded through an arithmetic coding method using probabilities that are adaptively changed according to the situation in consideration of the bitplane block mode around the current bitplane block. can do.

In addition, the method of coding the bitplane block mode as described above may be changed in various ways.

The "Context-based Arithmetic Encoding (CAE) coding" process is performed when the pixel values of the current bitplane block are not all '0' or '1', and the same position as the current bitplane block in the reference bitplane. Is performed when the error between the reference bitplane block of or the reference bitplane block that has performed motion compensation exceeds the allowable error range, and each pixel in the current bitplane block is in the periphery of each pixel in the intra mode. In the inter mode, binary arithmetic coding is performed based on the pixel information of each pixel and the pixel of the reference bitplane image corresponding to the current pixel and its surrounding pixel information. To perform. A detailed description of the "CAE coding" method will be given later. The "multiplexer" receives a motion vector of the current bitplane block, a mode of the current bitplane block, and a "CAE coding" result, and generates a "bitstream".

The flowchart of an embodiment of the intra picture mode determination method in the “mode determination” portion of FIG. 9 is shown in FIG. 10, and a detailed algorithm is as follows.

Step 1) If all pixel values of the current bitplane block are the same, the process branches to Step 2. If not, branch to Step 3.

Step 2) If all pixel values of the current bitplane block are '1', the mode of the current bitplane block is set to 'all_1'. If not, set the mode of the current bitplane block to 'all_0'.

Step 3) Set the current bitplane block mode to 'intraCAE'. The coding method of the 'intraCAE' mode will be described later in detail.

9 is a flowchart of an embodiment of the inter-picture mode determination method in the “mode determination” portion of FIG. 9, and a detailed algorithm is as follows.

Step 1) If the error (A _err ) between the reference bitplane block and the current bitplane block corresponding to the prediction value (MV _P ) of the motion vector of the current bitplane block is equal to or less than the tolerance range (B _err ). Branch to Step 2. If not, branch to Step 3.

Step 2) Set the current bitplane block mode to 'No Update Without MV'. In the 'No Update Without MV' mode, the reconstructed bitplane block of the current bitplane block is a reference bitplane block corresponding to the prediction value (MV _P ) of the motion vector from the periphery of the current bitplane block. Only the mode information of the plain block is transmitted, but no additional data is transmitted. The method of determining the predicted value (MV _P ) of the motion vector will be described in detail later.

Step 3) If all pixel values of the current bitplane block are the same, the process branches to Step 4. If not, branch to Step 5.

Step 4) If all pixel values of the current bitplane block are '1', set the mode of the current bitplane block to 'all_1'. If not, set the mode of the current bitplane block to 'all_0'.

Step 5) Perform motion prediction.

Step 6) If the error C _err between the reference bitplane block corresponding to the motion vector calculated in the motion prediction and the current bitplane block is equal to or smaller than the allowable error range B _err , the process branches to Step 7. If not, branch to Step 8.

Step 7) Set the mode of the current bitplane block to 'No Updata with MV'. 'No Updata with MV' mode uses the reconstructed bitplane block of the current bitplane block as a reference bitplane block corresponding to a motion vector calculated through motion prediction, and the mode information and motion of the current bitplane block. Only send vectors, no additional data

Step 8) Perform 'intraCAE' mode coding and 'interCAE' mode coding. The coding method of 'intraCAE' and 'interCAE' modes will be described later in detail.

Step 9) If the bit amount of the 'intraCAE' mode coding result is smaller than the bit amount of the 'interCAE' mode coding result, the process branches to Step 10. If not, branch to Step 11.

Step 10) Set the mode of the current bitplane block to 'intraCAE'. The 'intraCAE' mode coding method will be described later.

Step 11) Set the mode of the current bitplane block to 'interCAE'. The 'Inter CAE' mode coding method will be described later.

A flowchart of an embodiment of a method of determining a motion vector prediction value (MV _P ) of a current bitplane block is shown in FIG. 12. A detailed algorithm is as follows.

Step 1) If the left boundary of the current bitplane block is not the left boundary of the image and the left bitplane block mode (LeftMBMode) performs the time direction prediction in the current bitplane block, go to Step 2 Branch. If not, branch to Step 3.

Step 2) The motion vector prediction value MV _P of the current bitplane block is set to the motion vector MV _Left of the left bitplane block in the current bitplane block.

Step 3) If the upper boundary of the current bitplane block is not the boundary of the image (AboveBoundary), branch to Step 4. If the upper boundary of the current bitplane block is the boundary of the image (AboveBoundary), branch to Step 8.

Step 4) If the upper boundary of the current bitplane block is not the boundary of the image (AboveBoundary) and the upper bitplane block mode (AboveMBMode) performs the time direction prediction in the current bitplane block, go to Step 5 Branch. If not, branch to Step 6.

Step 5) The motion vector prediction value MV _P of the current bitplane block is set to the motion vector MV _Above of the upper bitplane block in the current bitplane block.

Step 6) The right or upper boundary of the current bitplane block is not the boundary of the image (AboveRightBoundary), and the upper right bitplane block mode (AboveRightMBMode) performs the time direction prediction (inter) in the current bitplane block. If yes, branch to Step 7. If not, branch to Step 8.

Step 7) The motion vector prediction value MV _P of the current _bitplane block is set to the motion vector MV _AboveRight of the upper right _bitplane block in the current _bitplane block.

Step 8) The motion vector prediction value MV _P of the current bitplane block is set to '0' for both the horizontal component value and the vertical component value.

In addition, the method of determining the motion vector prediction value MV _{P of the} bitplane block may be variously applied.

As an example, the motion vector prediction value MV _p of the current bitplane block may be determined using motion vectors of a bitplane block or a depth information map block adjacent to the current bitplane block.

(2) As another example, in the flowchart of FIG. 12, the motion vector prediction value of the current bitplane block (MV _p) is used by using the motion vector of the depth information map block instead of the neighboring bitplane blocks adjacent to the current bitplane block. ) Can be determined.

(3) As another embodiment, the motion vector MV _{Left of the left} bitplane block or the depth information map block and the motion vector MV _Above of the upper bitplane block or depth information map block in the current _bitplane block, The motion vector prediction value MV _p of the current _bitplane block may be determined using a median value of the motion vector MV _RightAbove of the upper right _bitplane block or the depth information map block. Here, if the upper right bit plane of the block or the depth information of the map block motion vector (MV _RightAbove) is not available in can be used instead of a motion vector (MV _LeftAbove) at the left upper bit plane or block depth map information block.

In addition, the method of determining the motion vector prediction value MV _{P of the} bitplane block as described above may be changed in various ways.

When the mode of the current bitplane block determined in the "mode determination" part of FIG. 9 is selected as 'intraCAE' and 'interCAE', a context-based arithmetic encoding (CAE) method is used. An example of the structure diagram of "encoding" is shown in FIG.

The process of “bit rate adjustment” of FIG. 13 is performed through a size conversion method when the current bitplane block mode is an 'intraCAE' mode and an 'interCAE' mode. It is created by down-sampling the current bitplane block to the conversion ratio (CR) 1/2 or 1/4 and then up-sampling it back to the current bitplane block size. Calculates the error between one bitplane block and the current bitplane block, and if the error is less than or equal to the tolerance, the block created by downsampling the current bitplane block to 1/2 or 1/4 Perform coding on. Otherwise, code the current bitplane block as it is. The flowchart for setting the conversion ratio is shown in FIG. 14, and the detailed algorithm is as follows.

Step 1) Set the conversion ratio (CR) to 1/4 for the current bitplane block, perform size conversion, that is, 1/4 downsampling and upsampling again, according to the conversion ratio of the current bitplane block. Create a bitplane block.

Step 2) The error (D _err ) between the current bitplane block and the block that downsampled to the conversion bit rate (CR) 1/4 in the current bitplane block and upsampled again is larger than the allowable error range (B _err ). Branch to Step 3. If not, exit the algorithm.

Step 3) Set the conversion ratio (CR) to 1/2 for the current bitplane block, perform size conversion, that is, half downsampling and upsampling again according to the conversion ratio of the current bitplane block. Create a block.

Step 4) Tolerance (B _err ) is the error (E _err ) between the current _bitplane block and the _bitplane block downsampled to the current bitplane block by the conversion ratio (CR) 1/2 and then upsampled again. If greater than the range, branch to Step 5. If not, exit the algorithm.

Step 5) Set the conversion ratio (CR) to 1 for the current bitplane block and terminate the algorithm.

In the process of “downsampling” in FIG. 13, downsampling of a reference bitplane block of a reference bitplane image corresponding to the current bitplane block according to the conversion ratio CR output in the “bit rate adjustment” process is performed. sampling so that it can be used for context calculation. If the conversion ratio CR is '1', downsampling is not performed.

In the "context calculation" process of FIG. 13, when the mode of the current bitplane block is the 'intraCAE' mode, the current bitplane block is based on pixel values around the pixels to be coded in the current bitplane block. A context template is configured by using pixel values around the pixel to be coded in the bitplane block, pixels of the reference bitplane image corresponding to the pixel to be coded in the current block, and pixel values thereof. In the "arithmetic encoding" process, a bitstream is generated by performing arithmetic encoding on a pixel to be currently coded by referring to a probability table having an index of a context template configured in the "context calculation" process.

The pixels used when constructing the context template in 'intraCAE' mode are as shown in FIG. 15 and 10-bit context templates in the form of 10 peripheral pixels (c9 c8 c7 c6 c5 c4 c3 c2 c1 c0) around the current pixel (X). After is formed, it is used as an index of the probability table in arithmetic coding. The pixels used when configuring the context template in the 'interCAE' mode are as shown in FIG. 16, four peripheral pixels around the current pixel X, and a pixel corresponding to the current pixel in the reference block corresponding to the current block. c6) and its surrounding four pixels form a 9-bit context template of the form (c8 c7 c6 c5 c4 c3 c2 c1 c0) and then use it as an index of the probability table in arithmetic coding.

Here, there are various ways to configure the context template in the 'intraCAE' mode and the 'interCAE' mode.

(1) As one example, in the 'intraCAE' mode, the context template may be configured using various numbers of pixels among neighboring pixels adjacent to the pixel X to be coded in the bitplane block. In an embodiment, in the 'intraCAE' mode, four neighboring pixels around the current pixel X may be configured to form a 4-bit context template in the form of (c3 c2 c1 c0).

(2) In another embodiment, in the 'interCAE' mode, a reference corresponding to pixels X to be neighbored adjacent to pixel X to be coded in the current bitplane block and to pixel X to be coded in the current bitplane block. The context template may be configured by using various numbers of pixels among the pixels of the bitplane image and neighboring pixels. In an embodiment, in the 'interCAE' mode, four neighboring pixels adjacent to the current pixel X may be configured to form a 4-bit context template in the form of (c3 c2 c1 c0).

In addition, the method of configuring the context template as described above may be changed in various ways.

In FIG. 13, "context calculation" and "arithmetic encoding" are repeated by the number of pixels of the current bitplane block or downsampled bitplane block. In this case, the order of coding the pixels of the current bitplane block can be adaptively performed from the top to the bottom to the left to the horizontal scan or the vertical scan. After performing both scan modes, the scan with less bit amount is performed. The mode is selected and the scan mode information is stored in the bitstream.

Here, the method of scanning the pixels of the current bitplane block can be applied in various ways.

(1) As one embodiment, as illustrated in FIG. 17A, pixels of the current bitplane block may be scanned from left to right from top to bottom or from top to bottom from left to right.

(2) As another example, as illustrated in (b) of FIG. 17, the pixels of the current bitplane block may be scanned from top to bottom or zigzag from right to left and then zigzag again from top to bottom. You can scan from left to right in zigzag order from bottom to top.

(3) As another example, as shown in (c) of FIG. 17, the pixel of the current bitplane block is scanned from the top left to the bottom right in the diagonal direction or the bottom left to the bottom right in the diagonal direction from the bottom left. Can be done.

(4) As another example, as shown in (d) of FIG. 17, the pixels of the current bitplane block are scanned from top left to top diagonally in zigzag order from top left to bottom right or diagonally zigzag from bottom left. (zigzag) You can scan from bottom left to top right.

In addition, the method of scanning the pixels of the current bitplane block as described above may be changed in various ways.

Another example of the method of performing "bitplane coding" of FIG. 9 is a method of applying the binary shape coding method of the above-described MPEG-4 Part 2 Visual.

① As an example of applying the binary shape coding method of MPEG-4 Part 2 Visual, coding may be performed by variously applying a conversion ratio (CR) of the “bit rate adjustment” part.

② As another example of a method of applying the binary shape coding method of MPEG-4 Part 2 Visual, the motion information of one or more specific bit planes in the block unit is not stored in the block unit but the motion information is not stored in each bit plane block. You can only save it.

The process of performing "CAE encoding" in "Bitplane Coding" of FIG. 9 is performed by the [Method # 1] Run Length Coding Method and the Variable Length Coding Method. And [Method # 2] Bitplane coding method using context-based adaptive arithmetic coding, and [Method # 3] Constant Area Coding (CAC) or Joint Bi-Level Image Processing Group (JBIG) method or Quad Tree method. It can be performed by replacing with the bitplane coding method using.

[Method # 1] Instead of performing "CAE encoding" in "Bitplane Coding" of FIG. 9, a bitplane using a Run Length Coding method and a Variable Length Coding method is used. An example of the proposed method for coding a block is shown in FIG. 18.

18 illustrates a method of coding a bitplane block using a run length coding method and a variable length coding method, and receives a bitplane block to perform bitplane coding. Output the bitstream. In the process of "pixel scanning and storing the run length", the input bitplane block performs a scan using the pixel scanning method of FIG. 17 to store lengths of runs for each symbol of the bitplane block. . Here, a method of counting the lengths of runs for each symbol is shown in FIG. 19, and a detailed algorithm is as follows.

Step 1) Set the previous pixel value (PrevPixel) to '0' and the length of the run (nCount) to '0'.

Step 2) The current pixel value CurrPixel is read through the function NextPixel () which reads the next pixel in the scanning order of FIG. 17.

Step 3) If the current pixel value (CurrPixel) is different from the previous pixel value (PrevPixel), branch to Step 5. If not, branch to Step 4.

Step 4) Increase the run length (nCount) to '1' and branch to Step 2.

Step 5) Save the run length (nCount).

Step 6) Set the run length (nCount) to '0'.

Step 7) If there is no pixel value read through the function NextPixel () for reading the next pixel according to the scanning sequence of FIG. 17, the algorithm ends. If not, branch to Step 2.

In the “variable length coding” process of FIG. 18, a bitstream is output by performing a variable length coding method on the input run lengths according to a probability distribution. In this case, a method for coding the input run lengths may be coded using an arithmetic coding method.

In the method of coding a bitplane block using the run length coding method and the variable length coding method of FIG. 18, a subblock is formed by dividing an input bitplane block into small blocks. Each of the subblocks may be coded, and an example of the proposed method is illustrated in FIG. 20.

In the bitplane coding method using the run length coding method and the variable length coding method of FIG. 20, a bitplane block is divided into subblocks and coded. It receives the sub-blocks and performs bitplane coding on each sub-block to output a bitstream. In the "subblock separation" process, the input bitplane block is divided into small blocks to form a subblock. The process of "pixel scanning and run length storing" and the "variable length coding" process are repeated as many times as the number of divided subblocks. In the "pixel scan and run length storage" process, a scan using the pixel scan method of FIG. 17 is performed on the input sub bit plane block to store a run length of the bit plane block. Herein, a method of determining the length of the run is shown in FIG. 19.

[Method # 2] Context-based adaptive arithmetic using coding information of neighboring bitplane blocks adjacent to the bitplane block to be coded instead of the method of performing "CAE encoding" in "bitplane coding" of FIG. Coding may be performed by applying a CABAC (Context-based Adaptive Binary Arithmetic Coding) method. An example of the proposed method is shown in FIG.

FIG. 21 illustrates a method of applying a context-based adaptive binary arithmetic coding (CABAC) method to a bitplane coding method using coding information of neighboring bitplane blocks adjacent to a bitplane block to be currently coded. A bitstream block is applied to a bitplane block, and a bitstream coding is performed by applying context-based adaptive arithmetic coding to output a bitstream. Here, the input bitplane block may be divided into small blocks to form a sub block, and the method of FIG. 21 may be applied to each sub block. In the context configuration process, a context template is generated using neighboring pixels adjacent to a pixel to be currently coded in the input bitplane block. In the "arithmetic coding" process, arithmetic coding is performed on the current pixel using the generated context template.

[Method # 3] Instead of performing the "CAE encoding" in "Bitplane Coding" of FIG. 9, the bitplane block to be currently coded is a constant area coding (CAC) or Joint Bi-Level Image Processing Group (JBIG). The bitplane coding method can be performed by using the method or the Quad Tree method.

As an example of a method for performing "bitplane decoding", the method used in binary shape coding of the international video standard MPEG-4 Part 2 Visual (ISO / IEC 14496-2) can be used. have. An example of the proposed "bitplane decoding" is shown in FIG. 22.

FIG. 22 illustrates a structure diagram for performing bitplane decoding, and receives a bitstream and performs bitplane decoding to output a reconstructed bitplane block. The reference picture stored in the reference picture buffer of FIG. 8 and the reconstructed picture stored in the reconstructed picture buffer of FIG. 8 are each bitplane through a "gray code conversion" process and a "bitplane separation" process. The reference bitplane image of the same bitplane as the bitplane block to be currently coded is input into the "motion compensation" and "CAE decoding" processes.

In the “DE-MUX” process of FIG. 22, the bitstream is received as an input and outputs a motion vector of the current bitplane block, a mode of the current bitplane block, and a bitstream to be used in the “CAE decoding” process.

Here, the mode of the bitplane block can be decoded in various ways.

(1) As one embodiment, the mode of the current bitplane block may be decoded through fixed length decoding.

(2) As another embodiment, the mode of the current bitplane block may be decoded through variable length decoding using a probability according to the frequency of appearance of the bitplane block mode. In this case, a previously calculated probability table may be used.

(3) As another embodiment, decoding the mode of the current bitplane block by using an arithmetic coding method using a probability adaptively changed according to the situation in consideration of the bitplane block mode around the current bitplane block can do.

In addition, the method of decoding the bitplane block mode as described above may be changed in various ways.

The decoding is performed according to the mode of the current bitplane block. If the mode of the bitplane block is 'No Update Without MV', motion compensation is performed by using the motion vector prediction value MV _P as a motion vector to output a reference bitplane block corresponding to the motion vector.

Here, a method of determining the motion vector prediction value MV _{P of the} bitplane block to be decoded may be variously applied.

(1) As an example, the motion vector prediction value MV _p of the current bitplane block may be determined using motion vectors of a bitplane block or a depth information map block adjacent to the bitplane block to be currently decoded. have.

(2) As another example, in the flowchart of FIG. 12, the motion vector prediction value of the current bitplane block using the motion vector of the depth information map block instead of the neighboring bitplane block adjacent to the bitplane block to be currently decoded. MV _p ) can be determined.

(3) As another embodiment, the motion vector MV _{Left of the left} bitplane block or the depth information map block and the motion vector MV _Above of the upper _bitplane block or depth information map block in the bitplane block to be currently decoded. ) And the motion vector prediction value MV _p of the current _bitplane block using the median of the motion vector MV _RightAbove of the upper right _bitplane block or the depth information map block. Here, if the upper right bit plane of the block or the depth information of the map block motion vector (MV _RightAbove) is not available in can be used instead of a motion vector (MV _LeftAbove) at the left upper bit plane or block depth map information block.

In addition, the method of determining the motion vector prediction value MV _{P of the} bitplane block to be decoded as described above may be changed in various ways.

If the mode of the bitplane block is 'No Update With MV', "motion compensation" is performed using the received motion vector value and the reference bitplane block corresponding to the motion vector is output. If the mode of the bitplane block is 'all_0', "same level block decoding" is performed, and all pixel values in the block are set to '0' and output. If the mode of the bitplane block is 'all_1', "same level block decoding" is performed, and all pixel values in the block are set to '1' and output. If the mode of the bitplane block is 'intraCAE', "Context-based Arithmetic Encoding (CAE) decoding" is performed, and binary arithmetic coding based on the surrounding pixel information of each pixel to be decoded in the current bitplane block. Binary Arithmetic Coding is performed. If the mode of the bitplane block is 'interCAE', "Context-based Arithmetic Encoding (CAE) decoding" is performed and corresponding to the current pixel and the surrounding pixel information of each pixel to be decoded in the current bitplane block. Binary Arithmetic Deoding is performed based on the pixels of the reference image and the surrounding pixel information.

When the mode of the current bitplane block is selected as 'intraCAE' and 'interCAE', a context-based Arithmetic Encoding (CAE) method is used, and an example of the CAE decoding structure is illustrated in FIG. 23. .

In the process of "conversion ratio decoding" of FIG. 23, a transform ratio CR is decoded from an input bitstream and output. In the "downsampling" process, the reference bitplane block is downsampled according to the input conversion ratio to be used for context calculation. If the conversion ratio CR is '1', downsampling is not performed. The configuration of the context template in the "context calculation" process is the same as that described in "context calculation" of FIG. 13. In the "arithmetic decoding" process, the current pixel is generated by performing an arithmetic decoding on the input bitstream with reference to a probability table having the context template configured in the "context calculation" process as an index.

Here, there are various ways to configure the context template in the 'intraCAE' mode and the 'interCAE' mode.

(1) As one example, in the 'intraCAE' mode, the context template may be configured using various numbers of pixels among neighboring pixels adjacent to the pixel X to be coded in the bitplane block. In an embodiment, in the 'intraCAE' mode, four neighboring pixels around the current pixel X may be configured to form a 4-bit context template in the form of (c3 c2 c1 c0).

(2) In another embodiment, in the 'interCAE' mode, a reference corresponding to pixels X to be neighbored adjacent to pixel X to be coded in the current bitplane block and to pixel X to be coded in the current bitplane block. The context template may be configured by using various numbers of pixels among the pixels of the bitplane image and neighboring pixels. In an embodiment, in the 'interCAE' mode, four neighboring pixels adjacent to the current pixel X may be configured to form a 4-bit context template in the form of (c3 c2 c1 c0).

In addition, the method of configuring the context template as described above may be changed in various ways.

In the "upsampling" process, up-sampling of the decoded bitplane block is performed to generate a reconstructed bitplane block. If the conversion ratio is '1', no upsampling is performed.

Another embodiment of the method of performing "bitplane decoding" of FIG. 22 is a method of applying the binary shape decoding method of the above-described MPEG-4 Part 2 Visual.

① As an example of applying the binary shape decoding method of MPEG-4 Part 2 Visual, decoding may be performed by variously applying a decoded transform ratio (CR) of the “bit rate adjustment” part.

② As another example of the method of applying the binary shape decoding method of MPEG-4 Part 2 Visual, a method of selectively using motion information stored in one or more specific bit planes in block units can be selectively used in each bit plane. have.

The method of performing "CAE decoding" in "bitplane decoding" of FIG. 22 includes the method [1]: a bitplane decoding method using a run length coding method and a variable length coding method. And [Method # 2] Bitplane decoding using context-based adaptive arithmetic coding, and [Method # 3] Constant Area Coding (CAC) or Joint Bi-Level Image Processing Group (JBIG) or Quad Tree method. It may be performed by being replaced by a bitplane decoding method using.

[Method # 1] Instead of performing "CAE encoding" in "Bitplane Coding" of FIG. 22, the bitplane is executed using the Run Length Coding method and the Variable Length Coding method. An example of the proposed method may be coded in a block as illustrated in FIG. 24.

FIG. 24 illustrates a method of decoding a bitplane block using a run length coding method and a variable length coding method, and reconstructs a bitplane block by receiving a bitstream. Outputs a bitplane block. In the "variable length decoding" process, the length of the run is output by variable length decoding of the input bitstream. In the "bitplane block configuration" process, a bitplane block is constructed by using information about the scan order of the bitstream and run lengths.

In the method of decoding a bitplane block using the run length coding method and the variable length coding method of FIG. 24, the input bitstream may be divided into subblocks to perform decoding. , An example of the proposed method is shown in FIG.

In the bitplane coding method using the run length coding method and the variable length coding method of FIG. 25, a bitplane block is divided into subblocks and decoded. The decoding is performed by dividing into subblocks to output a reconstructed bitplane block. The "variable length decoding" process and the "sub block construction" process are repeated as many times as the number of divided subblocks. In the "variable length decoding" process, the input bitstream is variable-length decoded to output run lengths for each subblock. In the "sub block construction" process, a sub bit plane block is constructed by using information about a scan order of the bitstream and run lengths. In the "bitplane block configuration" process, a bitplane block is output using subbitplane blocks.

<Decoder>

The method of adaptively selecting and coding a block unit bitplane coding method and a conventional block unit video coding method, and a method of adaptively selecting a bit plane to be coded by dividing a depth information map into bit plane units An example of a decoder structure diagram is illustrated in FIG. 26.

The bitstream input in FIG. 26 outputs a reconstructed depth information map block decoded through a bitplane decoding mode or an existing decoding mode. If the bitplane decoding mode is selected, the part marked 'C' is performed. 'C' receives the bitstream and repeatedly performs bitplane decoding as many as the number of bitplanes. The reconstructed depth information map block represented by N-bit is output by combining the images. If the existing decoding mode is selected, the part marked with 'D' is performed. 'D' receives the bitstream and receives the existing video coding method (MPEG-1, MPEG-2, MPEG-4 Part 2 Visual, H.264 / AVC, SVC, MVC, VC-1, AVS, KTA, etc.) and output the reconstructed depth information map block.

In FIG. 26, the bitplane decoding mode 'C' shows a decoder structure diagram of the bitplane unit coding method. In the decoder structure diagram of 'C', a unit for processing data is a block having a size of 16x16 pixels. The reference picture stored in the reference picture buffer of FIG. 26 and the reconstructed picture stored in the reconstructed picture buffer are input through a process of "bitplane decoding." In the "bitplane decoding" process, the bitstream is input and repeatedly performed as many as the number of encoded bitplanes. The bitstream is decoded for each bitplane to output n bitplane images. The method of performing "bitplane decoding" uses the "bitplane decoding" method of FIG. In the "bitplane combining" process, each of the output bitplane images is combined to output an image represented by n-bit. In the "inverse gray code conversion" process, the image represented by the gray code is restored to its original form. In the "depth information map construction" process, an image represented by n-bit is configured according to the number of bits of the actual depth information map, and the depth information map is output.

Here, the method of constructing the depth information map may be variously applied.

(1) As one embodiment, in order to configure the current block according to the number of bits of the actual depth information map, all values of the undecoded bitplane block may be set to '0 (0)'.

(2) As another embodiment, in order to configure the current block according to the number of bits of the actual depth information map, all values of the undecoded bitplane block may be set to '255 (1)'.

(3) As another embodiment, in order to configure the current block according to the number of bits of the actual depth information map, whether or not all of the decoded bitplane blocks are configured as '0 (0)' or all as '255 (1)'. The information on whether to configure may be decoded to construct an undecoded bitplane block.

(4) As another example, in order to configure the current block according to the number of bits of the actual depth information map, the undecoded bitplane block may be configured using pixel values of adjacent neighboring bitplane blocks. Here, the bitplane block may be configured through padding using pixel values of the neighboring bitplane blocks by decoding information on a corresponding padding direction from the bitstream.

In addition, the method of constructing the depth information map as described above may be changed in various ways.

The constructed depth information map is used for intra prediction in the existing decoding mode 'D' or stored in a reconstructed image buffer through a deblocking filter or output as a "reconstructed depth information map". In this case, the deblocking filter may be performed on the currently decoded block through information on whether the deblocking filter is performed on the currently decoded block decoded from the bitstream. The depth map configured here may be stored in the reconstructed image buffer without passing through the deblocking filter. In addition, the configured depth information map may be used for intra prediction after passing through a deblocking filter.

FIG. 26 shows a decoder of H.264 / AVC as an example of the existing decoding mode 'D'. In the decoder structure diagram of 'D', the data processing unit is a macroblock having a width of 16 × 16 pixels and decoding is performed in an intra mode or an inter mode. In the Intra mode, the switch inside the 'D' is switched to Intra, and in the Inter mode, the switch is switched to Inter. The main flow of the decoding process is to first generate a prediction block, and then decode the input bitstream to generate a reconstructed depth information map block by adding the result block and the prediction block. First, the generation of the prediction block is performed according to the intra mode and the inter mode. First, in intra mode, a prediction block is generated by performing spatial prediction using neighboring coded neighboring pixel values of the current block in the "intra prediction" process, and stored in the reference image buffer using a motion vector in the inter mode. The prediction block is generated by searching for a corresponding region in the reference image and performing "motion compensation". In the "entropy decoding" process, the input bitstream is subjected to entropy decoding according to a probability distribution to output quantized coefficients. The quantized coefficients are inversely quantized and inversely transformed to generate a reconstructed block through the prediction image and the adder, and then the blocking artifact is removed by the deblocking filter. Buffer ".

<How to select the mode>

There is a need for a mode selection method capable of adaptively selecting a block-based bitplane coding method and an existing block-based video coding method by adaptively selecting a coding method. 27, and the detailed algorithm is as follows.

Step 1) The cost of the current depth information map block is calculated in the existing video coding mode Cost _A and the cost of the bit plane coding mode Cost _B.

Step 2) If the cost (Cost _A ) of the existing video coding mode is smaller than the cost (Cost _B ) of the bitplane coding mode, the current depth information map block branches to Step 3. If not, branch to Step 4.

Step 3) The current depth information map block is coded in the existing video coding mode.

Step 4) Code the current depth information map block in bitplane coding mode.

Cost (Cost _i) for each coding mode, the coding mode decision flow chart of Fig. 27 may be calculated in various ways, an example of how to calculate the cost may be used. "Lagrangian optimization technique", one example of SAD (sum of The corresponding equation of how to use absolute difference) is shown in Equation 4.

SAD _i (sum of absolute difference) is the sum of the absolute values of the prediction error between the original image and the reconstructed image, and λ _MODE is the value generated by multiplying the square of the quantization parameter by the constant C (C = 0.85) Represents the "Lagransian" constant for the mode, and R _i represents the amount of bits generated when actual coding is performed in the corresponding macroblock mode. As another example of a method of calculating a cost, an SSD (sum of square difference) may be used.

< Bitplane  How to adjust bit rate when coding>

In bitplane unit coding, bitplane coding is performed by applying downsampling to a depth information map block or a binary block of a separated bitplane, or to separate a depth information map block into bitplane units. After that, a method of not coding a binary block of a specific bitplane among binary blocks of the separated bitplane may be used. In this case, when using a method that does not code a binary block of a specific bitplane, if information loss occurs when converting the depth information map to gray code, the correct value is restored when the gray code is restored to the existing depth information map. It cannot be restored. Therefore, in order to allow loss of information for bit rate control in the "bitplane coding" process, coding should be performed while increasing the number of bitplanes to be coded in order from MSB bitplane to LSB bitplane. .

A method of adaptively selecting and coding a block unit bitplane coding method and a conventional block unit video coding method of FIG. 8, and adaptively selecting a bit plane to be coded by separating a depth information map into bit plane units In the method of bitplane unit coding, a bitrate adjustment process is a method of adjusting bitrate by applying downsampling to a depth information map block or a binary block of a separated bitplane, and [method # 2] depth information. Method of adjusting bit rate by separating map block into bitplane units and not coding binary block of specific bitplane among binary blocks of separated bitplane, [Method # 3] [Method # 1] and [Method # 2] can be used to adjust the bit rate.

[Method # 1] An example of a method of adjusting a bit rate by applying downsampling to a depth information map block or a binary block of a separated bitplane is illustrated in FIG. 28.

FIG. 28 receives a depth information map block and applies downsampling to a depth information map block or a binary block of a separated bitplane to output a downsampled depth information map block or to output downsampled bitplane blocks. In the "bitplane separation" process, a depth information map block expressed as n-bit is received and divided into n bitplane blocks. In the "sampling rate determination" process, the optimum sampling rate is determined by receiving the depth information map block or the bitplane blocks separated by bitplane units. How to determine the optimal sampling rate will be discussed in detail later. In the "downsampling" process, the depth information map block is input and downsampling is performed according to an optimal sampling rate in the horizontal or vertical direction. Here, when downsampling the depth information map block, a method of performing downsampling by applying a filter may be used, and as shown in FIG. 29, a method of performing downsampling without applying a filter may be used.

In FIG. 29, the left image is the original image, and the right image is a result of performing 1/2 downsampling in the horizontal and vertical directions. The white and black blocks in the original image (the left image in FIG. 29) mean pixels, and the black blocks mean pixels that will be used to construct a downsampled image. Herein, each pixel may be a pixel value of the depth information map block, and may also be a pixel value of the bitplane block.

An example of a flowchart of a method of determining an optimal sampling rate in the “sampling rate determination” process of FIG. 28 is as shown in FIG. 30, and a detailed algorithm is as follows.

Step 1) Initialize the sampling rate to '1'.

Step 2) Multiply the sampling rate by 1/2.

Step 3) The downsampling is performed in the horizontal or vertical direction using the set sampling rate.

Step 4) After performing bitplane coding on the downsampled block, calculate the Cost _i value. The method of calculating the Cost _i value is described in detail in the above section.

Step 5) If the currently set sampling rate is the lowest sampling rate, branch to Step 6. If not, branch to Step 2. In this case, the lowest sampling rate may be set to an arbitrary value by the user.

Step 6) If the Cost _i value of the current downsampled block is the minimum among all the downsampled blocks, the flow branches to Step 7. If not, branch to Step 2.

Step 7) Set the sampling rate of the current downsampled block to an optimal sampling rate.

[Method # 2] In order to adjust the bit rate of the current depth information map block, the depth information map block is divided into bit plane units, and the binary blocks of a specific bit plane are not coded among the binary blocks of the separated bit plane. The bit rate may be adjusted through the method. For example, a method of determining a bit plane to be coded may be used. In this case, the method of determining the bit plane to be coded should determine the bit plane to be coded in order from the MSB bit plane to the LSB bit plane because of the characteristics of the gray coded depth information map. Therefore, the method of determining the bit plane to be coded may be referred to as a method of determining the number of bit planes to be coded. The method of determining the number of bitplanes to be coded can be determined in units of blocks by using a bit rate-distortion optimization method. Coded through length coding, the data is included in the bitstream.

An example of a flowchart of a method of determining the number of bit planes to be coded is shown in FIG. 31, and a detailed algorithm is as follows.

Step 1) Set the number of bit planes (N) to be coded as '0'.

Step 2) Increase the number (N) of bit planes to be coded by '1'.

Step 3) Set the coding mode from MSB bitplane to LSB bitplane in order according to the number of bit planes (N) to be coded. For example, if N is '1', it is a mode for coding only MSB non-plane. If N is '2', it is a mode for coding MSB and MSB-1 bit plane, and N is '3'. In the case of MSB, MSB-1 and MSB-2 bitplanes are coded, and if N is 'm', MSB, MSB-1, MSB-2, MSB-3,... Is a mode for coding the MSB- (m-1) bitplane.

Step 4) after performing the coding by the coding mode, and calculates the cost (Cost _i) the value of the encoding mode. The method of calculating the Cost _i value is described in detail in the above section.

Step 5) If the coding mode has a minimum Cost _i value among all the coding modes, the process branches to Step 6. If not, branch to Step 2.

Step 6) Determine the number of bitplanes to code through the coding mode with the lowest Cost _i value.

[Method # 3] To adjust the bit rate for the current depth information map block, the bit rate may be adjusted by combining [Method # 1] and [Method # 2]. As an example, [Method # 1] is performed on a depth information map to be coded to generate a downsampled depth information map, and [Method # 2] is performed to determine the number of bitplanes to be coded of a downsampled depth information map. By doing this, bit rate adjustment can be performed.

<Example of Overall Coding Flow and Implementation of Proposed Coding Method>

The overall coding flow of the method of adaptively selecting and coding a block unit bitplane coding method and a conventional block unit video coding method is shown in FIG. 32. A detailed algorithm is as follows.

Step 1) Performs mode decision on whether to perform bitplane unit coding or not coding in the existing depth information map block. As an example of a method of performing, the method of FIG. 27 may be used.

Step 2) If bitplane coding mode is selected during the mode decision, 'bitplane_coding_flag', which is flag information on whether to perform bitplane coding, is set to '1', and if the existing coding mode is selected, 'bitplane_coding_flag' Set 'to' 0 '.

Step 3) If bitplane_coding_flag is '0', branch to Step 4. If not, branch to Step 5.

Step 4) The current depth information map block is coded in the existing video coding mode.

Step 5) Code the current depth information map block in bitplane coding mode.

In the method of adaptively selecting the bitplane to be coded by dividing the depth information map into bitplane units, the overall coding flow of the method of determining the number of bitplanes to be coded by the bitrate adjustment method in bitplane unit coding is 33, the detailed algorithm is as follows. 33 illustrates an example in which the depth information map is represented by 8 bits.

Step 1) The method of FIG. 31 may be used as an example of a method of determining the number of bit planes to be coded and performing the same.

Step 2) Set the number of bitplanes to code (bitplane_num).

Step 3) If bitplane_num is not '8', branch to Step 4. If not, code the MSB, MSB-1, MSB-2, MSB-3, MSB-4, MSB-5, MSB-6, LSB bitplanes.

Step 4) If bitplane_num is not '7', branch to Step 5. If not, code the MSB, MSB-1, MSB-2, MSB-3, MSB-4, MSB-5, MSB-6 bitplanes.

Step 5) If bitplane_num is not '6', branch to Step 6. If not, code the MSB, MSB-1, MSB-2, MSB-3, MSB-4, MSB-5 bitplanes.

Step 6) If bitplane_num is not '5', branch to Step 7. If not, code the MSB, MSB-1, MSB-2, MSB-3, MSB-4 bitplanes.

Step 7) If bitplane_num is not '4', branch to Step 8. If not, code the MSB, MSB-1, MSB-2, and MSB-3 bitplanes.

Step 8) If bitplane_num is not '3', branch to Step 9. If not, code the MSB, MSB-1, and MSB-2 bitplanes.

Step 9) If bitplane_num is not '2', code the MSB bitplane. If not, code the MSB, MSB-1 bitplane.

A method of adaptively selecting and coding the block-based bitplane coding method and an existing block-based video coding method, and a method of adaptively selecting a bitplane to be coded by separating a depth information map into bitplane units. As an example, the macroblock layer (macroblock_layer) syntax may be modified as shown in Tables 1 and 2 below, in which H.264 / AVC is actually implemented. One, but divided by the limits of the ground)

"bitplane_coding_flag" indicates whether or not the current depth information map block is coded in bitplane coding mode. If "bitplane_coding_flag" value is '1', bitplane coding is performed. Coding was done in the traditional way. 'bitplane_num' represents the number of bit planes coded in the current depth information map block by the bit rate adjusting method using the method of FIG. Here, when bit rate adjustment is performed using the method of FIG. 28, 'bitplane_num' is changed to 'downsampling_rate', and 'downsampling_rate' is downsampling in the horizontal or vertical direction with respect to the current depth information map block. Sampling rate. Depth 'bab_type' indicates a mode of a bitplane block when bitplane unit coding is performed in the <encoder> method. 'mvd_x' and 'mvd_y' indicate horizontal and vertical components of a motion vector difference (MV _D ) of a motion vector in a current bitplane block. The difference value of the motion vector is obtained by difference between the motion vector (MV) of the current bitplane block and the motion vector prediction value (MV _P ) of the current bitplane block (MV _D = MV-MV _P ). The decoder calculates a motion vector by adding the difference value of the received motion vector to the motion vector prediction value (MV = MV _D + MV _p ). "scan_type" is flag information on whether to perform a horizontal scan or a vertical scan when performing a context-based arithmetic encoding (CAE) of the current bitplane block. When 0, horizontal scan is performed. When 'scan_type' is 1, vertical scan is performed. "binary_arithmetic_code ()" means data coded through Context-based Arithmetic Encoding (CAE).

In order to verify the superiority of the method proposed in the present invention, the inventor of the present invention actually implemented in JM (Joint Model) 13.2, which is the reference software of H.264 / AVC, and compared the existing method with the proposed method. In bitplane unit coding, the current bitplane coding mode allows only three modes, "all_0", "all_1", and "intraCAE". The baseline profile of H.264 / AVC was used as a condition of the existing coding method, and the depth information map of Ballet XGA (1024x768) 15Hz image was compared with the depth information map of Breakdancers XGA (1024x768) 15Hz image.

34 and 35 illustrate Rate-distortioon (RD) -Curve for Peak Singal-to-Noise Ratio (PSNR) vs. Bit Rate for the existing method (Anchor) and Proposed method using H.264 / AVC. Indicates. The graph results show that the performance of the proposed method is much better than that of the conventional method. When the performance is compared using the BD-PSNR method, which shows the average PSNR improvement, the performance is 0.87 dB ~ 0.89. The performance was improved by 12.62% to 12.89% when the performance was improved by using the BD-rate method, which showed a dB improvement and average bit-rate reduction.

<Device of invention>

In the present invention, a method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to code by separating a depth information map into bitplane units 36 is an encoder diagram of the method.

The input image shown in FIG. 36 is a depth information map expressed in n-bit, and the output is divided into blocks, and the output information is divided into bit plane units, and the bit plane to be coded is adaptively selected. A bitstream that is a result of being coded by adaptively selecting and coding a bitplane unit coding method and a conventional video coding method. The "reference picture buffer part" stores a depth information map previously coded and decoded in the time direction. The reconstructed image buffer unit stores the depth information map obtained by encoding and decoding the current depth information map. The "mode determination unit" receives a depth information map block expressed in n-bit and adaptively selects and codes the mode selection method of FIG. 27 (a block unit bitplane coding method and a conventional block unit video coding method). A mode selection method capable of adaptively selecting a method) to output information about a coding mode in a bitstream.

When an existing coding mode is selected in the "mode determination unit", "A" of FIG. 36, which is an existing coding method, is performed. In the "prediction unit", the reference image stored in the "reference image buffer unit" and the resultant image of the "decoding unit" and the image encoded and decoded by bitplane coding are received and predicted according to the Intra and Inter modes. Output the block and pass it to the "encoding unit". The "encoding unit" receives the prediction block and the depth information map and performs encoding to output the bitstream. The "decoding unit" receives the bitstream encoded by the prediction block and the "encoding unit", outputs a reconstructed depth information map, and delivers the reconstructed depth information map to the "prediction unit" and the "deblocking filter unit". The "deblocking filter unit" removes blocking artifacts generated during the coding process and stores the blocking artifacts in the reconstructed image buffer unit. In the "multiplexer", one bitstream is output by combining a bitstream output by performing a bitplane coding mode and a bitstream output by performing an existing coding mode.

When the bitplane coding mode is selected in the "mode determination unit", "B" of FIG. 36, which is a bitplane unit coding method, is performed. The "gray code converter" receives the depth information map block and converts the gray code. The "bitplane separation unit" receives a depth information map block expressed in N-bit and divides it into N bitplane blocks. The "bit rate controller" may or may not be used depending on the option. The bit rate controller receives a depth information map block and selects a bit plane to be coded by the method of FIG. 28 or 31, and the information on the number of bit planes to be coded is a bitstream. Is output. The reference picture stored in the reference picture buffer and the reconstructed picture stored in the reconstructed picture buffer are input to the "bitplane coding unit". The "bitplane coding unit" repeatedly encodes each bitplane by the number of bitplanes to be coded through the input reference image and the reconstructed image, and outputs the result as a bitstream. As a method of coding a bitplane block, a bitplane coding method using the shape coding method of the MPEG-4 Part 2 Visual in FIG. 9 may be used. Bitplane coding is selected to perform " bitplane coding ", and the coded data performs a " bitplane decoding unit " for coding a depth information map block to be input later. The "bitplane decoding unit" receives the encoded bitstream and repeatedly performs the bitplane number, decodes the input bitstream for each bitplane, and outputs n bitplane images. The "bitplane decoding unit" may use a bitplane coding method applying the shape decoding method of MPEG-4 Part 2 Visual in FIG. 22. In the "bitplane combiner", the N bitplanes output from the "bitplane decoder" are combined to N-bit. The "inverse gray code conversion unit" restores the depth information map block expressed in the gray code to its original form. The "depth information map constructing unit" outputs the reconstructed depth information map configured for the number of bits of the actual depth information map.

In the present invention, a method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to code by separating a depth information map into bitplane units A decoder device diagram of the method is shown in FIG.

In FIG. 37, the input is a bitstream, and the output is a bitplane unit coding method using a method of adaptively selecting a bitplane to be coded by dividing the depth information map into blocks, and then separating them into bitplane units. A reconstructed depth information map resulting from decoding by adaptively selecting and coding a video coding method. The reference image buffer unit stores the depth information map previously decoded in the time direction. The reconstructed image buffer unit stores the depth information map obtained by encoding and decoding the current depth information map. The coding mode determination unit receives the bitstream and decodes coding information of a depth information map to be currently decoded, and determines whether to decode in the bitplane decoding mode or the conventional decoding mode.

When the bitplane decoding mode is selected, 'C' of FIG. 37, which is a bitplane decoding method, is performed. The "bitplane decoding unit" is repeatedly executed by receiving the bitstream as many as the number of bitplanes, decodes the input bitstream for each bitplane, and outputs n bitplane images. The "bitplane decoding unit" may use a bitplane coding method applying the shape decoding method of MPEG-4 Part 2 Visual in FIG. 22. In the "bitplane combiner", the N bitplanes output from the "bitplane decoder" are combined to N-bit. The "inverse gray code conversion unit" restores the depth information map block expressed in the gray code to its original form. The "depth information map constructing unit" outputs the reconstructed depth information map block configured for the number of bits of the actual depth information map.

When the existing decoding mode is selected, 'D' of FIG. 37, which is an existing decoding method, is performed. In the "prediction unit", the reference image stored in the "reference image buffer unit", the resultant image of the "decoding unit", and the image decoded by bitplane coding are input, and the prediction block is converted according to the intra and inter modes. Output it to the "decoding part". The "decoding unit" receives the prediction block and the bitstream, decodes them, outputs a reconstructed depth information map block, and transfers them to the "prediction unit" and the "deblocking filter unit". The "deblocking filter unit" receives the reconstructed depth information map block, removes blocking artifacts generated during the coding process, and then stores or filters the reconstructed depth information map block in the "reconstructed image buffer unit". Outputs

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be construed to include various embodiments within the scope of the claims.

1 is an example of an actual image (left) and a depth information map image (right)

2 is an encoder structure diagram of H.264 / AVC

3 is a block diagram of a decoder of H.264 / AVC

4 is a three-dimensional graph representing the level of each pixel of the real image and the depth information map of FIG. (Left) graph of actual image, (right) graph of depth map

5 is a bitplane analysis: a bitplane representation of the real image of FIG. 1 (left), a bitplane of the depth information map (center), and a gray code applied to the bitplane of the depth information map (right)

6 is a bitplane analysis of an object boundary partial macroblock in a depth information map of a ballet image.

7 is a bitplane analysis of a background partial macroblock in a depth information map of a ballet image.

8 is a method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to be coded by separating a depth information map into bitplane units. An example of an encoder schematic of how to

9 is an example of a method of performing “bitplane coding”

10 is a method of determining an intra picture mode

11 is a method for determining an inter picture mode.

12 is a flowchart illustrating a method of determining a motion prediction value (MVP).

13 is an example of a "CAE encoding" structure diagram

14 is a flowchart for determining a conversion ratio (CR).

15 shows a context template for intraCAE

16 is a context template for interCAE

17 is an example of a scan sequence

18 is an example of a bitplane coding method using run length coding and variable length coding methods

19 shows how to count the lengths of Runs for each symbol.

20 is an example of a method of dividing a bitplane block into subblocks in a bitplane coding method using a run length coding method and a variable length coding method.

21 is an example of a bitplane coding method using a context-based adaptive arithmetic coding method

22 is an example of a method of performing "bitplane decoding"

23 is an example of a "CAE decoding" structure diagram

24 is an example of a bitplane decoding method using run length coding and variable length coding methods

25 illustrates an example of decoding a bitplane block by dividing it into subblocks in a bitplane coding method using a run length coding method and a variable length coding method.

FIG. 26 is a method of adaptively selecting and coding a block-based bitplane coding method and an existing block-based video coding method, and adaptively selecting a bitplane to be coded by separating a depth information map into bitplane units. An example of decoder structure diagram of how to

27 is an example of a coding mode determination method

28 is an example of a method of adjusting a bit rate by applying downsampling to a depth information map block or a binary block of a separated bitplane

29 is an example of a method of downsampling

30 illustrates an embodiment of a flowchart of a method of determining an optimal sampling rate.

31 is an embodiment of a flowchart of a method of determining the number of bitplanes to code

32 illustrates an embodiment of a coding mode execution method according to a "bitplane_coding_flag" value

33 is a flowchart illustrating a method of performing bitplane coding according to a value of "bitplane_num".

34 is a graph of a result of a depth information map experiment of a Ballet image (Anchor: conventional method, Proposed: proposed method)

35 is a graph showing the results of the depth information map experiment of the breakdanceres image (Anchor: Conventional Method, Proposed: Proposed Method)

36 is a diagram for adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to be coded by dividing a depth information map into bitplane units. Encoder device of how to

37 shows a method of adaptively selecting and coding a block-based bitplane coding method and a conventional block-based video coding method, and adaptively selecting a bitplane to be coded by separating a depth information map into bitplane units. Decoder device of the method.

Claims (26)

A method and apparatus for adaptively selecting and encoding a block-based bitplane coding method and an existing block-based video coding method. The method of claim 1, wherein an optimal mode is selected from a block-based bitplane coding method and a conventional block-based video coding method. 16. A method for encoding a depth information map, comprising: a bit rate adjusting step for obtaining a desired bit rate; Dividing the depth information map into bit plane units; Converting each pixel of the depth information map block into a gray code; Encoding a bitplane; And an entire encoding step of encoding the entire block. 4. The method and apparatus of claim 3, wherein each block is bit-plane encoded by dividing the depth information map into MxN pixel blocks. 4. The method of claim 3, wherein the method of encoding the depth information map comprises using a method of determining the number of bit planes to be coded in units of arbitrary blocks. 6. The method of claim 5, further comprising adjusting a bit rate by applying downsampling to a depth information map block or a binary block of a separated bit plane, and separating the depth information map block into bit plane units. A method, method and method of adjusting a bit rate by combining the above methods, which do not code a binary block of a particular bitplane among binary blocks. 4. The method of claim 3, further comprising: performing gray code conversion on each pixel of the reference picture and the reconstructed picture; Bitplane separation for the reference image and the reconstructed image; Predicting movement of the selected bitplane block; Compensating for the movement of the selected bitplane block through the predicted movement; Determining a mode of the selected bitplane block; CAE encoding the selected bitplane block; Generating a bitstream by collecting coding information of the selected bitplane block; And an entire encoding step of encoding the entire bitplane block. The method of claim 7, wherein the intra picture mode determination method and the inter picture mode determination method and the determined bit plane block mode are encoded in the method for determining the mode of the selected bit plane block of the depth information map block. 8. The method of claim 7, wherein the predicted value of the motion vector of the selected bitplane block of the depth information map block is determined using motion information of the neighboring bitplane block or the depth information map block adjacent to the selected bitplane block. 8. The method of claim 7, further comprising the steps of: CAE encoding a selected bitplane block of a depth information map block, comprising: adjusting a bit rate for the selected bitplane block; Downsampling a reference bitplane block; A context calculation step using a reconstructed bitplane image and a reference bitplane block or a downsampled reference bitplane block; An arithmetic encoding step using a probability table; And an entire encoding step of encoding the entire bitplane block. 12. The method of claim 10, wherein in CAE encoding the selected bitplane block of the depth information map block, a method of determining a conversion ratio for the selected bitplane block. 12. The method of claim 10, wherein in CAE encoding the selected bitplane block of the depth information map block, constructing a context template according to the mode of the selected bitplane block. 12. The method of claim 10, wherein in performing CAE encoding a selected bitplane block of the depth information map block, arithmetic encoding of pixels of the selected bitplane block. 11. The bitplane coding method according to claim 10, wherein the selected bitplane block of the depth information map block is formed using a run length coding method and a variable length coding method. 12. The method of claim 10, wherein context-based adaptive arithmetic encoding is applied to a selected bitplane block of a depth information map block. The bitplane coding method according to claim 10, wherein the selected bitplane block of the depth information map block is a CAC or JBIG method or a quad tree method. A method and apparatus for adaptively decoding a block unit bitplane coding method and a conventional block unit video coding method. Decoding a depth information map block, comprising: decoding a bitplane; Converting each pixel of the depth information map block into a gray code; Combining each bitplane; Constructing a depth information map; And an entire decoding step of decoding the entire block. 19. The method of claim 18, further comprising: performing a gray code conversion on each pixel of the reference picture and the reconstructed picture in decoding a block of the bitplane of the depth information map; Bitplane separation for the reference image and the reconstructed image; Separating decoding information of the selected bitplane block; Compensating for the movement of the selected bitplane block; Decoding the same level block for the selected bitplane block; CAE decoding the selected bitplane block; And an entire decoding step of decoding the entire bitplane block. 20. The method of claim 19, wherein the predicted value of the motion vector of the selected bitplane block of the depth information map block is determined using the motion information of the neighboring bitplane block or the depth information map block adjacent to the selected bitplane block. 20. The method of claim 19, further comprising: CAE decoding the selected bitplane block of the depth information map block, comprising: transform ratio decoding for the selected bitplane block; Downsampling a reference bitplane block; A context calculation step using a reconstructed bitplane image and a reference bitplane block or a downsampled reference bitplane block; An arithmetic decoding step using a probability table; Upsampling for the decoded bitplane block; And an entire decoding step of decoding the entire bitplane block. 22. The method of claim 21, wherein in CAE decoding the selected bitplane block of the depth information map block, constructing a context template according to the mode of the selected bitplane block. 22. The method of claim 21, wherein in the CAE decoding of the selected bitplane block of the depth information map block, the arithmetic decoding of pixels of the selected bitplane block, scanning the pixels of the selected bitplane block. 22. The method of claim 21, wherein the selected bitplane block of the depth information map block is formed using a run length decoding method and a variable length decoding method. 22. The method of claim 21, wherein context-based adaptive arithmetic decoding is applied to the selected bitplane block of the depth information map block. 22. The bitplane coding method according to claim 21, wherein the selected bitplane block of the depth information map block is a CAC or JBIG method or a quad tree method.

Images