KR20150114355A - Method and apparatus for encoding of video using depth image - Google Patents

Abstract

Disclosed are a method and an apparatus for encoding a video by using depth information. The method for encoding a video by using depth information includes the steps of: preparing a current coding unit (CU); identifying object information of the current CU from object information obtained from a depth image; and checking whether the current CU comprises a single object based on the object information and then predicting a divided structure of the current CU depending on whether the current CU comprises a single object.

Description

TECHNICAL FIELD The present invention relates to a video encoding method and apparatus using depth information,

The present invention relates to video encoding using depth information, and more particularly, to a method and apparatus for efficiently encoding an image by deriving object information using depth information.

Depth information images are widely used in 3D video encoding and have depths in new input devices such as Kinect cameras on Xbox game machines, Intel SENZ3D webcams, iSense 3D scanners on IPad, Google Tango smartphones, Information cameras can be used in many different 3D and 2D application applications.

On the other hand, 2D / 3D application applications are popularized with more 2D / 3D application services due to popularization and diffusion of depth information cameras, so that multimedia camera systems can be used for various information including depth camera. Do.

US Patent Publication No. 20140085416 entitled " Method and apparatus for compressing texture image in 3d video coding " Korean Patent Publication No. 2012-0137305 (entitled " Method for dividing a block and apparatus using such a method) Korean Patent Publication No. 2014-0048784 entitled " Method and Apparatus for Deriving Motion Information by Sharing Limited Depth Information Value "

An object of the present invention is to provide an image encoding method and apparatus capable of efficient encoding without deterioration in performance by using depth information in two-dimensional video encoding.

 A method of encoding an image using depth information according to an exemplary embodiment of the present invention includes: selecting a current coding unit (CU); Checking object information of the current CU from object information obtained from a depth image; And determining whether the current CU is a single object based on the object information and predicting a division structure of the current CU according to whether or not the current CU is composed of a single object.

According to another embodiment of the present invention, there is provided a method of encoding an image using depth information, comprising: selecting a current coding unit (CU); Determining whether the current CU is a single object based on object information of the current CU if the size of the current CU is greater than a predetermined value; And if the current CU is a single object, checking the object information of the neighboring CU of the current CU or the object information of the reference CU of the current CU, and based on the object information of the neighboring CU or the object information of the reference CU, And predicting the division structure of the current CU.

According to another embodiment of the present invention, there is provided a method of encoding an image using depth information, comprising: obtaining object information of a maximum coding unit (LCU) from a depth image; Predicting partitioned structure candidates of the LCU based on the object information; And determining an optimal partitioning structure among partitioning structure candidates of the LCU based on at least one of coding efficiency and picture quality information.

An apparatus for encoding an image using depth information according to an embodiment includes an object information verifying unit for verifying object information of a current coding unit (CU) from object information obtained from a depth image; And a partition structure predicting unit for determining whether the current CU is a single object based on the object information and predicting a partition structure of the current CU according to whether or not the current CU is a single object.

The image encoding apparatus using depth information according to another embodiment confirms the size of a current coding unit (CU), and if the size of the current CU is greater than a preset value, An object information verifying unit for verifying whether the current CU is composed of a single object based on the information; And if the current CU is a single object, checking the object information of the neighboring CU of the current CU or the object information of the reference CU of the current CU, and based on the object information of the neighboring CU or the object information of the reference CU, And a division structure predicting unit for predicting a division structure of the current CU.

According to another embodiment of the present invention, there is provided an image encoding apparatus using depth information, comprising: an object information verifying unit for verifying object information of a maximum coding unit (LCU) from a depth image; A division structure predicting unit for predicting the division structure candidates of the LCU based on the object information; And an optimal partitioning structure determining unit for determining an optimal partitioning structure among partitioning structure candidates of the LCU based on at least one of coding efficiency and picture quality information.

According to an embodiment of the present invention, a 2D general image is encoded using a depth information image acquired from a depth information camera, thereby efficiently encoding 2D images.

1 is an exemplary view of a depth information map of a general image and a general image.
Figure 2 is an illustration of a Kinect input device.
Figure 3 shows a webcam product.
Figure 4 shows an iSense 3D scanner device.
Figure 5 shows a Google Tango smartphone.
6 is a diagram for explaining an HEVC encoding apparatus.
7 is a diagram for explaining an example of image encoding using a HEVC encoder in a smartphone.
8 is a view for explaining an example of an HEVC including a depth image in a smartphone.
Fig. 9 shows examples of dividing into a plurality of units of an image.
FIG. 10 shows an example of determining a CU division structure in units of LCUs.
11 is a diagram showing an example of a depth information image and labeling information.
12 is a diagram showing an example of a general image.
13 is a diagram showing an example of a depth information map for the general image of Fig.
14 illustrates an image encoding method according to an embodiment of the present invention.
FIG. 15 illustrates an image encoding method according to another embodiment of the present invention.
16 illustrates an image encoding method according to another embodiment of the present invention.
17 and 18 are flowcharts for explaining an example of a divided structure predicting process according to an embodiment of the present invention.
19 is a diagram for explaining a configuration of an image encoding apparatus according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Thus, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. Furthermore, all of the conditional terms and embodiments listed herein are, in principle, intended only for the purpose of enabling understanding of the concepts of the present invention, and are not intended to be limiting in any way to the specifically listed embodiments and conditions .

For example, it should be understood that the block diagrams herein represent conceptual aspects of exemplary circuits embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudo code, and the like are representative of various processes that may be substantially represented on a computer-readable medium and executed by a computer or processor, whether or not the computer or processor is explicitly shown .

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary view of a depth information map of a general image and a general image. Figure 2 is an illustration of a Kinect input device.

Referring to FIG. 1, FIG. 1 (A) shows an image actually photographed through a camera, and FIG. 1 (B) shows a depth image of a real image, that is, a depth information image (or a depth information map). Depth Information means information indicating the distance between a camera and an actual object.

This depth information image is mainly used to generate three-dimensional virtual viewpoint images. As a result of this study, a joint standardization group of ISO / IEC Moving Picture Experts Group (MPEG) and ITU-T VCEG (Video Coding Experts Group) 3D video standardization is currently underway in JCT-3V (Joint Collaborative Team on 3D Video Coding Extension Development).

The 3D video standard includes standards for advanced data formats and related technologies that can support stereoscopic images as well as stereoscopic images using general images and their depth information images.

In November 2010, Microsoft released Kinect sensor as a new input device for XBOX-360 game devices. It is a device that recognizes human motion and connects it to a computer system. As shown in Figure 2, But also includes a 3D Depth sensor. In addition, the Kincet can generate RGB images and depth information maps (Depth Map) up to 640x480 with the image device and provide them to the connected computer. In 2014, Intel announced a 720p CREATIVE SENZ3D webcam with a 320x240 depth sensor for laptops. Apple released iSense as a 3D scanner for iPad with RGB camera and Depth sensor, and Google launched Tango Smart Phone.

Figure 3 shows a webcam product.

Referring to FIG. 3, a CREATIVE SENZ3D webcam is shown, wherein FIG. 3A shows a SENZ3D webcam product and FIG. 3B shows a SENZ3D webcam prototype.

Figure 4 shows an iSense 3D scanner device, and Figure 5 shows a Google Tango smartphone.

4 (A) shows an iSense product, and (B) shows an example of a scanning process through iSense. FIG. 5A shows a Google Tango smartphone product, and FIG. 5B shows a Google Tango smartphone prototype.

The advent of visual devices such as Kinect, iSense 3D Scanner, Intel SENZ3D webcam, and Google Tango smartphone has made it possible to popularize a variety of applications such as high-end 2D and 3D games and video services, An information camera or a video device with a sensor is being popularized.

As such, the future video system is not only a service for two-dimensional general video, but also a type in which a two-dimensional and three-dimensional application video service is basically provided by combining a general video camera with a depth camera, or an input assistance in a handheld system It is expected to develop in the form of devices.

A video system in which a general camera and a depth camera are fundamentally combined is a new method of using depth information in a two-dimensional video codec as well as using depth information in a three-dimensional video codec.

Also, in a camera system including a depth information camera, general image encoding can be performed using the existing video codec as it is. AVC, MVC, SVC, HEVC, SHVC, 3D-AVC, MPEG-2, MPEG-4, H.261, H.262, H.263, H.264 / AVC, , 3D-HEVC, VC-1, VC-2, VC-3, and the like, and can be encoded with various other codecs.

6 is a diagram for explaining an HEVC encoding apparatus.

As an example of a method of encoding an actual image and its depth information map, the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG), which have the highest coding efficiency among the video coding standards developed so far, Encoding can be performed using one HEVC (High Efficiency Video Coding). An example of a coding structure of HEVC is shown in Fig. As shown in FIG. 6, the HEVC includes various new algorithms such as coding unit and structure, inter prediction, intra prediction, interpolation, filtering, and transform. 6 is a block diagram showing an example of the configuration of the image encoding apparatus, and shows the configuration of an encoding apparatus according to the HEVC codec.

At this time, SAO (Sample Adaptive Offset) may be provided between the filter unit of FIG. 6 and the reference image buffer. The SAO may add a proper offset value to the pixel value to compensate for coding errors.

Since the HEVC performs inter prediction coding, i.e., inter prediction coding, the currently encoded image needs to be decoded and stored for use as a reference image. Accordingly, the quantized coefficients are inversely quantized in the inverse quantization unit and inversely transformed in the inverse transformation unit. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 175 and a reconstruction block is generated. The reconstruction block that has passed through the filter section is stored in the reference image buffer.

7 is a diagram for explaining an example of image encoding using a HEVC encoder in a smartphone. 8 is a view for explaining an example of an HEVC including a depth image in a smartphone.

Referring to FIG. 7, a general form in a smartphone to which an HEVC encoder is applied may be a form in which an image photographed through a smartphone is encoded using an HEVC encoder, and a service is provided with a coded image.

However, if an image is captured through a smart phone including a Depth camera, the texture and the depth are independently generated as shown in FIG. 8, and the optimization is performed using the correlation with the general image using the depth information Through the HEVC encoder through the reduction of the complexity can get better images.

In the conventional technique disclosed in U.S. Patent Publication No. 20140085416, a block is merged by confirming information about an object of a current block from a depth map. However, It has not been disclosed at all whether to divide and encode the unit.

Korean Patent Laid-Open Publication No. 2001-0137305 and Japanese Laid-Open Patent Application No. 2014-0048784, which are related to Patent Document 2, can not disclose contents using depth information, It is not clearly presented.

Fig. 9 shows examples of dividing into a plurality of units of an image.

The high-efficiency video coding scheme performs coding by dividing an image into LCU (LCU: Largest Coding Unit) units, which is a basic unit of a coding unit (CU) when performing coding.

Here, the CU plays a role similar to a macro block (MB), which is a basic block in H.264 / AVC, which is a conventional video codec, but unlike an MB having a fixed size of 16x16, You can set the size to. In addition, the divided LCU for encoding can be divided into several CUs having a smaller size than the LCU for efficient encoding of the image.

Referring to FIG. 9, a 64x64 LCU may be divided into a plurality of CUs in various manners.

9A shows an example in which a 64x64 LCU having a division depth value of 0 is divided into 32x32 CUs having a division depth of 1. FIG.

FIG. 9B shows an example of dividing one of CUs having a size of 32x32 by a division depth 2, and FIG. 9C shows an example of dividing two of 32x32 CUs into a division depth 2.

FIG. 9D shows an example including CUs divided by the division depth 3.

Thus, the LCU or CU partition structure candidates may exist in various ways.

The LCU division structure is division information of the encoding unit. In the step of determining the optimal LCU partition structure after creating the various LCU partition structures as described above and storing them in the candidate LCU partition structure, one of the LCU partition structure candidates is selected as the optimal LCU partition structure in units of LCU . By using this method, encoding is performed on the basis of an LCU division structure adaptive to the characteristics of an image in units of LCU, so that efficient encoding can be performed in terms of encoding efficiency and image quality.

FIG. 10 shows an example of determining a CU division structure in units of LCUs.

In the HEVC encoder, it is possible to differently determine the division method of the encoding unit in the LCU according to the characteristics of the image. That is, in order to determine an optimal LCU division structure, various types of division structures can be encoded. As a method for determining an optimal LCU division structure, a prediction mode (Intra, Inter mode, etc.) , And then a prediction mode of the corresponding encoding unit is determined according to the amount of bits to be encoded and the image quality.

For example, as shown in FIG. 9, a 64x64 coding unit having a depth of 0 is coded using an Intra mode and an Inter mode, and an optimal mode is stored. A 64x64 coding unit is divided into four, The encoding can be performed recursively in the Intra mode and the Inter mode, respectively. In this case, the prediction mode can be selected independently for each of the four 32x32 size coding units. Also, encoding is performed by dividing into 4 16x16 encoding units even in a 32x32 encoding unit. The encoding unit is divided and encoded in the recursive manner, and the most efficient division unit of the encoding unit is determined in terms of bit amount and image quality.

For example, if the bit amount and image quality of the encoded four 32x32 size encoding units are more efficient than those encoding with 64x64 size encoding units, the encoding unit is determined to be divided into four 32x32 size encoding units. When encoding an image in an encoder (image encoder), the optimal encoding unit distribution is searched for every number of cases where the LCU can be divided, which increases the computational complexity of the encoder.

For efficient compression performance, the computational complexity increases as the coding efficiency is judged for many cases.

11 is a diagram showing an example of a depth information image and labeling information.

The method shown in FIG. 10 and the conventional CU division structure determination method do not consider object information for neighboring blocks. This is because, in the case of the 2D image, if there is no depth camera, the object information in the image must be extracted through the 2D image analysis. Therefore, there is no method using the object information in the existing 2D video coding method.

For the same reason, in the case of HEVC, there is no coding method using object information at all in the method of determining the division structure. However, if the object information can be considered by using the Depth camera, it is possible to know the correlation according to the object configuration information of the corresponding CU in the division structure determination method, and if the division structure can be determined with efficient prediction, the coding complexity is efficiently reduced.

Accordingly, when the depth information is used in the partition structure determination method, the coding complexity can be efficiently reduced when determining the object information of the corresponding CU or the structure of the object region and predicting an efficient partition structure for the corresponding region.

The object information derived from the depth information can be obtained by directly converting the depth information into a simple object information image by applying a labeling algorithm to the depth information image. FIG. 11A shows a depth information image, and FIG. 11B shows an object information image labeled by applying a Watersherd labeling algorithm to the image shown in FIG.

12 is a diagram showing an example of a general image. 13 is a diagram showing an example of a depth information map for the general image of Fig.

As shown in FIG. 12 and FIG. 13, the portion corresponding to the object boundary region has a high probability of having a complicated CU partition structure, and is likely to have a relatively simple partition structure when it is judged as an object or background region.

Therefore, in the case of obtaining the object information for the current encoding region using the depth information, it is determined that the current encoding region is determined as a complex division structure composed of a plurality of CUs or a simple division structure composed of a small number of CUs. Can be predicted. This can reduce the amount of computation by limiting the determination of low-probability partition structures. The method proposed in the present invention can predict and encode a highly probable partition structure using depth information in the partition structure determination.

In the conventional two-dimensional video codec, algorithms are designed without reflecting the use of depth information at all. However, since the actual image and its depth information image have a large correlation, it is possible to utilize the depth information to encode the two-dimensional image, Can be considered.

The basic principle of the present invention is to use the depth information in the motion estimation method to efficiently use the depth information obtained from the depth information camera for encoding in the 2D video codec.

For example, when the objects of the general image are classified by using the depth information image, the coding complexity of the general image can be greatly reduced.

In the block-based encoding codec, a plurality of objects may exist in the block, and different encoding methods may be applied to each object based on the depth information image. .

14 illustrates an image encoding method according to an embodiment of the present invention.

Referring to FIG. 14, in step 1410, the image encoding apparatus selects a current coding unit to be a candidate for a partition structure candidate determination.

In step 1420, the image encoding apparatus checks the object information of the current CU from the object information obtained from the depth image.

In step 1430, the image encoding apparatus checks whether the current CU is a single object based on the object information, and predicts the division structure of the current CU according to whether the current CU is a single object or not.

For example, the partition structure of the CU predicted in step 1430 may be (A), (B), or (D) shown in FIG.

According to the embodiment shown in FIG. 14, it is possible to predict the partition structure of the CU by determining object information of the CU through the depth information. When it is judged that a certain CU has a single object through the depth information, the coding complexity can be reduced by determining the division structure candidate by not dividing the CU further.

Accordingly, in step 1430 of predicting the current partition structure of the CU, it may include determining that the current CU is not partitioned if the current CU is a single object.

In step 1430, when the current CU and the neighboring CUs of the current CU are each a single object and the current CU and the neighboring CUs of the current CU belong to the same object, the image encoding apparatus determines not to divide the current CU The division structure prediction may be terminated.

If the current CU and the neighboring CUs of the current CU are each a single object and the neighboring CUs of the current CU and the current CU belong to the same object in step 1430, It may be determined that the current CU is not divided and the divided structure prediction may be terminated if it is greater than a predetermined value.

In step 1430, the image encoding apparatus may determine not to divide the current CU if the size of the current CU is smaller than a preset value and the current CU is a single object.

Whether or not an arbitrary CU is composed of a single object can be determined through distribution of a depth value distribution of a CU obtained from a depth information image or distribution of a label value of a CU obtained from an object information label image.

For example, if the label value distribution of the CU is a single value, or if the difference between the maximum value and the minimum value of the depth value distribution of the CU is equal to or less than a predetermined value, it can be determined that the object information of the CU is a single object.

Therefore, the object information of the CU may be "distribution of label value of CU" or "distribution of depth value of CU" or "

FIG. 15 illustrates an image encoding method according to another embodiment of the present invention.

Fig. 15 shows an example of determining an optimal division structure in LCU units.

Referring to FIG. 15, in step 1510, the image encoding apparatus obtains object information of a maximum coding unit (LCU) from a depth image.

At this time, the object information may be the label value of the object information image labeled by the labeling algorithm.

In step 1520, the image encoding apparatus predicts the segmentation structure candidates of the LCU based on the object information.

In this case, in step 1520, it is determined whether the current coding unit is composed of a single object based on object information of the current coding unit, and if the current coding unit is a single object and the size of the current coding unit is smaller than a preset value And ending the division structure candidate prediction process if the division candidate prediction process is completed.

In step 1530, the image encoding apparatus determines an optimal division structure among the divided structure candidates of the LCU based on at least one of coding efficiency and image quality information.

According to the embodiment shown in FIGS. 14 and 15, the division structure of the current CU can be determined through the depth information of the current CU and the reference CU.

For example, in the case of a CU having a size of 32x32, if the current CU is configured as a single object through the depth information, the upper block and the left block of the current CU are determined as the same object as the current CU, If the size of the CU is 32x32 or more, the optimal CU division structure can be determined and terminated without further dividing the current CU.

If the upper block and the left block CU can not be used (for example, the current CU is located at the edge of the screen and there is no upper or left block), if the reference CU is composed of a single object and the size is 32x32 or more, The optimal CU partition structure can be determined and terminated without further partitioning the current CU.

In addition, when the CU having a size of 16x16 is determined that the current CU is composed of a single object through the depth information, the image encoding apparatus can determine the optimal CU division structure and terminate the current CU without further dividing the current CU .

16 illustrates an image encoding method according to another embodiment of the present invention.

FIG. 16 is a flowchart illustrating the LCU unit division structure determination process shown in FIG. 15 in more detail. For example, step 1510 of FIG. 15 may include steps 1601 and 1605 of FIG. 15 may include steps 1610 to 1630, 1660, and 1670 of FIG.

In step 1610, the image encoding apparatus starts the CU unit recursion process. In step 1620, the image encoding apparatus encodes the CU of the current division depth and stores the CU in the optimal CU candidate. For example, the CU of the current division depth may be (A) in FIG.

In step 1630, the image encoding apparatus performs an early determination process of a divided structure using depth information. At this time, the division structure early determination process may be a division structure candidate prediction process shown in FIGS. 14, 17, and 18. FIG.

The CU may be determined by dividing the CU through an early determination process of the partition structure in step 1630. In step 1660, the CU may be divided into four CUs having a 1 partition depth increased as shown in FIG. 9B.

9 (A), (B), and (C), the image encoding apparatus determines the CU division structure candidates in step 1650 in consideration of the coding efficiency and the like for each of the divided structure candidates The optimal CU partition structure can be determined.

Once the optimal CU partition structure is determined, the CU unit recursion process may be terminated at step 1650.

17 and 18 are flowcharts for explaining an example of a divided structure predicting process according to an embodiment of the present invention.

Referring to FIG. 17, the current CU encoding and CU candidate storing step 1701 may be the same as step 1605 of FIG.

In step 1703, the image coding apparatus selects a current coding unit (CU) and determines whether the current CU size is 32x32 or more.

If the size of the current CU is 32x32 or more, step 1705 is performed; otherwise, step 1731 is performed.

In step 1705, it is determined whether the current CU is a single object. If the CU is a single object, step 1707 is performed. Otherwise, step 1711 is performed.

In step 1707, it is determined whether the upper and left CUs of the current CU are available and are made of the same object as the current CU, and each of them is composed of a single object.

If the condition of step 1707 is satisfied, step 1709 is performed. Otherwise, step 1721 is performed.

If it is determined in step 1709 that the sizes of the upper and left CUs of the neighboring CUs are 32x32 or more, it is determined that the current CU is not divided any more, and step 1720 of determining the optimal CU division structure may be performed.

If it is determined in step 1709 that the sizes of the upper and left CUs are not 32x32 or more, step 1711 is performed.

If it is possible to divide by referring to the division depth of the current CU in step 1711, the CU is divided in step 1713, and the recursive process calling step according to CU is performed in step 1715.

In step 1711, if the current depth of the CU is the largest depth and the CU can not be further divided, step 1720 can be performed.

Referring to steps 1703 and 1705, if the size of the current CU is greater than a preset value, the method determines whether the current CU is a single object based on the object information of the current CU .

Referring to steps 1707, 1709, and 1721, when the current CU is a single object, the image encoding method according to an exemplary embodiment of the present invention includes: And predicting the partition structure of the current CU based on the object information of the neighboring CU or the object information of the reference CU.

In this case, the neighboring CU is the upper CU and the left CU of the current CU, and the step of predicting the division structure of the current CU includes a step of determining whether each of the upper CU and the left CU consists of a single object, And determines that the current CU is not divided if the size of the upper CU and the left CU is greater than or equal to the preset value, and if the size of the upper CU and the left CU is smaller than the predetermined value And determining that the current CU is further divided by one level.

If it is determined in step 1707 that the upper CU and the left CU are not each a single object, it is determined in step 1721 whether the reference CU is composed of a single object.

More specifically, if it is determined in step 1721 that the reference CU is available and composed of a single object, it is determined in step 1723 whether the size of the reference CU is 32x32 or more.

If it is determined in step 1723 that the size of the reference CU is greater than or equal to 32x32, it is determined that the current CU is no longer divided, and step 1720 of determining the optimal CU partition structure may be performed.

Referring to steps 1721 and 1723, the image encoding method according to an embodiment determines that the current CU is not divided if the reference CU is a single object and the size of the reference CU is greater than or equal to the preset value, And determining that the current CU is further divided by one level if the reference CU is not made of a single object. At this time, dividing the CU by one level means that the current CU is divided into four CUs whose division depth is increased by one.

If it is determined in step 1703 that the size of the CU is not 32x32 or more, it is determined in step 1731 whether the size of the CU is 16x16. If the size of the CU is 16x16, it is determined in step 1733 whether the current CU is a single object.

If it is determined in step 1733 that the current CU is composed of a single object, it is determined that the current CU is no longer divided, and step 1720 of determining the optimal CU division structure may be performed.

According to an embodiment of the present invention, there is a method of determining the same object and a single object, a method of obtaining object information from a depth information label image by applying a labeling algorithm from depth information, and object information is expressed by a label It is possible to make a judgment on the same object or a single object through the label distribution in the object area in the object configuration, whether the label is the same or not.

Accordingly, the object information of the current CU, the object information of the neighboring CU, and the object information of the reference CU may be the label value of the object information image labeled by the labeling algorithm.

Referring to FIG. 18, steps 1801, 1820, 1813, and 1815 are the same as steps 1701, 1720, 1713, and 1715 of FIG.

In step 1803, the image encoding apparatus selects a current coding unit (CU) and determines whether the size of the current CU is 32x32 or more.

If the current CU size is 32x32 or more, step 1805 is performed; otherwise, step 1831 is performed.

In step 1805, it is determined whether the current CU is composed of a single label. If the CU is composed of a single label, step 1807 is performed. Otherwise, step 1811 is performed.

At this time, whether or not the label is a single label can be set when the label values included in the CU are all the same value or when the difference between the maximum value and the minimum value is within a predetermined range.

In step 1807, it is determined whether the upper and left CUs of the current CU are available and are made of the same label as the current CU, and are made up of a single label.

If the condition of step 1807 is satisfied, step 1809 is performed. Otherwise, step 1821 is performed.

If it is determined in step 1809 that the sizes of the upper and left CUs of the neighboring CUs are 32x32 or more, it is determined that the current CU is no longer divided, and step 1820 of determining the optimal CU division structure may be performed.

If it is determined in step 1809 that the sizes of the upper and left CUs are not 32x32 or more, step 1811 is performed.

If it is possible to divide by referring to the division depth of the current CU in step 1811, the CU is divided in step 1813 and the recursive process calling step according to CU is performed in step 1815.

In step 1811, if the current depth of the CU is the largest depth and the CU can not be further divided, step 1820 may be performed.

Referring to steps 1803 and 1805, if the size of the current CU is greater than or equal to a preset value, the image encoding method according to an embodiment determines whether the current CU is a single object based on the object information of the current CU .

In addition, referring to steps 1807, 1809, and 1821, the image encoding method according to an embodiment of the present invention includes the step of, when the current CU is a single object, the object information of the neighboring CU of the current CU or the object information of the reference CU of the current CU And predicting the partition structure of the current CU based on the object information of the neighboring CU or the object information of the reference CU.

In this case, the neighboring CU is the upper CU and the left CU of the current CU, and the step of predicting the division structure of the current CU includes a step of determining whether each of the upper CU and the left CU consists of a single object, And determines that the current CU is not divided if the size of the upper CU and the left CU is greater than or equal to the preset value, and if the size of the upper CU and the left CU is smaller than the predetermined value And determining that the current CU is further divided by one level.

If it is determined in step 1807 that the upper CU and the left CU are not formed as a single label, it may be determined in step 1821 whether the reference CU is composed of a single label.

More specifically, if it is determined in step 1821 that the reference CU is available and consists of a single label, it is determined in step 1823 whether the size of the reference CU is 32x32 or more.

If it is determined in step 1823 that the size of the reference CU is greater than or equal to 32x32, it is determined that the current CU is no longer divided, and step 1820 of determining the optimal CU partition structure is performed.

Referring to steps 1821 and 1823, the image encoding method according to an embodiment determines that the current CU is not divided if the reference CU is a single object and the size of the reference CU is greater than or equal to the preset value, And determining that the current CU is further divided by one level if the reference CU is not made of a single object. At this time, dividing the CU by one level means that the current CU is divided into four CUs whose division depth is increased by one.

If the size of the CU is not greater than 32x32, it is determined in step 1831 whether the size of the CU is 16x16. If the size of the CU is 16x16, it is determined in step 1833 whether the current CU is composed of a single label.

If it is determined in step 1833 that the current CU is composed of a single label, it is determined that the current CU is not divided any more, and step 1820 of determining the optimal CU division structure may be performed.

19 is a diagram for explaining a configuration of an image encoding apparatus according to an embodiment of the present invention.

 Referring to FIG. 19, the image encoding apparatus 1900 may include an object information verifying unit 1910, a divided structure predicting unit 1920, and an optimal partitioning structure determining unit 1930. The image encoding apparatus 1900 may further include an object information generating unit for obtaining a label value of the object information image labeled by the labeling algorithm.

The object information verification unit 1910 extracts the object information of the current coding unit (CU) from the object information obtained from the depth image, and confirms the object information.

The object information identifying unit 1910 checks the size of the current coding unit (CU) and, based on the object information of the current CU obtained from the depth image, if the size of the current CU is greater than a predetermined value It can be confirmed whether the current CU is a single object.

In addition, the object information identifying unit 1910 extracts object information of a maximum coding unit (LCU) from the depth image and confirms the object information.

The partition structure predicting unit 1920 can determine whether the current CU is a single object based on the object information and predict the partition structure of the current CU according to whether the current CU is a single object or not.

At this time, the partition structure predicting unit 1920 may determine that the current CU is not divided if the current CU is a single object.

At this time, if the current CU and the neighboring CUs of the current CU are each a single object and the neighboring CUs of the current CU and the current CU belong to the same object, the partitioned structure predicting unit 1920 does not divide the current CU And the division structure prediction can be terminated.

When the current CU and the neighboring CUs of the current CU are each a single object and the neighboring CUs of the current CU and the current CU belong to the same object, the split structure predicting unit 1920 predicts the size of the neighboring CU It is determined that the current CU is not divided and the division structure prediction can be terminated if it is greater than a predetermined value.

In addition, the partition structure predicting unit 1920 may determine that the current CU is not partitioned when the size of the current CU is smaller than a preset value and the current CU is a single object.

In addition, when the current CU is a single object, the partition structure predicting unit 1920 may check the object information of the neighboring CU of the current CU or the object information of the reference CU of the current CU, The division structure of the current CU can be predicted based on the object information of the reference CU.

In addition, the divided structure predicting unit 1920 can predict the divided structure candidates of the LCU based on the object information.

The optimum division structure determination unit 1930 determines an optimal division structure among the division structure candidates of the LCU based on at least one of coding efficiency and picture quality information.

[Table 1] shows experimental results of applying the embodiment shown in FIG. 18 to HEVC.

Experimental results show that the coding complexity is reduced without degradation of image quality with the same quality in subjective image quality.

[Table 1]

In the embodiments of the present invention, the object range or the application range may vary depending on the block size or the division depth of the CU.

In this case, the variable (i.e., size or depth information) for determining the application range may be set to use a predetermined value by the encoder or the decoder, use a predetermined value according to the profile or level, Is described in the bit stream, the decoder may obtain the value from the bit stream and use it. When the application range is changed according to the CU division depth, it is as shown in [Table 2]. The method A may be a method which applies only to a depth of a predetermined depth value or more, the method B may be a method which applies only to a predetermined depth value or less, and the method C may have a method of applying only a predetermined depth value.

[Table 2]

Table 2 shows an example of a method of determining the application range to which the methods of the present invention are applied when the given CU division depth is 2. (O: applied to the depth, X: not applied to the depth.)

In the case where the methods of the present invention are not applied to all the depths, an arbitrary indicator may be used to indicate in the bitstream, and a value one greater than the maximum value of the CU depth may be used as the CU depth value indicating the coverage, .

In addition, each of the above-described methods can be applied to the color difference block differently depending on the size of the luminance block, and can be applied to the luminance signal image and the color difference image.

[Table 3] shows an example in which different methods are applied depending on the size of the luminance block and the color difference block.

[Table 3]

Among the modified methods of Table 3, Method 1 is the case where the size of the luminance block is 8 (8x8, 8x4, 2x8, etc.) and the size of the color difference block is 4 (4x4, 4x2, 2x4) The method of constructing a merge list according to an embodiment of the present invention can be applied to a luminance signal and a color difference signal.

Among the above-mentioned modified methods, if the luminance block is 16 (16x16, 8x16, 4x16) and the color difference block is 4 (4x4, 4x2, 2x4) The merge list construction method according to the embodiment of the present invention may be applied to the luminance signal and not to the color difference signal.

In another modified method, the merge list construction method according to the embodiment of the present invention is applied only to the luminance signal, and may not be applied to the color difference signal. Conversely, the merge list construction method according to the embodiment of the present invention is applied only to the color difference signal, and may not be applied to the luminance signal.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

Selecting a current coding unit (CU);
Checking object information of the current CU from object information obtained from a depth image; And
Determining whether the current CU is a single object based on the object information, and predicting a division structure of the current CU according to whether or not the current CU is composed of a single object
The depth information including the depth information. The method according to claim 1,
Wherein the step of predicting the partition structure of the current CU comprises:
And determining that the current CU is not divided if the current CU is a single object
Image coding method using depth information. The method according to claim 1,
Wherein the step of predicting the partition structure of the current CU comprises:
If the current CU and the neighboring CUs of the current CU are each a single object and the neighboring CUs of the current CU and the current CU belong to the same object, the current CU is determined not to be divided and the split structure prediction is terminated Process
Image coding method using depth information. The method according to claim 1,
Wherein the step of predicting the partition structure of the current CU comprises:
If the current CU and the neighboring CUs of the current CU are each a single object and the neighboring CUs of the current CU and the current CU belong to the same object, And ending the partition structure prediction.
Image coding method using depth information. The method according to claim 1,
Wherein the step of predicting the partition structure of the current CU comprises:
Determining that the current CU is not divided if the size of the current CU is smaller than a preset value and the current CU is a single object
Image coding method using depth information. Selecting a current coding unit (CU);
Determining whether the current CU is a single object based on object information of the current CU if the size of the current CU is greater than a predetermined value; And
If the current CU is a single object, confirms the object information of the neighboring CU of the current CU or the object information of the reference CU of the current CU and updates the current CU based on the object information of the neighboring CU or the object information of the reference CU Step of predicting the partition structure of CU
The depth information including the depth information. The method according to claim 6,
Wherein the neighboring CU is an upper CU and a left CU of the current CU,
Determining whether each of the upper CU and the left CU comprises a single object;
And determines that the current CU is not divided if the upper CU and the left CU are each a single object and the size of the upper CU and the left CU is greater than or equal to the predetermined value, Determining that the current CU is further divided by one level if it is smaller than the set value
The depth information including the depth information. 8. The method of claim 7,
Wherein the step of predicting the partition structure of the current CU comprises:
Determining whether the reference CU is composed of a single object when the upper CU and the left CU are not each a single object; And
Determining that the current CU is not divided if the reference CU is a single object and the size of the reference CU is greater than or equal to the predetermined value and if the reference CU is not a single object, Step of determining to divide
The depth information including the depth information. 9. The method according to any one of claims 1 to 8,
Wherein the object information is a label value of an object information image obtained by labeling a depth image by a labeling algorithm
Image coding method using depth information. 9. The method according to any one of claims 6 to 8,
The object information of the current CU, the object information of the neighboring CU, and the object information of the reference CU are label values of an object information image labeled by a labeling algorithm
Image coding method using depth information. Obtaining object information of a maximum coding unit (LCU: Largest Coding Unit) from a depth image;
Predicting partitioned structure candidates of the LCU based on the object information; And
Determining an optimal partitioning structure among partitioning structure candidates of the LCU based on at least one of coding efficiency and picture quality information
The depth information including the depth information. 13. The method of claim 12,
Wherein the object information is a label value of an object information image obtained by labeling a depth image by a labeling algorithm
Image coding method using depth information. 12. The method of claim 11,
The step of predicting the division structure candidates of the LCU comprises:
Determining whether the current coding unit comprises a single object based on object information of a current coding unit; And
If the current coding unit is a single object and the size of the current coding unit is smaller than a predetermined value, ending the division structure candidate prediction process
The depth information including the depth information. An object information verifying unit for verifying object information of a current coding unit (CU) from object information obtained from the depth image; And
A division structure predicting unit for predicting a division structure of the current CU according to whether the current CU is composed of a single object based on the object information,
The depth information including the depth information. 15. The method of claim 14,
The partition structure predicting unit determines that the current CU is not divided if the current CU is a single object
Image encoding apparatus using depth information. 15. The method of claim 14,
Wherein the partition structure predicting unit determines that the current CU and the neighboring CUs of the current CU are each a single object and that the current CU and the neighboring CUs of the current CU belong to the same object, End structure prediction
Image encoding apparatus using depth information. 15. The method of claim 14,
If the current CU and the neighboring CUs of the current CU are each a single object and the neighboring CUs of the current CU and the neighboring CUs belong to the same object, if the size of the neighboring CU is greater than a predetermined value It is determined that the current CU is not divided and the division structure prediction is terminated
Image encoding apparatus using depth information. 15. The method of claim 14,
The partition structure predicting unit determines that the current CU is not divided if the size of the current CU is smaller than a preset value and the current CU is a single object
Image encoding apparatus using depth information. If the size of the current CU is greater than a predetermined value, it is determined whether the current CU is a single object based on the object information of the current CU obtained from the depth image An object information verifying unit; And
If the current CU is a single object, confirms the object information of the neighboring CU of the current CU or the object information of the reference CU of the current CU and updates the current CU based on the object information of the neighboring CU or the object information of the reference CU A division structure predicting unit for predicting the division structure of the CU,
The depth information including the depth information. An object information verifying unit for verifying object information of a maximum coding unit (LCU: Largest Coding Unit) from a depth image;
A division structure predicting unit for predicting the division structure candidates of the LCU based on the object information; And
An optimal division structure determining unit for determining an optimal division structure among the division structure candidates of the LCU based on at least one of coding efficiency,
The depth information including the depth information.

Images