KR102424941B1 - Apparatus And Method For Adaptably Selecting Merging Candidates In 3 Dimensional Image Encoding And Decoding - Google Patents

Abstract

According to the present invention, the steps of determining whether a merge candidate used in a neighboring block is also usable in the current block and, if the merge candidate used in the neighboring block is also available in the current block, selects the merge candidate used in the neighboring block There is provided a 3D image decoding method comprising the step of setting a merge candidate of the current block.

Description

Apparatus And Method For Adaptably Selecting Merging Candidates In 3 Dimensional Image Encoding And Decoding

The present invention relates to an apparatus and method for encoding/decoding a 3D image, and more particularly, to an apparatus and method for encoding/decoding a 3D image for adaptively selecting a merge candidate.

The development of the broadcast communication industry has caused the worldwide spread of a broadcast service having a high definition (HD) resolution, and as a result, many users have become accustomed to HD images. Users who have become accustomed to HD image quality want images with higher image quality and higher resolution, and many organizations have spurred the development of next-generation imaging devices to meet users' demands. Due to this, today, FHD (Full HD) and UHD (Ultra High Definition) supported images can be accessed, and FHD and UHD supported images enable users to view high-resolution images.

Users want not only images with high picture quality and high resolution, but also 3D images that can feel three-dimensional. For this reason, various organizations have come to develop 3D images to satisfy the needs of users.

A 3D image requires not only texture information but also depth map information. A 3D image requires a much larger amount of information than a conventional 2D image, and when encoding/decoding is performed on a 3D image using an image encoding/decoding apparatus and method for a 2D image, sufficient There was a problem that encoding/decoding efficiency did not come out.

An object of the present invention is to provide an apparatus and method for encoding/decoding a 3D image using motion information located other than a current block.

Another object of the present invention is to provide an apparatus and method for encoding/decoding a 3D image using motion information of a block adjacent to a current block.

Another object of the present invention is to provide an apparatus and method for encoding/decoding a 3D image using motion information of a block spatially adjacent to a current block.

Another object of the present invention is to provide an apparatus and method for encoding/decoding a 3D image using motion information of a block temporally adjacent to a current block.

Another object of the present invention is to provide an apparatus and method for encoding/decoding a 3D image using motion information of blocks existing at different viewpoints.

Another object of the present invention is to provide an apparatus and method for encoding/decoding a 3D image using motion information of blocks existing in a depth information map.

According to an embodiment of the present invention, determining whether a merge candidate used in a neighboring block is also available in the current block, and if the merge candidate used in the neighboring block is also available in the current block, the merge candidate used in the neighboring block and setting the merge candidate as a merge candidate of the current block.

According to another embodiment of the present invention, the step of determining whether a merge candidate used in the neighboring block is also available in the current block, and if the merge candidate used in the neighboring block is also available in the current block, the merge candidate used in the neighboring block and setting the merge candidate as a merge candidate of the current block.

According to another embodiment of the present invention, a determination unit that determines whether a merge candidate used in a neighboring block is also usable in the current block, and when a merge candidate used in a neighboring block is also available in the current block, in the neighboring block and a setting unit configured to set the used merge candidate as a merge candidate of the current block.

According to another embodiment of the present invention, a determination unit that determines whether a merge candidate used in a neighboring block is also usable in the current block, and when a merge candidate used in a neighboring block is also available in the current block, in the neighboring block and a setting unit configured to set the used merge candidate as a merge candidate of the current block.

According to the present invention, motion information located other than the current block can be used.

According to the present invention, motion information of a block adjacent to the current block may be used.

According to the present invention, motion information of a block spatially adjacent to the current block may be used.

According to the present invention, motion information of a block temporally adjacent to the current block may be used.

According to the present invention, motion information of a block existing at a different time can be used.

According to the present invention, motion information of a block existing in the depth information map can be used.

1 schematically shows the basic structure of a 3D video system.
2 is a diagram illustrating an example of an actual image and a depth information map image of a “balloons” image.
3 is a diagram schematically illustrating a split structure of an image when encoding and decoding an image.
4 illustrates a form of a prediction unit (PU) that the encoding unit (CU) may include.
5 is an example illustrating a structure of inter view prediction in a 3D video codec.
6 is an example illustrating a process of encoding and/or decoding an actual image (texture view) and a depth information map (depth view) in a 3D video encoder and/or decoder.
7 is a block diagram illustrating a configuration of a video encoder according to an embodiment.
8 is a block diagram illustrating a configuration of a video decoder according to an embodiment.
9 is a diagram illustrating an example of a prediction structure for a 3D video codec.
10 is an example illustrating neighboring blocks used to construct a merge candidate list for a current block.
11 is a flowchart illustrating a method of constructing a merge motion candidate list in inter prediction.
12 is a flowchart illustrating a method of using a motion candidate used in a neighboring block in a current block when encoding a 3D image according to an embodiment of the present invention.
13 is a flowchart of a method in which a motion candidate used in a neighboring block is used in a current block when encoding a 3D image is applied to 3D-HEVC according to another embodiment of the present invention.
14 is a block diagram of an apparatus for using, in a current block, a motion candidate used in a neighboring block when encoding a 3D image according to another embodiment of the present invention.
15 is a flowchart illustrating a method of using a motion candidate used in a neighboring block in a current block when decoding a 3D image according to another embodiment of the present invention.
16 is a block diagram of an apparatus for using a motion candidate used in a neighboring block in a current block when decoding a 3D image according to another embodiment of the present invention.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described concretely with reference to drawings. In describing the embodiments of the present specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present specification, the detailed description thereof will be omitted.

When it is said that a component is “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be In addition, the content described as “including” a specific configuration in the present invention does not exclude configurations other than the corresponding configuration, and it means that additional configurations may be included in the practice of the present invention or the scope of the technical spirit of the present invention .

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In addition, the components shown in the embodiment of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is made of separate hardware or a single software component. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each Integrated embodiments and separate embodiments of components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only the essential components to implement the essence of the present invention except for the components used for improving the performance, and a structure including only the essential components excluding the optional components used for improving the performance Also included in the scope of the present invention.

The three-dimensional video provides a three-dimensional effect as seen and felt in the real world to the user through a three-dimensional stereoscopic display device. In this regard, the Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V), a joint standardization group of the Moving Picture Experts Group (MPEG) of ISO/IEC and the Video Coding Experts Group (VCEG) of ITU-T, has Video standardization is in progress.

1 schematically shows the basic structure of a 3D video system.

Referring to FIG. 1 , a 3D video (3DV) system may include a sender and a receiver. In this case, the 3D video system of FIG. 1 may be a basic 3D video system considered in the 3D video standard, and the 3D video standard uses a real image and a depth information map corresponding to the real image to obtain a stereoscopic image. In addition, it may include standards for advanced data formats and related technologies that can support the reproduction of autostereoscopic images.

The transmitter may create multi-view video content. Specifically, the transmitting side may generate video information using a stereo camera and a multi-view camera, and may generate a depth map (or depth view) using the depth information camera. Also, the transmitting side may convert the 2D image into a 3D image by using a converter. The transmitting side may generate video content of N (N≥2) views (ie, multiple views) by using the generated video information and depth information map. In this case, the N-view image content may include N-view video information, depth-map information thereof, and additional information related to the camera. N-view video content may be compressed using a multi-view video encoding method in a 3D video encoder, and the compressed video content (bitstream) may be transmitted to a receiving terminal through a network.

A receiver may provide a multi-view image by decoding the image content received from the transmitter. Specifically, on the receiving side, a video decoder (eg, a three-dimensional video decoder, a stereo video decoder, a two-dimensional video decoder, etc.) decodes a transmitted bitstream using a multi-view video decoding method to restore an N-view image. can In this case, virtual view images of more than N views may be generated using the restored N-view image and a depth-image-based rendering (DIBR) process. The generated virtual view images of more than N views are reproduced according to various stereoscopic display devices (eg, N-view display, stereo display, two-dimensional display, etc.) to provide a three-dimensional image to the user.

2 is a diagram illustrating an example of an actual image and a depth information map image of a “balloons” image.

Figure 2 (a) shows the image of “balloons” used in the 3D video encoding standard of MPEG, an international standardization organization. FIG. 2(b) shows a depth information map image for the “balloons” image shown in FIG. 2(a). The depth information map image shown in (b) of FIG. 2 expresses depth information displayed on the screen by 8 bits per pixel.

A depth map is used to generate a virtual viewpoint image, and the depth map calculates the distance between a camera and a real object in the real world (depth information corresponding to each pixel with the same resolution as the real image) in a constant bit. expressed in numbers. In this case, the depth information map may be acquired using a depth information map camera or an actual general image (texture).

The depth information map acquired using the depth information map camera mainly provides reliable depth information in a still object or scene, but there is a problem that the depth information map camera operates only within a certain distance. In this case, the depth information map camera may use a depth measuring technique based on a laser, a structured light technique, or a time-of-flight of light (TFL) technique.

The depth information map may be generated using an actual general image (Texture) and a disparity vector (Disparity Vector). The disparity vector refers to information indicating a difference in viewpoint between two normal images. The disparity vector compares any one pixel at the current time point with pixels at another time point and finds the most similar pixel, the pixel at the current time point and the pixel at another time point (any pixel at the current time point). It can be obtained through the distance between one pixel and the most similar pixel).

The actual image and its depth information map may be images acquired from not only one camera but also multiple cameras. Images obtained from multiple cameras may be independently encoded, or encoded/decoded using a general 2D video encoding/decoding codec. In addition, since there is a correlation between views in images obtained from multiple cameras, images obtained from multiple cameras may be encoded using predictions between different views in order to increase encoding efficiency.

3 is a diagram schematically illustrating a split structure of an image when encoding and decoding an image.

In order to efficiently divide an image, encoding and decoding may be performed for each coding unit (CU). A unit is a term that combines a syntax element and a block containing image samples. The division of a unit may mean dividing a block corresponding to the unit.

Referring to FIG. 3 , after sequentially dividing an image 300 into units of a Largest Coding Unit (LCU) (hereinafter, referred to as LCU), a split structure is determined for each LCU. In this specification, LCU may be used in the same meaning as a Coding Tree Unit (CTU). The division structure means a distribution of coding units (hereinafter, referred to as CUs) for efficiently encoding an image in the LCU 310 , and this distribution divides one CU by half of its horizontal and vertical sizes. It may be determined according to whether to split into CUs. A divided CU may be recursively divided into four CUs whose horizontal size and vertical size are reduced by half with respect to the divided CU in the same manner.

In this case, the division of the CU may be recursively divided up to a predefined depth. The depth information is information indicating the size of a CU, and may be stored for each CU. For example, the depth of the LCU may be 0, and the depth of the Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, the LCU is a coding unit having the largest size as described above, and the Smallest Coding Unit (SCU) is a coding unit having the smallest size.

The depth of the CU increases by 1 whenever division is performed from the LCU 310 to half of the horizontal and vertical sizes. For example, if the size of a CU at a specific depth L is 2Nx2N, the size of the CU is still 2Nx2N when division is not performed, and the size of the CU becomes NxN when division is performed. At this time, the depth of the NxN size CU becomes the depth L+1. That is, the size of N corresponding to the size of the CU is halved whenever the depth increases by 1.

Referring to FIG. 3 , the size of an LCU having a minimum depth of 0 may be 64×64 pixels, and a size of an SCU having a maximum depth of 3 may be 8×8 pixels. In this case, the depth of the CU (LCU) of 64x64 pixels is 0, the depth of the CU of 32x32 pixels is 1, the depth of the CU of 16x16 pixels is 2, and the depth of the CU (SCU) of 8x8 pixels can be expressed as 3.

In addition, information on whether to split a specific CU may be expressed through 1-bit split information for each CU. This partition information may be included in all CUs except for the SCU. For example, when a CU is not split, 0 may be stored in the partition information, and 1 may be stored in the partition information when a CU is split.

4 illustrates a form of a prediction unit (PU) that the encoding unit (CU) may include.

A CU that is no longer split among CUs split from an LCU may be split or partitioned into one or more prediction units.

A prediction unit (hereinafter, referred to as a PU) is a basic unit for performing prediction, and is encoded and decoded in any one of a skip mode, an inter mode, and an intra mode, and is formed in various forms according to each mode. can be partitioned.

Referring to FIG. 4 , in the case of the skip mode, the 2Nx2N mode 410 having the same size as the CU may be supported without a partition of the CU.

In the case of inter mode, there are 8 partitioned types for CU, for example, 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435 ), nLx2N mode 440 , and nRx2N mode 445 may be supported.

In the case of the intra mode, the 2Nx2N mode 410 and the NxN mode 425 may be supported for the CU.

5 is an example illustrating a structure of inter view prediction in a 3D video codec.

For view 1 (View 1) and view 2 (View 2), inter-view prediction can be performed using view 0 (View 0) as a reference image, and the encoding sequence is view 1 (View 1) and view 2 (View 2). ) should be encoded before view 0 (View 0).

In this case, since View 0 can be independently coded regardless of other views, it is called an independent view. On the other hand, since View 1 and View 2 are encoded using View 0 as a reference image, they are called Dependent Views. An independent view image may be encoded using a general 2D video codec. On the other hand, since the dependent view image needs to perform inter-view prediction, it may be encoded using a 3D video codec including an inter-view prediction process.

Also, in order to increase the encoding efficiency of view 1 and view 2, view 1 and view 2 may be encoded using a depth information map. For example, when encoding a real image and a depth information map thereof, the real image and the depth information map may be encoded and/or decoded independently of each other. Alternatively, when encoding the real image and the depth information map, the real image and the depth information map may be encoded and/or decoded depending on each other as shown in FIG. 6 .

6 is an example illustrating a process of encoding and/or decoding an actual image (texture view) and a depth information map (depth view) in a 3D video encoder and/or decoder.

Referring to FIG. 6 , the 3D video encoder may include a real image encoder for encoding a texture view and a depth information map encoder for encoding a depth information map. have.

In this case, the real image encoder may encode the real image using the depth information map encoded by the depth information map encoder. Conversely, the depth information map encoder may encode the depth information map using the real image encoded by the real image encoder.

The 3D video decoder may include a texture decoder for decoding an actual image and a depth information map decoder for decoding a depth information map.

In this case, the real image decoder may decode the real image using the depth information map decoded by the depth information map decoder. Conversely, the depth information map decoder may decode the depth information map using the real image decoded by the real image decoder.

7 is a block diagram illustrating a configuration of a video encoder according to an embodiment.

7 shows an embodiment of a video encoder applicable to a multi-view structure, and the video encoder for a multi-view structure may be implemented by extending a video encoder for a single-view structure. In this case, the video encoder of FIG. 7 may be used in the actual image encoder and/or the depth information map encoder of FIG. 6 , and the encoder may mean an encoding device.

Referring to FIG. 7 , the video encoder 700 includes an inter prediction unit 710 , an intra prediction unit 720 , a switch 715 , a subtractor 725 , a transform unit 730 , a quantization unit 740 , and entropy encoding. It includes a unit 750 , an inverse quantizer 760 , an inverse transform unit 770 , an adder 775 , a filter unit 780 , and a reference picture buffer 790 .

The video encoder 700 may perform encoding on an input image in an intra mode or an inter mode and output a bitstream.

Intra prediction refers to intra picture prediction, and inter prediction refers to inter picture prediction or inter-view prediction. In the intra mode, the switch 715 is switched to the intra mode, and in the inter mode, the switch 715 is switched to the inter mode.

After generating a prediction block for a block (current block) of the input picture, the video encoder 700 may encode a difference between the current block and the prediction block.

In the intra mode, the intra prediction unit 720 may use a pixel value of an already encoded block around the current block as a reference pixel. The intra prediction unit 720 may generate prediction samples for the current block by using the reference pixel.

In the inter mode, the inter prediction unit 710 may obtain a motion vector specifying a reference block corresponding to the input block (current block) from the reference picture stored in the reference picture buffer 790 . The inter prediction unit 710 may generate a prediction block for the current block by performing motion compensation using the motion vector and the reference picture stored in the reference picture buffer 790 .

In the multi-view structure, inter prediction applied in the inter mode may include inter-view prediction. The inter prediction unit 710 may construct an inter-view reference picture by sampling the picture of the reference view. The inter-prediction unit 710 may perform inter-view prediction by using a reference picture list including an inter-view reference picture. A reference relationship between views may be signaled through information specifying a dependency between views.

Meanwhile, when the picture of the current view and the picture of the reference view have the same size, sampling applied to the reference view picture may mean copying a sample from the reference view picture or generating a reference sample by interpolation. When the resolutions of the current view picture and the reference view picture are different, sampling applied to the reference view picture may mean upsampling or downsampling. For example, when resolutions between views are different, the inter-view reference picture may be configured by upsampling the reconstructed picture of the reference view.

Whether the picture of which view is used to construct the inter-view reference picture may be determined in consideration of coding cost and the like. The encoder may transmit information specifying a view to which a picture to be used as an inter-view reference picture belongs to the decoding apparatus.

Also, a view referenced in inter-view prediction, that is, a picture used for prediction of a current block in the reference view may be a picture of the same access unit (AU) as the current picture (prediction target picture in the current view).

The subtractor 725 may generate a residual block (residual signal) by the difference between the current block and the prediction block.

The transform unit 730 may perform transform on the residual block to output a transform coefficient. When a transform skip mode is applied, the transform unit 730 may skip transform for the residual block.

The quantization unit 740 may quantize the transform coefficient according to the quantization parameter and output a quantized coefficient.

The entropy encoder 750 may entropy-encode the values calculated by the quantizer 740 or the encoding parameter values calculated during the encoding process according to a probability distribution to output a bitstream. The entropy encoder 750 may entropy-encode information for video decoding (eg, a syntax element, etc.) in addition to pixel information of the video.

The encoding parameter is information necessary for encoding and decoding, and may include information that is encoded in an encoder and transmitted to a decoding apparatus, such as a syntax element, as well as information that can be inferred during encoding or decoding.

The residual signal may mean a difference between an original signal and a predicted signal, and a signal in which the difference between the original signal and the predicted signal is transformed or a signal in which the difference between the original signal and the predicted signal is transformed and quantized. may mean In block units, the residual signal may be referred to as a residual block.

When entropy encoding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence and a large number of bits are allocated to a symbol having a low probability of occurrence to express the symbol, so that the size of the bit string for the symbols to be encoded is increased. can be reduced. Accordingly, compression performance of image encoding may be improved through entropy encoding.

For entropy encoding, an encoding method such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoder 750 may perform entropy encoding using a variable length coding (VLC) table. In addition, the entropy encoder 750 derives a binarization method of a target symbol and a probability model of a target symbol/bin, and then performs entropy encoding using the derived binarization method or probability model. You may.

The quantized coefficient may be inverse quantized by the inverse quantizer 760 and inversely transformed by the inverse transform unit 770 . The inverse-quantized and inverse-transformed coefficients may be added to the prediction block through an adder 775 and a reconstructed block may be generated.

The reconstructed block passes through the filter unit 780, and the filter unit 780 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed picture. can do. The reconstructed block passing through the filter unit 780 may be stored in the reference image buffer 790 .

8 is a block diagram illustrating a configuration of a video decoder according to an embodiment.

8 shows an embodiment of a video decoder applicable to a multi-view structure, and the video decoder for a multi-view structure may be implemented by extending a video decoder for a single-view structure.

In this case, the video decoder of FIG. 8 may be used in the actual image decoder and/or the depth information map decoder of FIG. 6 . For convenience of explanation, in this specification, 'decoding' and 'decoding' may be used interchangeably, or 'decoding device' and 'decoder' may be used interchangeably.

Referring to FIG. 8 , the video decoder 800 includes an entropy decoding unit 810, an inverse quantization unit 820, an inverse transform unit 830, an intra prediction unit 840, an inter prediction unit 850, and a filter unit ( 860) and a reference picture buffer 870.

The video decoder 800 may receive a bitstream output from the encoder, perform decoding in an intra mode or an inter mode, and output a reconstructed image, that is, a reconstructed image.

In the intra mode, the switch may be switched for intra prediction, and in the inter mode, the switch may be switched for inter prediction.

The video decoder 800 may generate a reconstructed block, that is, a reconstructed block by obtaining a reconstructed residual block from the received bitstream, generating a prediction block, and adding the reconstructed residual block and the prediction block. .

The entropy decoding unit 810 may entropy-decode the input bitstream according to a probability distribution and output information such as a quantized coefficient and a syntax element.

The quantized coefficient is inverse quantized by the inverse quantization unit 820 and inversely transformed by the inverse transform unit 830 . A reconstructed residual block may be generated by inverse quantizing/inverse transforming the quantized coefficient.

In the intra mode, the intra prediction unit 840 may generate a prediction block for the current block by using pixel values of an already encoded block around the current block.

In the inter mode, the inter prediction unit 850 may generate a prediction block for the current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 870 .

In the case of a multi-view structure, inter prediction applied in the inter mode may include inter-view prediction. The inter prediction unit 850 may construct an inter-view reference picture by sampling the picture of the reference view. The inter prediction unit 850 may perform inter-view prediction by using a reference picture list including inter-view reference pictures. A reference relationship between views may be signaled through information specifying a dependency between views.

Meanwhile, when the current view picture (the current picture) and the reference view picture have the same size, sampling applied to the reference view picture may mean copying a sample from the reference view picture or generating a reference sample by interpolation. When the resolutions of the current view picture and the reference view picture are different, sampling applied to the reference view picture may mean upsampling or downsampling.

For example, if inter-view prediction is applied between views when resolutions between views are different, an inter-view reference picture may be configured by upsampling the reconstructed picture of the reference view.

In this case, information specifying a view to which a picture to be used as an inter-view reference picture belongs may be transmitted from the encoder to the decoder.

Also, a view referenced in inter-view prediction, that is, a picture used for prediction of a current block in the reference view may be a picture of the same access unit (AU) as the current picture (prediction target picture in the current view).

The reconstructed residual block and the prediction block are added by an adder 855 to generate a reconstructed block. In other words, a reconstructed sample or a reconstructed picture is generated by adding the residual sample and the prediction sample.

The reconstructed picture is filtered by the filter unit 860 . The filter unit 860 may apply at least one of a deblocking filter, SAO, and ALF to a reconstructed block or a reconstructed picture. The filter unit 860 outputs a reconstructed picture that has been reconstructed or filtered. The reconstructed image may be stored in the reference picture buffer 870 and used for inter prediction.

7 and 8, each module has been described as performing different functions, but the present invention is not limited thereto, and one module may perform two or more functions. For example, the operations of the intra prediction unit and the inter prediction unit in FIGS. 7 and 8 may be performed in one module (prediction unit).

Meanwhile, in FIGS. 7 and 8 , it has been described that one encoder/decoder processes both encoding/decoding for multi-views, but this is for convenience of description, and the encoder/decoder may be configured for each view.

In this case, the encoder/decoder of the current view may perform encoding/decoding of the current view using information of the other view. For example, the prediction unit (inter prediction unit) of the current view may perform intra prediction or inter prediction on the current block using pixel information or reconstructed picture information of another view.

Here, only prediction between views has been described as an example, but regardless of whether the encoder/decoder is configured for each view or one device processes multi-views, encoding/decoding of the current layer can be performed using information from other views. have.

In the present invention, the description of the view may be equally applied to a layer supporting scalability. For example, in the present invention, a view may be a layer.

9 is a diagram illustrating an example of a prediction structure for a 3D video codec. For convenience of explanation, FIG. 9 shows an actual image acquired from three cameras and a prediction structure for encoding a depth information map corresponding to the real image.

In FIG. 9 , three real images acquired from three cameras are indicated as T0, T1, and T2 according to a view, and three depth information maps corresponding to the real image are D0 and D1 according to a view, respectively. , denoted by D2. Here, T0 and D0 are images obtained at view 0 (View 0), T1 and D1 are images obtained at view 1 (View 1), and T2 and D2 are images obtained at view 2 (View 2). In this case, the rectangle shown in FIG. 9 represents an image (picture).

Each picture (picture) is divided into an I picture (Intra picture), P picture (Uni-prediction picture), and B picture (Bi-prediction picture) according to the encoding/decoding type, and each picture is an encoding/decoding type of each picture. can be encoded/decoded according to In the I picture, the video itself is encoded without inter prediction, in the P picture, inter prediction is performed using only reference pictures existing in one direction, and in the B picture, inter prediction can be performed using the reference pictures existing in both directions. In this case, an arrow in FIG. 9 indicates a prediction direction. That is, the actual image and its depth information map may be encoded/decoded depending on each other depending on the prediction direction.

In order to perform encoding/decoding of an image through inter prediction, motion information of the current block is required. As a method of inferring the motion information of the current block, there are a method of using the motion information of the current block and an adjacent block, a method using a temporal correlation within the same viewpoint, or a method using a correlation between viewpoints in an adjacent viewpoint. The methods may be mixed and used in one picture. Here, the current block refers to a block on which prediction is performed. The motion information may mean a motion vector, a reference picture number, and/or a prediction direction (eg, whether unidirectional prediction or bidirectional prediction, temporal correlation, or inter-view correlation, etc.).

In this case, the prediction direction may be largely divided into unidirectional prediction and bidirectional prediction according to the use of a reference picture list (RefPicList). One-way prediction is divided into forward prediction (Pred_L0: Prediction L0) using a forward reference picture list (LIST 0, L0) and backward prediction (Pred_L1: Prediction L1) using a backward reference picture list (LIST 1, L1). do. In addition, the bidirectional prediction (Pred_BI: Prediction BI) uses both the forward reference picture list (LIST 0) and the backward reference picture list (LIST 1), and it can be said that both forward prediction and backward prediction exist, and the forward reference picture A case in which two forward predictions exist by copying the list (LIST 0) to the backward reference picture list (LIST 1) may also be included in the bidirectional prediction.

Whether the prediction direction is determined can be defined using predFlagL0 and predFlagL1. In this case, predFlagL0 is an indicator indicating whether to use the forward reference picture list (List 0), and predFlagL1 corresponds to an indicator indicating whether to use the backward reference picture list (List 1). For example, in case of unidirectional prediction and forward prediction, predFlagL0 may be '1' and predFlagL1 may be '0'. In case of unidirectional prediction and backward prediction, predFlagL0 may be '0' and predFlagL1 may be '1'. In the case of bidirectional prediction, predFlagL0 may be '1' and predFlagL1 may be '1'.

10 is an example illustrating neighboring blocks used to construct a merge candidate list for a current block.

A merge mode is one of the methods for performing inter prediction, and in the merge mode, as motion information (eg, at least one of a motion vector, a reference picture list, and a reference picture index) of the current block, a neighboring block of the current block motion information can be used. In this case, using the motion information of the neighboring block as the motion information of the current block is called merging, motion merging, or merging motion.

In the merge mode, merge motion in units of a coding unit (CU, hereinafter referred to as 'CU') and merge motion in units of a prediction unit (PU, hereinafter referred to as 'PU') are possible.

When the merge motion is performed in units of blocks (eg, CUs or PUs) (hereinafter, referred to as 'blocks' for convenience of description), information on whether to perform the merge motion for each block partition and the current block Information on which block to perform a merge motion with among neighboring blocks adjacent to .

In order to perform the merge motion, a merging candidate list may be constructed.

The merge candidate list indicates a list of motion information and may be generated before the merge mode is performed. Here, the motion information of the merge candidate list may be motion information of a neighboring block adjacent to the current block or new motion information created by combining motion information already present in the merge candidate list. The motion information (eg, motion vector and/or reference picture index) of the neighboring block may be motion information specified by the neighboring block or stored in the neighboring block (used for decoding of the neighboring block).

At this time, as shown in FIG. 10 , the neighboring blocks are spatially adjacent to the current block (neighboring blocks) (A, B, C, D, E), and the current block and temporal (temporal) It may include a co-located block (H or M) corresponding to the co-located block. The co-located candidate block refers to a block at a corresponding position within a co-located picture that temporally corresponds to a current picture including the current block. If the H block in the co-located picture is available, the H block may be determined as the co-located candidate block, and if the H block is not available, the M block in the co-located picture may be determined as the co-located candidate block.

When constructing the merge candidate list, motion information of neighboring blocks (A, B, C, D, E) and candidate blocks (H or M) at the same location are merge candidates constituting the merge candidate list of the current block. It is determined whether it can be used as That is, motion information of a block available for inter prediction of the current block may be added to the merge candidate list as a merge candidate.

For example, as a method of constructing a merge candidate list for block X, 1) First, when the neighboring block A is available, the neighboring block A is included in the merge candidate list. 2) Then, only when the motion information of the neighboring block B is not the same as the motion information of the neighboring block A, the neighboring block B is included in the merge candidate list. 3) In the same way, the neighboring block C is included in the merge candidate list only when the motion information of the neighboring block C is different from the motion information of the neighboring block B, and 4) the motion information of the neighboring block D is combined with the motion information of the neighboring block C Only when are different, the neighboring block D is included in the merge candidate list. In addition, 5) the neighboring block E is included in the merge candidate list only when the motion information of the neighboring block E is different from the motion information of the neighboring block D, 6) Finally, the neighboring block H (or M) is included in the merge candidate list make it That is, each neighboring block may be added to the merge candidate list in the order of A→B→C→D→E→H (or M) blocks. Here, the same motion information may mean using the same motion vector, the same reference picture, and the same prediction direction (unidirectional or bidirectional).

Here, the expression of adding the neighboring block to the merge candidate list as a merge candidate and the expression of adding the motion information of the neighboring block to the merge candidate list as a merge candidate are mixed, but this is for convenience of explanation, and the two expressions are substantially different don't For example, a neighboring block as a merge candidate may mean motion information of the corresponding block.

11 is a flowchart illustrating a method of constructing a merge motion candidate list in inter prediction.

Referring to FIG. 11 , the inter prediction unit derives a merge motion candidate, that is, a merge candidate ( S1100 ). In order to efficiently encode the motion information of the video encoding/decoding object block as described above, surrounding motion information may be used for the merge candidate. In 3D video encoding/decoding, surrounding motion information includes motion parameter inheritance (MPI), inter-view merging candidate (IvMC), A1, B1, B0, and inter-view disparity vector candidate ( inter-view disparity vector candidate (IvDC), view synthesis prediction (VSP), A0, B2, shift inter-view (ShiftIV), Col, Bi, and Zero candidates.

In this case, MPI means motion information inheriting the above-described motion information of an actual image, IvMC means motion information using inter-view merge, and A1 is motion information of a block located to the left of the prediction object block as described above. means In addition, B1 means motion information of a block located above the prediction object block as described above, B0 means motion information of a block located on the right side of the prediction object block as described above, and IvDC is the above-described variation It means motion information derived by using a vector (ie, using a parallax). In addition, VSP means motion information derived by synthesizing viewpoints, A0 means motion information of a block located at the lower left side of the prediction object block as described above, and B2 means the upper left side of the prediction object block as described above. It means motion information of a block located in , and ShiftIV means motion information derived using the corrected parallax. Col is motion information derived through a block at a corresponding position in a co-located picture temporally corresponding to the current picture including the current block as described above, and Bi is motion induced using both directions. It means a candidate, and Zero means a zero vector.

Thereafter, the inter prediction unit inserts the derived merge candidates into the merge candidate list ( S1110 ). In the inter prediction unit, in order to increase entropy encoding efficiency, merge candidates are added to the list in the order in which they are statistically most selected from among the derived merge motion candidates. That is, the merge candidate with the most statistical selection is inserted at the first position of the merge candidate list, and the merge candidate with the second highest statistical selection is inserted at the second position of the merge candidate list, and even after that, the inter prediction unit In , based on statistical information, a process of sequentially adding merge candidates that are statistically frequently selected to the merge candidate list is performed.

For example, in 3D video encoding/decoding, in the inter prediction unit, the merge candidates are MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, and Zero candidates in the order of the merge candidates. It can be inserted into a list.

<Table 1>

Table 1 is an example of a rate at which merge candidates are selected from textures. The candidates in Table 1 correspond to the above-described MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, and Zero candidates, respectively, and Table 1 shows MPI, IvMC, A1, B1, B0 , IvDC, VSP, A0, B2, ShiftIV, Col, Bi, Zero candidates are selected, respectively, and the ratio of MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi during entropy encoding. , the average value of the Zero candidates is shown. At this time, since the MPI candidate is not used in the texture, IvMC occupies the 0th index, and IvMC is selected with the highest ratio.

For example, as shown in Table 1, when the probability that IvMC is selected is 78%, when entropy encoding/decoding is performed by setting IvMC to the 0th index when encoding/decoding a 3D image, the fewest bits are added to the most-selected merge candidates. Therefore, the encoding/decoding efficiency is improved. After that, entropy encoding is generally performed in the order of candidates A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, and Zero. As a result, encoding/decoding efficiency is improved.

When constructing the merge candidate list in image encoding/decoding, various types of candidates are used as described above, and the contents of the various types of candidates are as described above.

In this case, the merge candidate selected by the current block is similar to the merge candidate selected from the neighboring block. That is, the motion information finally selected from the current block is highly likely to be similar to the motion candidate selected from the neighboring block, and the neighboring block refers to a spatial neighboring block and neighboring blocks generated in 3D image encoding/decoding. can The spatial neighboring blocks may mean A0, A1, B0, B1, B2, Col, and the like, and the specific contents of the spatial neighboring blocks are as described above. The neighboring blocks generated in 3D image encoding/decoding may mean MPI, IvMC, IvDC, VSP, ShiftIV, Col, Bi, and Zero candidates. Details of neighboring blocks generated in 3D image encoding/decoding are as described above same.

Hereinafter, the present invention proposes an apparatus and method for setting a merge candidate of a neighboring block as a merge candidate of a current block when a neighboring block uses a specific candidate. That is, the present invention proposes a method in which a 3D image encoding/decoding apparatus uses a motion candidate used in a neighboring block as a motion candidate of the current block when encoding/decoding the current block.

12 is a flowchart illustrating a method of using a motion candidate used in a neighboring block in a current block when encoding a 3D image according to an embodiment of the present invention.

Referring to FIG. 12 , the 3D image encoding apparatus determines whether a motion candidate used in a block located near the current block can be used as a motion candidate of the current block ( S1210 ). In this case, the block located near the current block may mean a spatially adjacent block of the current block as described above, or may mean a neighboring block generated in 3D image encoding as described above.

When a motion candidate used in a block located near the current block can be used as a motion candidate of the current block, the 3D image encoding apparatus sets the motion candidate used in the neighboring block as the motion candidate of the current block (S1230). Here, the motion candidate used in the neighboring block may mean motion information used in the neighboring block, and the motion candidate of the current block may refer to the motion information of the current block. In this case, the motion information may include a motion vector, reference information (reference direction, reference picture, reference block, etc.), and the like, and the motion vector may include direction information specifying the position of a referenced block in inter prediction.

In this case, the 3D image encoding apparatus may not encode the index of a candidate to be used as motion information in the merge candidate list of the current block when encoding/decoding the current block.

If the motion candidate used in the neighboring block cannot be used in the current block, the 3D image encoding apparatus generates a motion candidate index and transmits it to the 3D encoding/decoding apparatus (S1220). In this case, the motion candidate index may mean information about merge candidates inserted into the merge candidate list.

13 is a flowchart of a method in which a motion candidate used in a neighboring block is used in a current block when encoding a 3D image is applied to 3D-HEVC according to another embodiment of the present invention.

Referring to FIG. 13 , the 3D image encoding apparatus does not encode a merge candidate if any one of the following conditions (1, 2) is satisfied.

1. When the current picture is not a depth map, and a co-located block in the same previous picture as the current view uses an inter-view merging candidate (IVMC), and an inter-view motion candidate can be used in the current block.

2. When the current picture is a depth map, and the co-located block in the texture corresponding to the current picture uses a Motion Parameter Inheritance (MPI) candidate, and a motion inheritance candidate is available in the current block.

According to the above-described conditions, more specific details of a method of applying a motion candidate used in a neighboring block to a current block to 3D-HEVC when encoding a 3D image are as follows.

The 3D encoding apparatus determines whether the layerID is not 0 (S1310).

If the layerID is not 0, the 3D encoding apparatus determines VpsDepthFlag[layerID] (S1320).

When VpsDepthFlag[layerID] exists, the 3D encoding apparatus determines whether mpi_flag[nuh_layer_id] exists, colMergType satisfies T, and T is usable (S1330).

If mpi_flag[nuh_layer_id] exists, colMergType satisfies T, and T is available, the 3D encoding apparatus sets the merge index to 0 ( S1340 ).

In step S1320, if VpsDepthFlag[layerID] does not exist, the 3D video encoding apparatus determines whether iv_mv_pred_flag[nuh_layer_id] exists, colMergType is IvMC, and IvMC is available (S1350).

If 1) layerID is not 0 in step S1310, that is, when layerID is 0, 2) in step S1330, mpi_flag[nuh_layer_id] does not exist, colMergType does not satisfy T, or T is not available. In case 3) in step S1350, when iv_mv_pred_flag[nuh_layer_id] does not exist, colMergType is not IvMC, or IvMC is not available, the 3D encoding apparatus decodes the merge index (S1360).

14 is a block diagram of an apparatus for using, in a current block, a motion candidate used in a neighboring block when encoding a 3D image according to another embodiment of the present invention.

Referring to FIG. 14 , the inter prediction unit 1400 includes a determination unit 1410 , a setting unit 1420 , and an index generation unit 1430 . Here, the inter prediction unit may be included in the above-described image encoding apparatus.

The determiner 1410 determines whether a motion candidate used in a block located near the current block can be used as a motion candidate of the current block. In this case, the detailed content of determining whether the motion candidate used in the block located in the vicinity of the current block in the determination unit 1410 can be used as the motion candidate of the current block is as described above.

When a motion candidate used in a block located in the vicinity of the current block can be used as a motion candidate of the current block, the setting unit 1420 sets the motion candidate used in the neighboring block as a motion candidate of the current block. can be set. In this case, when the motion candidate used in the block located in the vicinity of the current block in the setting unit 1420 can be used as the motion candidate of the current block, the 3D image encoding apparatus uses the motion candidate used in the neighboring block as the motion of the current block. Specific details of setting the candidate are the same as described above.

When the motion candidate used in the neighboring block cannot be used in the current block, the index generator 1430 may generate a motion candidate index and transmit it to the 3D encoding/decoding apparatus. In this case, when the index generator cannot use the motion candidate used in the neighboring block in the current block, the detailed content of generating the motion candidate index and transmitting it to the 3D encoding/decoding apparatus is as described above.

15 is a flowchart illustrating a method of using a motion candidate used in a neighboring block in a current block when decoding a 3D image according to another embodiment of the present invention.

Referring to FIG. 15 , the 3D image decoding apparatus determines whether a motion candidate used in a block located near the current block can be used as a motion candidate of the current block ( S1510 ). In this case, a block located near the current block may mean a spatially adjacent block of the current block as described above, or may mean a neighboring block generated in 3D image encoding as described above.

When a motion candidate used in a block located near the current block can be used as a motion candidate of the current block, the 3D image decoding apparatus sets the motion candidate used in the neighboring block as the motion candidate of the current block (S1520). Here, the motion candidate used in the neighboring block may mean motion information used in the neighboring block, and the motion candidate of the current block may refer to the motion information of the current block. In addition, the detailed content of the motion candidate is as described above.

When the 3D image decoding apparatus encodes/decodes the current block, it may not decode the index of a candidate to be used as motion information in the merge candidate list of the current block.

If the motion candidate used in the neighboring block cannot be used in the current block, the 3D image decoding apparatus receives the motion candidate index (S1530). In this case, the motion candidate index may mean information about merge candidates inserted into the merge candidate list.

16 is a block diagram of an apparatus for using a motion candidate used in a neighboring block in a current block when decoding a 3D image according to another embodiment of the present invention.

Referring to FIG. 16 , the inter prediction unit 1600 may include a determining unit 1610 , an index receiving unit 1620 , and a setting unit 1630 . In this case, the inter prediction unit may be included in the above-described image encoding apparatus and/or decoding apparatus.

The determiner 1610 determines whether a motion candidate used in a block located near the current block can be used as a motion candidate of the current block. In this case, details of determining whether a motion candidate used in a block located near the current block in the determination unit 1610 can be used as a motion candidate of the current block is as described above.

The index receiver 1620 receives a motion candidate index. The motion candidate index is used when generating a merge candidate in the current block when the motion candidate used in the neighboring block cannot be used in the current block, and the detailed contents of the motion candidate index are as described above.

When a motion candidate used in a block located near the current block can be used as a motion candidate of the current block, the setting unit 1630 sets the motion candidate used in the neighboring block as a motion candidate of the current block. can be set. In this case, when the motion candidate used in the block located in the vicinity of the current block in the setting unit 1630 can be used as the motion candidate of the current block, the motion candidate used in the neighboring block in the setting unit 1630 is used in the motion of the current block. Specific details of setting the candidate are the same as described above.

In the above-described embodiments, the methods are described on the basis of a flowchart as a series of steps or units, but the present invention is not limited to the order of steps, and some steps may occur in a different order or at the same time as other steps as described above. can In addition, those of ordinary skill in the art will recognize that the steps shown in the flowchart are not exclusive, other steps may be included, or that one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You will understand.

The above-described embodiments include examples of various aspects. It is not possible to describe every possible combination for representing the various aspects, but one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, the present invention is intended to cover all other substitutions, modifications and variations falling within the scope of the following claims.

The above-described method according to the present invention may be produced as a program to be executed by a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape. , a floppy disk, an optical data storage device, and the like, and also includes those implemented in the form of a carrier wave (eg, transmission through the Internet).

The computer-readable recording medium is distributed in a network-connected computer system, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the method can be easily inferred by programmers in the art to which the present invention pertains.

In addition, although preferred embodiments of the present invention have been shown and described, the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (10)

determining whether a merge candidate used in a neighboring block is also usable in a current block;
setting the merge candidate used in the neighboring block as a merge candidate of the current block when the merge candidate used in the neighboring block is also usable in the current block; and
When a merge candidate used in a neighboring block is not available in the current block, receiving a motion candidate index indicating a merge candidate generated in the current block from an image decoding apparatus,
The neighboring blocks are spatial neighboring blocks of the current block and neighboring blocks generated in a 3D image encoding/decoding process, wherein the spatial neighboring blocks include blocks A0, A1, B0, B1 and B2, and the 3D image encoding/decoding process The neighboring blocks generated in the decoding process include MPI, IvMC, IvDC, VSP, ShiftIV, Col, Bi, and Zero code blocks,
The motion candidate index includes information about a merge candidate inserted into a merge candidate list,
video decoding method. delete delete The method according to claim 1,
If the merge candidate used in the neighboring block is also available in the current block,
The current picture is not a depth map, the same in the previous picture as the current view
An image decoding method comprising a case where a location block uses an inter-view motion candidate (IVMC) and an inter-view motion candidate can be used in the current block. The method according to claim 1,
If the merge candidate used in the neighboring block is also available in the current block,
A video decoding method comprising a case where the current picture is a depth map, a co-located block in a texture corresponding to the current picture uses a motion inheritance candidate, and a motion inheritance candidate is available in the current block. processor; and
a memory for storing at least one instruction executed by the processor;
The at least one command is
an instruction for determining whether a merge candidate used in a neighboring block is also usable in the current block;
a command for setting the merge candidate used in the neighboring block as a merge candidate of the current block when the merge candidate used in the neighboring block is also usable in the current block; and
and a command to receive a motion candidate index indicating a merge candidate generated in the current block from an image decoding apparatus when the merge candidate used in the neighboring block is not available in the current block,
The neighboring blocks are spatial neighboring blocks of the current block and neighboring blocks generated in a 3D image encoding/decoding process, wherein the spatial neighboring blocks include blocks A0, A1, B0, B1 and B2, and the 3D image encoding/decoding process The neighboring blocks generated in the decoding process include MPI, IvMC, IvDC, VSP, ShiftIV, Col, Bi, and Zero code blocks,
The motion candidate index includes information about a merge candidate inserted into a merge candidate list,
video decoding device. delete delete 7. The method of claim 6,
If the merge candidate used in the neighboring block is also available in the current block,
The current picture is not a depth map, the same in the previous picture as the current view
An apparatus for decoding an image, comprising a case where a location block uses an inter-view motion candidate (IVMC) and an inter-view motion candidate is available in the current block. 7. The method of claim 6,
If the merge candidate used in the neighboring block is also available in the current block,
A video decoding apparatus comprising a case in which the current picture is a depth map, a co-located block in a texture corresponding to the current picture uses a motion inheritance candidate, and a motion inheritance candidate is available in the current block.

Images