KR20150102748A - A Method And Apparatus For Deriving Temporal Inter-View Motion Information Of Sub-Prediction Unit - Google Patents

Abstract

The present invention provides a method for encoding a 3D image which includes the steps of: deriving a merge candidate; determining whether the change of the priority of the merge candidate is necessary; changing the priority of the merge candidate if the change of the priority is necessary; and adding the merge candidate on a merge candidate list based on the changed priority of the merge candidate.

Description

[0001] The present invention relates to a method and an apparatus for deriving motion information between temporal viewpoints in units of sub-prediction units,

More particularly, the present invention relates to a coding / decoding method for rearranging merged motion candidate lists in inter-view prediction and an apparatus using the same.

The continued development of the information and communications industry has led to the worldwide spread of high definition (HD) resolution broadcast services, which has led many users to become accustomed to HD video.

Users who are accustomed to HD picture quality want images with higher image quality and higher resolution, and many organizations have spurred development on next generation imaging equipment to meet the demand of users. As a result, it is now possible to access images that support FHD (Full HD) and UHD (Ultra High Definition).

Users want 3-D images that can feel stereoscopic images as well as images with high image quality and high resolution. Various organizations have developed 3D images to meet the needs of users.

However, since the 3D image includes not only the texture information but also the depth map information, the 3D image includes more data than the 2D image. Therefore, there is a problem in that sufficient coding / decoding efficiency is not obtained when the three-dimensional image is encoded / decoded according to the conventional image encoding / decoding process.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an efficient generation method of a merge candidate list and an apparatus using the same.

Another object of the present invention is to provide a method for changing a priority of a merge candidate when inserting a remainder candidate into a merge candidate list and a device using the merge candidate.

Another object of the present invention is to provide a method for increasing the probability that an index 0 is selected as an optimal candidate by changing a priority of a merge candidate when inserting a remainder candidate into a merge candidate list, .

Another object of the present invention is to improve the efficiency of entropy coding by changing the priority of the merge candidate and increasing the probability that the index 0 of the merge candidate list is selected as the best candidate when inserting the merge candidate into the merge candidate list And a device using the same.

According to an embodiment of the present invention, there is provided a method for managing a merge candidate, the method comprising: deriving a merge candidate; determining whether a change in priority of the merge candidate is necessary; And adding the merge candidate to the merge candidate list based on the priority of the modified merge candidate.

According to another embodiment of the present invention, there is provided a method for managing a merge candidate, the method comprising: deriving a merge candidate; determining whether a change in priority of the merge candidate is necessary; And adding the merge candidate to the merge candidate list based on the priority of the changed merge candidate.

According to another embodiment of the present invention, when changing a priority of a merge candidate and an inducing unit for deriving a merge candidate, the priority of the merge candidate is changed, and the merge candidate is changed to the priority of the changed merge candidate And a generation unit for adding the generated candidate candidate list to the merge candidate list.

According to another embodiment of the present invention, when changing a priority of a merge candidate and an inducing unit for deriving a merge candidate, the priority of the merge candidate is changed, and the merge candidate is changed to the priority of the changed merge candidate Based on the merged candidate list, and adding the merged candidate list to the merge candidate list.

According to the present invention, it is possible to efficiently generate a merge candidate list.

According to the present invention, when the merge candidate is inserted into the merge candidate list, the priority order of merge candidate can be changed.

According to the present invention, when inserting a merge candidate into a merge candidate list, the priority of the merge candidate can be changed to increase the probability that the zero index of the merge candidate list is selected as the best candidate.

According to the present invention, when inserting a merge candidate into a merge candidate list, the efficiency of entropy encoding can be increased by changing the priority of the merge candidate to increase the probability that the 0 index of the merge candidate list is selected as the best candidate .

1 schematically shows a basic structure of a three-dimensional video system.
2 is a diagram showing an example of an actual image of a "balloons" image and an image of a depth information map.
3 is a diagram schematically showing a divided structure of an image when encoding and decoding an image.
Fig. 4 shows a form of a prediction unit (PU) that the encoding unit (CU) can include.
FIG. 5 shows an example of a structure of inter view prediction in a three-dimensional video codec.
FIG. 6 shows an example of a process of encoding and / or decoding a texture view and a depth view in a 3D video encoder and / or decoder.
7 is a block diagram illustrating a configuration according to an embodiment of a video encoder.
8 is a block diagram illustrating a configuration according to an embodiment of the video decoder.
9 is a diagram showing an example of a prediction structure for a three-dimensional video codec.
FIG. 10 shows an example of neighboring blocks used to construct a merge list for the current block.
11 is a flowchart of a method of constructing a merged motion candidate list in inter prediction.
12 schematically shows a hierarchical-B structure in 3D image coding / decoding.
13 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to an embodiment of the present invention.
14 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to another embodiment of the present invention.
15 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to another embodiment of the present invention.
16 is a flowchart for generating a merge candidate list by changing a priority of merge candidates according to another embodiment of the present invention.
17 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to another embodiment of the present invention.
18 is a block diagram schematically illustrating the structure of an inter-prediction unit according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . In addition, the content of " comprising " a specific configuration in the present invention does not exclude a configuration other than the configuration, and means that additional configurations can be included in the practice of the present invention or the technical scope of the present invention .

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

In addition, the components shown in the embodiments of the present invention are shown independently to represent different characteristic functions, and do not mean that the components are composed of separate hardware or software constituent units. That is, each constituent unit is included in each constituent unit for convenience of explanation, and at least two constituent units of the constituent units may be combined to form one constituent unit, or one constituent unit may be divided into a plurality of constituent units to perform a function. The integrated embodiments and separate embodiments of the components are also included within the scope of the present invention, unless they depart from the essence of the present invention.

In addition, some of the components are not essential components to perform essential functions in the present invention, but may be optional components only to improve performance. The present invention can be implemented only with components essential for realizing the essence of the present invention, except for the components used for the performance improvement, and can be implemented by only including the essential components except the optional components used for performance improvement Are also included in the scope of the present invention.

3D video provides a three-dimensional stereoscopic display device that gives the user the same stereoscopic feel as they would see and feel in the real world. In this regard, Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V), a joint standardization group of ISO / IEC's Moving Picture Experts Group (MPEG) and ITU-T VCEG (Video Coding Experts Group) Video standardization is in progress.

1 schematically shows a basic structure of a three-dimensional video system.

Referring to FIG. 1, a 3D video (3DV) system may include a sender and a receiver. The 3D video system of FIG. 1 may be a basic 3D video system considered in the 3D video standard. The 3D video standard may use a depth information map corresponding to the real video and the real video to generate a stereoscopic video As well as advanced data formats that can support the playback of autostereoscopic images and standards for related technologies.

The transmitting side can generate multi-view video content. Specifically, the transmitting side can generate video information using a stereo camera and a multi-view camera, and generate a depth map (depth map) using a depth information camera. Further, on the transmitting side, a two-dimensional image can be converted into a three-dimensional image using a converter. The transmitting side can generate the image content of N (N? 2) view (i.e., the multi-viewpoint) using the generated video information and the depth information map. At this time, the video content at time N may include video information at time N, depth-map information thereof, and camera-related additional information. The video content at the time point N can be compressed using the multi-view video encoding method in the three-dimensional video encoder, and the compressed video content (bit stream) can be transmitted to the reception-side terminal through the network.

The receiver can decode the image content received from the transmitter and provide a multi-view image. Specifically, on the receiving side, a bitstream transmitted using a multi-view video decoding method is decoded in a video decoder (for example, a three-dimensional video decoder, a stereo video decoder, a two-dimensional video decoder, etc.) . At this time, virtual view images having N or more viewpoints may be generated using the reconstructed N view image and the depth information-based Rendering (DIBR) process. The generated virtual viewpoint images are reproduced in accordance with various stereoscopic display devices (for example, an N-view display, a stereo display, a two-dimensional display, etc.) to provide stereoscopic images to the user.

2 is a diagram showing an example of an actual image of a "balloons" image and an image of a depth information map.

Figure 2 (a) shows a "balloons" image being used in the 3D standardization of MPEG, an international standardization organization. FIG. 2B shows a depth information map image for the "balloons" image shown in FIG. 2A. The depth information map image shown in (b) of FIG. 2 represents depth information on the screen in 8 bits per pixel.

The depth map is used to generate a virtual view image. The depth information map is a map of the distance between the camera and the actual object in the real world (depth information corresponding to each pixel at the same resolution as the real image) It is expressed in numbers. At this time, the depth information map can be acquired by using a depth information map camera or by using an actual general image (texture).

The depth information map obtained by using the depth information map camera mainly provides reliable depth information in a stopped object or a scene, but the depth information map camera operates only within a certain distance. At this time, the depth information map camera can use a laser or a structured light technique or a depth measurement technique based on Time-of-Flight of Light (TFL).

The depth information map may be generated using the actual texture (texture) and the disparity vector (Disparity Vector). The disparity vector is information indicating the time difference between two general images. The variation vector is obtained by comparing an arbitrary one pixel at the current time with the pixels at the other time and finding the most similar pixel, A pixel most similar to a pixel).

The actual image and its depth information map may be images obtained from multiple cameras as well as one camera. The images obtained from the plurality of cameras can be independently encoded and can be encoded / decoded using a general two-dimensional video encoding / decoding codec. Also, since images obtained from a plurality of cameras have correlation between views, images obtained from a plurality of cameras can be encoded using prediction between different viewpoints in order to increase the coding efficiency.

3 is a diagram schematically showing a divided structure of an image when encoding and decoding an image.

It is possible to perform coding and decoding for each coding unit (CU) in order to efficiently divide an image. A unit is a combination of syntax elements and blocks containing image samples. When a unit is divided, it may mean that a block corresponding to the unit is divided.

Referring to FIG. 3, the image 300 is sequentially divided into units of a maximum coding unit (LCU) (LCU), and a divided structure is determined for each LCU. In this specification, an LCU can be used in the same sense as a coding tree unit (CTU). The division structure means a distribution of a coding unit (hereinafter referred to as a CU) for efficiently encoding an image in the LCU 310. This distribution is a distribution of four CUs reduced to half of the horizontal size and the vertical size Quot; CU " A partitioned CU can be recursively partitioned into four CUs whose halftone and vertical sizes are reduced by half for CUs partitioned in the same manner.

At this time, the division of the CU can be recursively divided to a predetermined depth. The depth information is information indicating the size of the CU, which can be stored for each CU. For example, the depth of the LCU may be zero and the depth of the Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, the LCU is a coding unit having a maximum size as described above, and the SCU (Smallest Coding Unit) is a coding unit having a minimum size.

The depth of the CU increases by one each time the LCU 310 divides into halves and halves. For example, if the size of the CU is 2Nx2N at a certain depth L, the size of the CU is still 2Nx2N when the division is not performed, and the size of the CU is NxN when the division is performed. At this time, the depth of the NxN size CU becomes the depth L + 1. That is, the size of N corresponding to the size of the CU decreases by half every time the depth increases by one.

Referring to FIG. 3, the size of the LCU having the minimum depth of 0 is 64x64 pixels, and the size of the SCU having the maximum depth of 3 may be 8x8 pixels. At this time, the depth of CU (LCU) of 64x64 pixels is 0, the depth of CU of 32x32 pixels is 1, the depth of CU of 16x16 pixels is 2, and the depth of CU (SCU) of 8x8 pixels is 3.

In addition, information on whether to divide a specific CU can be expressed through division information of 1 bit for each CU. This division information can be included in all the CUs except for the SCU. For example, when the CU is not divided, 0 can be stored in the division information, and when the CU is divided, 1 can be stored in the division information.

Fig. 4 shows a form of a prediction unit (PU) that the encoding unit (CU) can include.

A CU that is no longer split among the CUs segmented from the LCU may be partitioned or partitioned into one or more prediction units.

The prediction unit (hereinafter, referred to as PU) is a basic unit for performing prediction, and is encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. It can be partitioned.

Referring to FIG. 4, in the skip mode, the 2Nx2N mode 410 having the same size as the CU can be supported without partitioning the CU.

In the case of the inter mode, there are eight partitioned forms for the CU, such as 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435 ), nLx2N mode 440, and nRx2N mode 445, for example.

In the case of the intra mode, the 2Nx2N mode 410 and the NxN mode 425 can be supported for the CU.

FIG. 5 shows an example of a structure of inter view prediction in a three-dimensional video codec.

View 1 and View 2 can perform inter-view prediction using View 0 as a reference image and the coding order is View 1 and View 2 (View 0) must be encoded first.

At this time, View 0 can be independently encoded regardless of other viewpoints, so it is referred to as an independent view. On the other hand, View 1 and View 2 are referred to as Dependent View because they are encoded using View 0 as a reference image. The independent viewpoint image can be encoded using a general two-dimensional video codec. On the other hand, since the dependent view image needs to perform the inter-view prediction, it can be encoded using the 3D video codec including the inter-view prediction process.

Also, in order to increase the coding efficiency of View 1 and View 2, View 1 and View 2 can be encoded using a depth information map. For example, when coding the actual image and its depth information map, the actual image and depth information map can be encoded and / or decoded independently of each other. Or when the actual image and the depth information map are encoded, the actual image and the depth information map may be encoded and / or decoded depending on each other as shown in FIG.

FIG. 6 shows an example of a process of encoding and / or decoding a texture view and a depth view in a 3D video encoder and / or decoder.

Referring to FIG. 6, a 3D video encoder may include a texture encoder for encoding a texture view and a depth encoder for encoding a depth view. have.

At this time, the actual image encoder can encode the real image using the depth information map encoded by the depth information map encoder. Conversely, the depth information map encoder can encode the depth information map using the actual image encoded by the actual image encoder.

The 3D video decoder may include an actual image decoder for decoding an actual image and a depth decoder for decoding a depth information map.

At this time, the actual image decoder can decode the real image using the depth information map decoded by the depth information map decoder. In contrast, the depth information map decoder can decode the depth information map using the actual image decoded by the actual image decoder.

7 is a block diagram illustrating a configuration according to an embodiment of a video encoder.

FIG. 7 shows an embodiment of a video encoder applicable to a multi-view structure, and a video encoder for a multi-view structure may be implemented by extending a video encoder for a single view structure. At this time, the video encoder of FIG. 7 may be used in the real image encoder and / or depth information map encoder of FIG. 6, and the encoder may mean an encoding device.

7, the video encoder 700 includes an inter prediction unit 710, an intra prediction unit 720, a switch 715, a subtractor 725, a transform unit 730, a quantization unit 740, An inverse quantization unit 760, an inverse transform unit 770, an adder 775, a filter unit 780, and a reference picture buffer 790.

The video encoder 700 may encode an input image in an intra mode or an inter mode and output a bitstream.

Intra prediction refers to intra picture prediction, and inter prediction refers to inter picture prediction or inter-view prediction. In the intra mode, the switch 715 is switched to the intra mode, and in the inter mode, the switch 715 is switched to the inter mode.

The video encoder 700 may generate a prediction block for a block of the input picture (current block), and then may code the difference between the current block and the prediction block.

In the intra mode, the intraprediction unit 720 can use the pixel value of the already coded block around the current block as a reference pixel. The intra prediction unit 720 may generate prediction samples for the current block using the reference pixels.

In the inter mode, the inter-prediction unit 710 can obtain a motion vector that specifies a reference block corresponding to an input block (current block) in a reference picture stored in the reference picture buffer 790. [ The inter prediction unit 710 can generate a prediction block for a current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 790. [

In a multi-view structure, inter prediction that is applied in inter mode may include inter-view prediction. The inter prediction unit 710 may construct an inter-view reference picture by sampling a picture of the reference view. The inter prediction unit 710 can perform the inter-view prediction using the reference picture list including the inter-view reference pictures. A reference relationship between views can be signaled through information that specifies dependencies between views.

On the other hand, when the picture of the current view and the picture of the reference view are the same size, sampling applied to the reference view picture may mean generation of a reference sample by copying or interpolation from the reference view picture. The sampling applied to the reference view picture may mean upsampling or downsampling when the resolution of the current view picture and the reference view picture are different. For example, if the resolution between views is different, an inter-view reference picture may be constructed by up-sampling the reconstructed picture of the reference view.

Which picture of a view is used to construct an inter-view reference picture can be determined in consideration of coding cost and the like. The encoder may transmit to the decoding apparatus information specifying a view to which a picture to be used as an inter-view reference picture belongs.

The picture referred to in the inter-view prediction, that is, the picture used for prediction of the current block in the reference view may be a picture of the same Access Unit (AU) as the current picture (the current picture to be predicted).

The subtractor 725 can generate a residual block (residual signal) by a difference between the current block and the prediction block.

The transforming unit 730 may perform a transform on the residual block to output a transform coefficient. In the case where the transform skip mode is applied, the transforming unit 730 may omit the transform for the residual block.

The quantization unit 740 may quantize the transform coefficient according to the quantization parameter to output a quantized coefficient.

The entropy encoding unit 750 can output the bitstream by entropy encoding the values calculated by the quantization unit 740 or the encoding parameter values calculated in the encoding process according to the probability distribution. The entropy encoding unit 750 may entropy-encode information (e.g., a syntax element) for video decoding in addition to the pixel information of the video.

The encoding parameter is information necessary for encoding and decoding, and may include not only information encoded in an encoder such as a syntax element and transmitted to a decoding apparatus, but also information that can be inferred in a coding or decoding process.

The residual signal may be a difference between the original signal and the prediction signal, and may be a signal in which the difference between the original signal and the prediction signal is transformed, or a difference between the original signal and the prediction signal is converted and the quantized signal . In block units, the residual signal may be referred to as a residual block.

When entropy coding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence, and a large number of bits are allocated to a symbol having a low probability of occurrence, so that the size of a bit string for the symbols to be coded Can be reduced. Therefore, the compression performance of the image encoding can be enhanced through the entropy encoding.

For entropy encoding, an encoding method such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) may be used. For example, the entropy encoding unit 750 may perform entropy encoding using a Variable Length Coding / Code (VLC) table. Further, the entropy encoding unit 750 derives a binarization method of a target symbol and a probability model of a target symbol / bin, and then performs entropy encoding using the derived binarization method or probability model You may.

The quantized coefficients can be inversely quantized in the inverse quantization unit 760 and inversely transformed in the inverse transformation unit 770. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 775, and a reconstruction block can be generated.

The restoration block passes through a filter unit 780 and the filter unit 780 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) can do. The reconstruction block through the filter unit 780 may be stored in the reference image buffer 790.

8 is a block diagram illustrating a configuration according to an embodiment of the video decoder.

FIG. 8 shows an embodiment of a video decoder applicable to a multi-view structure, and a video decoder for a multi-view structure may be implemented by extending a video decoder for a single view structure.

At this time, the video decoder of Fig. 8 can be used in the actual picture decoder and / or depth information map decoder of Fig. For convenience of explanation, 'decryption' and 'decoding' may be used in combination or 'decryption device' and 'decoder' may be used in combination.

8, the video decoder 800 includes an entropy decoding unit 810, an inverse quantization unit 820, an inverse transform unit 830, an intra prediction unit 840, an inter prediction unit 850, 860 and a reference picture buffer 870.

The video decoder 800 receives the bit stream output from the encoder and decodes the bit stream into an intra mode or an inter mode, and outputs a reconstructed image, that is, a reconstructed image.

In the intra mode, the switch is switched for intra prediction, and in the inter mode, the switch can be switched for inter prediction.

The video decoder 800 may obtain a residual block reconstructed from the input bitstream, generate a prediction block, and add the reconstructed residual block and the prediction block to generate a reconstructed block, i.e., a reconstructed block .

The entropy decoding unit 810 can entropy-decode the input bitstream according to the probability distribution, and output information such as quantized coefficients and syntax elements.

The quantized coefficients are inversely quantized in the inverse quantization unit 820 and inversely transformed in the inverse transformation unit 830. The reconstructed residual block can be generated by inverse-quantizing / inverse-transforming the quantized coefficients.

In the intra mode, the intra prediction unit 840 can generate a prediction block for the current block using the pixel value of the already coded block around the current block.

In the inter mode, the inter-prediction unit 850 can generate a prediction block for a current block by performing motion compensation using a motion vector and a reference picture stored in the reference picture buffer 870.

In the case of a multi-view structure, inter prediction that is applied in the inter mode may include inter-view prediction. The inter prediction unit 850 may construct a picture of an inter-view reference by sampling a picture of the reference view. The inter-prediction unit 850 can perform the inter-view prediction using the reference picture list including the inter-view reference pictures. A reference relationship between views can be signaled through information that specifies dependencies between views.

On the other hand, if the current view picture (current picture) and the reference view picture are of the same size, sampling applied to the reference view picture may mean generation of a reference sample by copying or interpolation from the reference view picture. The sampling applied to the reference view picture may mean upsampling or downsampling when the resolution of the current view picture and the reference view picture are different.

For example, if inter-view prediction is applied between views when the inter-view resolution is different, an inter-view reference picture may be constructed by up-sampling the reconstructed picture of the reference view.

At this time, the information specifying the view to which the picture to be used as the inter-view reference picture belongs may be transmitted from the encoder to the decoder.

The picture referred to in the inter-view prediction, that is, the picture used for prediction of the current block in the reference view may be a picture of the same Access Unit (AU) as the current picture (the current picture to be predicted).

The restored residual block and the prediction block are added by the adder 855, and a restoration block is generated. In other words, the residual sample and the prediction sample are added together to generate a reconstructed sample or reconstructed picture.

The restored picture is filtered by the filter unit 860. The filter unit 860 may apply at least one of the deblocking filter, SAO, and ALF to the restoration block or the restored picture. The filter unit 860 outputs a reconstructed picture that has been modified or filtered. The reconstructed image is stored in the reference picture buffer 870 and can be used for inter prediction.

In FIGS. 7 and 8, each of the modules performs different functions. However, the present invention is not limited thereto, and one module may perform two or more functions. For example, in FIGS. 7 and 8, operations of the intra prediction unit and the inter prediction unit may be performed in one module (prediction unit).

7 and 8, one encoder / decoder processes all of the encoding / decoding for multi-view. However, this is for convenience of explanation, and the encoder / decoder may be configured for each view.

In this case, the encoder / decoder of the current view can perform encoding / decoding of the current view using information of another view. For example, the prediction unit (inter prediction unit) of the current view may perform intra prediction or inter prediction on the current block using pixel information of another view or reconstructed picture information.

Here, only the prediction between views is described as an example. However, the encoder / decoder can encode / decode the current layer using information of another view, regardless of whether the encoder / have.

The description of the view in the present invention can be similarly applied to a layer supporting scalability. For example, in the present invention, a view may be a layer.

9 is a diagram showing an example of a prediction structure for a three-dimensional video codec. For ease of explanation, FIG. 9 shows a prediction structure for encoding a depth information map corresponding to an actual image and an actual image acquired from three cameras.

In FIG. 9, the three actual images obtained from the three cameras are represented by T0, T1, and T2 according to the view, and the three depth information maps corresponding to the actual image are represented by D0 and D1 , And D2. Here, T0 and D0 are images acquired at View 0, T1 and D1 are images acquired at View 1, and T2 and D2 are images acquired at View 2, respectively. At this time, the rectangle shown in Fig. 9 represents an image (picture).

Each picture is divided into an I picture (Intra Picture), a P picture (Uni-prediction Picture), and a B picture (Bi-prediction Picture) according to the coding / decoding type. Each picture is coded / decoded As shown in FIG. In the I-picture, the video itself is encoded without inter-prediction. In the P-picture, the inter-prediction is performed using only the reference image existing in the unidirection, and the inter-prediction is performed using the reference pictures existing in both directions in the B-picture. At this time, the arrows in Fig. 9 indicate the prediction direction. That is, the actual image and its depth information map can be encoded / decoded depending on each other depending on the prediction direction.

In order to perform encoding / decoding of an image through inter prediction, motion information of the current block is required. As a method for deriving motion information of the current block, there are a method of using motion information of a block adjacent to the current block, a method of using temporal correlation within the same point of time, or a method of using inter-view correlation at an adjacent point. The method can be used interchangeably in one picture. Here, the current block refers to a block on which prediction is performed. The motion information may mean a motion vector, a reference picture number, and / or a prediction direction (e.g., whether unidirectional prediction, bi-directional prediction, temporal correlation, or inter-view correlation).

At this time, the prediction direction can be largely divided into unidirectional prediction and bidirectional prediction according to use of a reference picture list (Ref PictureList). The unidirectional prediction is divided into forward prediction (Pred_L0) using the forward reference picture list (LIST0, L0) and backward prediction (Pred_L1: Prediction L1) using the backward reference picture list (LIST1, do. The bidirectional prediction (Pred_BI) can be used to indicate that both the forward reference picture list (LIST0) and the backward reference picture list (LIST1) are both forward prediction and backward prediction. The list (LIST 0) is copied to the reverse reference picture list (LIST 1), and two forward predictions can be included in the bi-directional prediction.

The prediction direction can be defined using predFlagL0, predFlagL1. At this time, predFlagL0 is an indicator for indicating whether to use the forward reference picture list (List 0), and predFlagL1 corresponds to an indicator for indicating whether to use the backward reference picture list (List 1). For example, in the case of unidirectional prediction and forward prediction, predFlagL0 may be '1' and predFlagL1 may be '0'. In case of unidirectional prediction and backward prediction, predFlagL0 may be '0' and predFlagL1 may be ' In the case of bi-directional prediction, predFlagL0 may be '1' and predFlagL1 may be '1'.

FIG. 10 shows an example of neighboring blocks used to construct a merge candidate list for a current block.

The merge mode is one of the methods for performing inter prediction. In the merge mode, the merge mode is used as the motion information of the current block (e.g., at least one of a motion vector, a reference picture list, and a reference picture index) Can be used. Here, using motion information of a neighboring block as motion information of a current block is called merging, motion merging, or merging motion.

In the merge mode, a merging motion in units of a coding unit (CU) (hereinafter referred to as a 'CU') and a merging motion in units of a prediction unit (PU) are possible.

In the case of performing a merge movement with a block unit (for example, a CU or a PU) (hereinafter referred to as a 'block' for convenience of explanation), information about whether to perform a merge movement for each block partition, It is necessary to provide information on which of the neighboring blocks adjacent to the block should be merged.

A merging candidate list can be constructed to perform the merging motion.

The merge candidate list represents a list of motion information and may be generated before the merge mode is performed. Here, the motion information of the merge candidate list may be motion information of neighboring blocks adjacent to the current block, or may be new motion information created by combining motion information existing in the merge candidate list. Motion information (e.g., a motion vector and / or a reference picture index) of a neighboring block may be motion information specified by a neighboring block or stored in neighboring blocks (used for decoding a neighboring block).

10, neighboring blocks A, B, C, D, and E are located adjacent to the current block in a spatial manner, neighboring blocks A, B, C, D, and E, (H or M) at the same position corresponding to the corresponding block. A candidate block at the same position refers to a block at a corresponding position in a co-located picture corresponding to the current picture temporally corresponding to the current block. If the intra-picture H block in the same position is available, the H block is determined to be a candidate block at the same position. If the H block is not available, the intra-picture intra-picture block at the same position can be determined as a candidate block at the same position.

The motion information of the neighboring blocks A, B, C, D, and E and the candidate block H or M of the same position are merged candidates constituting the merge candidate list of the current block, It is judged whether or not it can be used as That is, the motion information of blocks available for inter prediction of the current block may be added to the merge candidate list as merge candidates.

For example, as a method of constructing a merge candidate list for an X block, 1) first, if neighboring block A is available, neighboring block A is included in the merge candidate list. 2), the neighboring block B is included in the merge candidate list only when the motion information of the neighboring block B is not the same as the motion information of the neighboring block A. 3) Including the neighboring block C in the merge candidate list only when the motion information of the neighboring block C is different from the motion information of the neighboring block B in the same manner, and 4) Only the neighboring block D is included in the merge candidate list. (5) Include the neighboring block E in the merge candidate list only if the motion information of the neighboring block E is different from the motion information of the neighboring block D. (6) Finally, the neighboring block H includes the neighboring block H . That is, each neighboring block may be added to the merge candidate list in order of A, B, C, D, E, and H (or M) blocks. Here, the same motion information may mean using the same motion vector, the same reference picture, and the same prediction direction (unidirectional, bidirectional).

Here, a combination of adding a neighboring block to a merge candidate list as a merge candidate and adding the motion information of a neighboring block to a merge candidate list as a merge candidate is mixed, but this is for convenience of explanation, I do not. For example, a neighboring block as a merge candidate may be referred to as motion information of the corresponding block.

11 is a flowchart of a method of constructing a merged motion candidate list in inter prediction.

Referring to FIG. 11, the inter-prediction unit derives a merged motion candidate, that is, a merge candidate (S1100). As described above, in order to efficiently encode motion information of a video encoding / decoding target block, neighboring motion information may be used in the remainder candidate. In 3D video coding / decoding, neighboring motion information includes motion parameter inheritance (MPI), inter-view merging candidate (IvMC), A1, B1, B0, View synthesis prediction (VSP), A0, B2, Shift Interview (Shift IV), Col, Bi, and Zero candidates for the inter-view disparity vector candidates IvDC.

Herein, MPI denotes motion information inheriting motion information of the actual image, IvMC denotes motion information using a merge between views, and A1 denotes motion information of the block located on the left side of the predicted block, . As described above, B1 means motion information of a block located on the upper side of the prediction target block, B0 means motion information of a block located on the upper right side of the prediction target block as described above, IvDC indicates the above- Means motion information derived by using a vector (i.e., using a parallax). A0 denotes motion information of a block located on the lower left side of the predicted block, and B2 denotes motion information of a block located on the upper left side of the predicted block as described above. And Shift IV denotes motion information derived using the corrected time difference. Col is motion information derived through a block at a corresponding position in a co-located picture temporally corresponding to a current picture including a current block, Bi is a motion induced using a bidirectional motion, And Zero means a zero vector.

Thereafter, the derived merge candidates are inserted into the merge candidate list in step S1110. In order to increase the entropy coding efficiency, the inter-prediction unit adds the merge candidates to the list in order of statistically most selected among the derived merging motion candidates. That is, the merge candidate having the statistically greatest selection is inserted at the first position of the merge candidate list, the merge candidate having the second largest selection is inserted at the second position of the merge candidate list, , The process of adding the statistically selected merge candidates to the merge candidate list in order is performed based on the statistical information.

For example, in the three-dimensional video coding / decoding, the inter-prediction unit calculates the merged candidate in the order of MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, Can be inserted into the list.

<Table 1>

Table 1 is an example of the percentage of merge candidates selected in the texture. The candidates in Table 1 correspond to the MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, Zero candidates described above, and MPI, IvMC, A1, B1, B0 IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi in the entropy encoding, and the ratio in which the IvDC, VSP, A0, B2, ShiftIV, Col, Bi, , And the average value of the Zero candidates. At this time, since MPI candidates are not used in texture, IvMC occupies the 0th index, and IvMC is selected as the highest ratio.

For example, as shown in Table 1, when the probability that IvMC is selected is 78%, when IvMC is set to the 0-th index in the 3-dimensional image coding / decoding and entropy encoding / decoding is performed, the least number of bits The coding / decoding efficiency is improved. Then, entropy encoding is performed in the order of A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, and Zero candidates. Encoding / decoding efficiency is improved.

12 schematically shows a hierarchical-B structure in 3D image coding / decoding.

Referring to Figure 12, ... The rectangles denoted by R3 are pictures at the reference point, and C0 ... ... The rectangles denoted by C3 denote pictures at the current point in time. In the hierarchical-B structure, motion information of a picture at the start of an arrow can be referred to when coding a picture for an end portion of an arrow. The numbers after R and C mean level, and the lower the level, the higher the distance between reference pictures. At this time, the distance between the reference pictures can be maximum 8 at the low level, and the error between the motion information of the picture to be referred to and the motion information of the current picture to be referred to becomes larger as the picture of lower level becomes low, so that the accuracy of the IvMC candidate may be lowered. In addition, in the hierarchical-B structure, motion information can be referred to as a structure indicated by a dotted line when IvMC candidates are derived.

<Table 2>

Table 2 shows an example of the ratio of selecting merge candidates in the real image when the inter-frame distance in the temporal direction is large in the hierarchical-B structure. For example, in the entropy encoding, the ratio of selecting IvMC candidates having a value of 0 is merely 2%, and when the average of the A1 candidates selected most frequently as merge candidates is 0.4, the efficiency of the three-dimensional image encoding / decoding decreases It comes. That is, when the A1 candidate is entropy-coded with 0, although the data can be sufficiently compressed, the A1 candidate is entropy-coded by 6 of 0 and 4 by 1, and the image encoding / decoding efficiency is lowered A problem may arise.

In addition to the problems described in Table 2, there may also be a problem due to differences in candidate selection ratios that occur depending on the accuracy of the depth map. In 3D video coding, parallax information can be derived using depth map. In this case, the selectivity of the candidates utilizing the parallax information such as IVDC, VSP, and IVSHIFT increases in a sequence having a high accuracy of the depth map. On the contrary, if the accuracy of the depth map is low, the selectivity of IVDC, VSP, and IVSHIFT candidates decreases, The selectivity of the COL candidate using the motion information of the block in the reference picture at the same position as the current picture within the viewpoint may increase as indicated by the solid line in Fig. Due to the inefficiency of the entropy encoding / decoding due to the difference between the index of the merge candidate list allocated to the merge candidate and the ratio of merge candidates selected as described above, the image encoding / decoding efficiency may be degraded.

As described above, since the selectivity of the merge candidates changes according to the image encoding / decoding situation, the existing method of generating the merge candidate list based on the merge candidate priorities can not maximize the efficiency of entropy encoding. Accordingly, in order to increase the entropy encoding efficiency at the merge candidate, an image encoding / decoding method in which frequently used merge candidates are inserted intensively in the vicinity of the 0 index of the merge candidate list by using an adaptive candidate priority, and A device is needed.

The main variables that can influence the distribution of the selection ratios of the merge candidates are the view number, whether the frame is a depth map, the temporal distance from the reference frame, or the type of the slice. The department can adaptively adjust the priority of the merge candidate to insert the merge candidate into the merge candidate list.

13 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to an embodiment of the present invention.

Referring to FIG. 13, in the inter-prediction unit, a merged motion candidate, that is, a merge candidate is derived (S1300). The concrete procedure for inducing the merge candidate is as described above. As described above, candidates for MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, Can be selected.

Thereafter, the inter-prediction unit determines whether it is necessary to change the order in which the remainder candidate is inserted into the merge list in the current situation, that is, to change the priority of the remainder candidate (S1310).

If it is necessary to change the priority for the merge candidate, the inter prediction unit changes the priority order of the merge candidate (S1330). For example, in the hierarchical-B structure, when the inter-frame distance in the temporal direction is large, a method of moving the priority of the IvMC candidate back can be used.

If the priority of the merge candidate is changed, a merge candidate list is generated in the order of the changed merge candidates, that is, the changed merge candidate is added to the merge candidate list (S1340).

If it is not necessary to change the priority of the merge candidate, the priority of the existing merge candidate is used in the inter-prediction unit (S1320). Then, in the inter prediction unit, a merge candidate list is generated in the order of merge candidates, that is, merge candidates are added to the merge candidate list in the existing order (S1340).

14 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to another embodiment of the present invention.

In Fig. 14, when the absolute error between the POC of the current picture (Picture Order Count, picture number indicating the reproduction order) and the POC of the reference picture of the PU to be encoded is 8 or more, the priority order of the IvMC candidate is changed to 2. In this case, curPOC is the POC of the current picture, and RefPOC is the POC of the reference picture of the current PU (prediction unit).

Specifically, the merged motion candidate, that is, the merge candidate is derived in the inter prediction unit (S1400). In this case, MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, Zero candidates can be selected as the merge candidate.

Then, the inter-prediction unit determines whether the absolute value of the difference between the POC of the current picture and the POC of the reference picture is greater than or equal to eight (S1410).

If the absolute value of the difference between the POC of the current picture and the POC of the reference picture is equal to or greater than 8, the priority of the IvMC candidate is changed to 2 in the inter-prediction unit (S1430).

When the priority of the IvMC candidate is changed to 2 in the inter-prediction unit, the inter-prediction unit inserts the modified merge candidate into the merge candidate list (S1440).

If the absolute value of the difference between the POC of the current picture and the POC of the reference picture is not greater than or equal to 8, the inter prediction unit uses the ranking of the existing merge candidate (S1420). Then, in the inter prediction unit, the merge candidate is inserted into the merge candidate list in the order of existing merge candidates (S1440).

Table 3 below shows the driving process of the apparatus for changing the priority of IvMC candidates according to the POC difference.

<Table 3>

15 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to another embodiment of the present invention.

Referring to FIG. 15, the merged motion candidate, that is, merge candidate is derived in the inter prediction unit (S1500). In this case, MPI, IvMC, A1, B1, B0, IvDC, VSP, A0, B2, ShiftIV, Col, Bi, Zero candidates can be selected as the merge candidate.

Then, the inter-prediction unit determines whether the accuracy of the depth map of the sequence is equal to or greater than a threshold (S1510). In this case, the criterion for determining the accuracy of the depth map is a method of relatively comparing the selection ratio between the Col candidate and the parallax information candidate. In addition, there is a method of directly determining by the user performing encoding, encoding the encoded data by a parameter of a sequence, and transmitting the encoded data. For example, in the case of 3D-HEVC, it can be transmitted through high-level syntax such as SPS or VPS.

If the accuracy of the depth map of the sequence is equal to or greater than the threshold value, the priority of the Col candidate is changed to 4 in the inter-prediction unit (S1530).

If the priority of the Col candidate is changed to 4 in the inter-prediction unit, the inter-prediction unit inserts the modified merge candidate into the merge candidate list (S1540).

If the accuracy of the depth map of the sequence is smaller than the threshold value, the inter prediction unit uses the rank of the existing merge candidate (S1520). Thereafter, in the inter prediction unit, the merge candidate is inserted into the merge candidate list in the order of existing merge candidates (S1540).

Table 4 below shows the driving process of the apparatus whose priority is changed according to the accuracy of the depth map.

<Table 4>

Table 5 shows an example of a method of encoding and transmitting an accuracy value of a depth map through a high-level syntax such as SPS or VPS in a 3D-HEVC. At this time, in Table 5, the syntax of "depth_accuracy" is added to the VPS.

<Table 5>

16 is a flowchart for generating a merge candidate list by changing a priority of merge candidates according to another embodiment of the present invention.

Referring to FIG. 16, in the inter-prediction unit, candidate priority information for a specific state is stored in slice units, and the priority stored in the next slice coding, which is the same as the corresponding state, can be used. That is, in the inter-prediction unit, the priority of the merge candidate can be determined in the order of the candidates having the high selection ratio, based on the statistics of the merge candidate selection ratios based on the encoding results of a certain number or more. In this case, the specific state may be a specific value of the distance from the reference picture, or a slice type such as P and B, and the specific state may be any other feature. If the order of the selection ratios of the current candidate and the current candidate does not match after the slice coding ends, the priorities of the candidate candidates for the current state are rearranged so that they can be applied to the next slice coding, have.

Specifically, in the inter prediction unit, the priority order of the merge candidate is changed (S1600). In this case, the merge candidate may use the merge candidate selection ratio based on the encoding result at least a certain number of times.

Thereafter, the inter prediction unit performs coding on the current slice (S1610).

After the slice is encoded, the inter-prediction unit determines whether the statistics of the ratio to the merge candidate actually selected and the priority of the merge candidates correspond (S1620).

If the statistics of the ratio to the merge candidate actually selected by the inter-prediction unit does not correspond to the priority of the merge candidates, the inter-prediction unit rearranges the priorities of merge candidates for the current state (S1630).

<Table 6>

Table 6 shows the driving process when using statistics of candidate selection ratios for specific states. In Table 6, the encoding result of all the slices that are the same as the current state, not the current slice encoding result, may be used, or a range of encoding results may be utilized. When the candidate priorities for the current state are modified in 3-B of Table 6, the candidate priorities may be modified to match the selected ratio results, or the selected ratio results and the previously stored candidate priorities may be weighted , And the priority may be gradually changed in accordance with the repetition of the slice encoding.

17 is a flowchart for generating a merge candidate list by changing a priority of a merge candidate according to another embodiment of the present invention.

Referring to FIG. 17, when the candidate priority is changed during image encoding, priority information can be transmitted to the outside.

Specifically, the priority of the merge candidate is changed in the inter-prediction unit (S1700). In this case, the merge candidate may use the merge candidate selection ratio based on the encoding result at least a certain number of times.

Thereafter, the inter prediction unit performs coding on the current slice (S1710).

After the slice is encoded, the inter-prediction unit determines whether the statistics of the ratio to the merge candidate actually selected and the priority of the merge candidates correspond (S1720).

If the statistics of the ratio to the merge candidate actually selected by the inter-prediction unit does not correspond to the priority of the merge candidates, the inter-prediction unit rearranges the priorities of merge candidates for the current state (S1730).

After the inter-prediction unit rearranges the priority of the merge candidate for the current state, the changed priority information is included in the slice header and can be encoded (S1740).

Table 7 shows an example in which the update information of the candidate priority for a specific state is encoded. The update information of the candidate priority for the specific state can be appended using the &quot; merging_cand_priority_change_flag &quot; flag. If the merging_cand_priority_change_flag flag is 1, the priority order of the updated merge candidates is encoded in order. At this time, the priority of the i-th candidate is encoded as merging_cand_priority [i]. In addition, the priority update information for the merge candidate may be coded in a header of a different type than the slice header.

<Table 7>

18 is a block diagram schematically illustrating the structure of an inter-prediction unit according to an embodiment of the present invention.

Referring to FIG. 18, the inter-prediction unit 1800 may include a motion candidate inducing unit 1810 and a candidate list generating unit 1820. At this time, the inter prediction unit 1800 may be included in the coding apparatus or the decoding apparatus described above.

The motion candidate induction unit 1810 may refer to an induction unit for the merge candidate, and the motion candidate induction unit may derive motion information of a block located in the vicinity of the current block, i.e., merge candidate in inter prediction. At this time, the concrete method for deriving the merge candidate is as described above.

The candidate list generation unit 1820 can generate a list of merge candidates, that is, a merge candidate list, and the concrete method of generating merge candidates in the candidate list generation unit 1820 is as described above.

In the above-described embodiments, although the methods are described on the basis of a flowchart as a series of steps or units, the present invention is not limited to the order of the steps, and some steps may occur in different orders or simultaneously . It will also be understood by those skilled in the art that the steps depicted in the flowchart illustrations are not exclusive and that other steps may be included or that one or more steps in the flowchart may be deleted without affecting the scope of the invention You will understand.

The above-described embodiments include examples of various aspects. While it is not possible to describe every possible combination for expressing various aspects, one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the invention include all alternatives, modifications and variations that fall within the scope of the following claims.

The method according to the present invention may be implemented as a program for execution on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include a ROM, a RAM, a CD- , A floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet).

The computer readable recording medium may be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And, functional programs, codes and code segments for implementing the above method can be easily inferred by programmers of the technical field to which the present invention belongs.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

Claims (1)

Deriving a merge candidate;
Determining whether a change to the priority of the merge candidate is needed;
Changing a priority of the remainder candidate when the change of the priority of the remainder candidate is needed; And
Adding the merge candidate to the merge candidate list based on the priority of the modified merge candidate
Dimensional image encoding method.

Images