KR20160002194A - Adaptive merging candidate selection method and apparatus - Google Patents

Abstract

The present invention relates to a method for encoding/decoding a video and a device thereof. The method comprises a method for adaptively selecting a motion candidate index without transmitting the motion candidate index. According to the present invention, video encoding efficiency can be increased.

Description

METHOD AND APPARATUS FOR SELECTION OF ADAPTIVE MERGE CANDIDATE {

The present invention relates to a video encoding / decoding method and apparatus.

3D video provides users with a stereoscopic effect as if they are seeing and feeling in the real world through a 3D stereoscopic display device. As a result of this research, the Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V), a joint standardization group of ISO / IEC Moving Picture Experts Group (MPEG) and ITU-T VCEG (Video Coding Experts Group) Video standards are in progress. The 3D video standard includes standards for advanced data formats and related technologies that can support playback of autostereoscopic images as well as stereoscopic images using real images and their depth information maps.

The present invention proposes a method for increasing video coding efficiency.

The present invention includes a method for adaptively selecting a motion candidate index without transmitting it

The present invention can increase the coding efficiency of video coding.

Figure 1. Basic structure and data format of 3D video system
2. An example of a real image of a "balloons" image and a depth information map image: (a) a real image, (b) a depth information map
3. An example of a method of dividing a CTU (CTU (LCU)) into CU units
Figure 4. Example of partition structure of PU
Figure 5. An example of the inter-view prediction structure in 3D Video Codec
Figure 6. Implementation of 3D Video Encoder / Decoder
Figure 7. Example of the prediction structure of 3D Video Codec
8. An example of neighboring blocks of a current block used as a merging motion candidate list
Figure 9. Existing process where merged motion candidates are added to the list
10. Suggested adaptive merge candidate selection method
11. Example of proposed adaptive merge candidate selection method

3D video provides users with a stereoscopic effect as if they are seeing and feeling in the real world through a 3D stereoscopic display device. As a result of this research, the Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V), a joint standardization group of ISO / IEC Moving Picture Experts Group (MPEG) and ITU-T VCEG (Video Coding Experts Group) Video standards are in progress. The 3D video standard includes standards for advanced data formats and related technologies that can support playback of autostereoscopic images as well as stereoscopic images using real images and their depth information maps.

The basic 3D video system considered in the 3D video standard is shown in FIG. On the transmitting side, the image content of N (N ≥ 2) viewpoints is acquired by using a stereo camera, a depth information camera, a multi-view camera, and a two-dimensional image into a three-dimensional image. The obtained image content may include video information of the N view point, its depth map information, camera-related additional information, and the like. The video content at time point N is compressed using the multi-view video encoding method, and the compressed bitstream is transmitted to the terminal through the network. The receiving side decodes the transmitted bit stream using the multi-view video decoding method, and restores the N view image. The reconstructed N-view image generates virtual view images at N or more viewpoints by a depth-image-based rendering (DIBR) process. The generated virtual viewpoint images are reproduced in accordance with various stereoscopic display devices to provide stereoscopic images to the user.

The depth information map used to generate the virtual viewpoint image is a representation of the distance between the camera and the actual object in the real world (depth information corresponding to each pixel at the same resolution as the real image) in a fixed number of bits. The method of generating the depth information map can be divided into a method of acquiring using a camera and a method of automatically generating an actual image using a texture image. In the case of a method of acquiring using a camera, there is a problem that the depth information camera operates only within a certain distance. On the other hand, in the case of a method of automatically generating using a real general image, a depth information map is generated using a disparity between two general images. In other words, the method compares an arbitrary one pixel at the current time with the pixels at the surrounding viewpoint, finds the pixel of the best matching area, and expresses the distance between the pixels by the depth information. FIG. 2 shows an example of a depth information map automatically generated by using a real general image. FIG. 2 shows an example of a "balloons" image (FIG. 2 (a)) used in the 3D video coding standard of MPEG, (Fig. 2 (b)). The depth information map shown in FIG. 2 actually represents depth information on the screen in 8 bits per pixel.

As an example of a method of encoding an actual image and its depth information map, encoding can be performed using H.264 / AVC (MPEG-4 Part 10 Advanced Video Coding), and another example of MPEG Picture Experts Group) and VCEG (Video Coding Experts Group), which are standardized by the High Efficiency Video Coding (HEVC).

The HEVC performs coding with a CU (CU) to efficiently encode an image. 3 is a diagram illustrating a method of dividing a CU within a CTU (Coding Tree Unit) (hereinafter, also referred to as CTU (LCU) (Largest Coding Unit)) when coding an image.

As shown in FIG. 3, the HEVC sequentially divides an image into CTU (LCU) units, and then determines a divided structure by CTU (LCU) units. A partition structure means a distribution of CUs for efficiently encoding an image in a CTU (LCU), which distribution can be determined by determining whether the CU is divided into four CUs whose size is reduced by half the length and half. A partitioned CU can be recursively partitioned into four CUs, which are reduced in half by half in the same manner. At this time, the division of the CU can be performed to a predetermined depth. The depth information is information indicating the size of the CU and is stored in all the CUs. The depth of the underlying CTU (LCU) is zero, and the depth of the SCU is a predefined maximum depth. The depth of the CU increases by one every time a division is made in the horizontal and vertical halves from the CTU (LCU). For each depth, CUs that are not partitioned are sized to 2Nx2N, and when partitioning is performed, they are divided into 4 CUs of NxN size. The size of N decreases by half every time the depth increases by one. Figure 5 shows an example where the size of the smallest coding unit (SCU) with a minimum depth of 0 is 64 × 64 pixels and the maximum depth is 3 × 8 pixels. A 64x64 pixel CU (CTU (LCU)) has a depth of '0', a 32x32 pixel CU has a depth of 1, a 16x16 pixel CU has a depth of 2, and an 8x8 CU (SCU) has a depth of 3. Also, information on whether to divide a specific CU is represented by division information, which is 1-bit information for each CU. This partition information is included in all the CUs except for the SCU. When not partitioning the CU, '0' is stored in the partition information, and '1' is stored in case of partitioning.

Such a CU is an encoding unit and can have a coding mode in units of one CU. That is, each of the CUs can be divided into intra picture coding (also referred to as MODE_INTRA or INTRA) mode or inter picture coding (also referred to as MODE_INTER or INTER). The inter-picture coding (MODE_INTER) mode can be divided into a MODE_INTER mode and a MODE_SKIP (also referred to as SKIP) mode.

PU (Prediction Unit) is a unit of prediction. As shown in FIG. 4, one CU may be divided into several PUs (PUs) to be predicted. If the coding mode of one CU is INTRA, all the PUs in the CU are encoded into INTRA, and if the encoding mode of one CU is INTER, all the PUs in the CU are encoded into INTER. For example, when one CU is INTRA, the partition structure of the PU may include only PU 2Nx2N and PU NxN. For example, if one CU is INTER, then the PU may have all of the partition structures of FIG.

The actual image and its depth information map can be images obtained from multiple cameras as well as one camera. Images obtained from multiple cameras can be independently encoded, and a general two-dimensional video coding codec can be used. In addition, since images acquired from a plurality of cameras have correlation between viewpoints, images can be encoded using different inter-view prediction in order to increase the encoding efficiency. In one embodiment, FIG. 5 shows an example of an inter-view prediction structure for images acquired from three cameras.

5, view 1 is an image acquired from a camera located on the left side based on view 0 (View 0), view 2 (view 2) is an image acquired from a camera located on the right side based on view 0 . View 1 and View 2 perform inter-view prediction using View 0 as a reference picture and the coding order is View 1 and View 2 (View 0) must be encoded first. At this time, View 0 can be independently encoded regardless of other viewpoints, so it is referred to as an independent view. On the other hand, View 1 and View 2 are called Dependent View because View 0 is used as a reference image. The independent viewpoint image can be encoded using a general two-dimensional video codec. On the other hand, since the dependent view image must perform the inter-view prediction, it can be encoded using the 3D video codec including the inter-view prediction process.

Also, in order to increase the coding efficiency of view 1 (view 1) and view 2 (view 2), it is possible to encode using depth information map. For example, when an actual image and its depth information map are encoded, they can be independently encoded / decoded. Also, when the actual image and the depth information map are encoded, they can be encoded / decoded dependently on each other as shown in FIG. In one embodiment, an actual image can be encoded / decoded using an already-encoded / decoded depth information map, and conversely, a depth information map can be encoded / decoded using an already encoded / decoded real image.

In one embodiment, an encoding prediction structure for encoding an actual image obtained in three cameras and a depth information map thereof is shown in Fig. In FIG. 7, three actual images are shown as T0, T1, and T2 according to the viewpoint, and three depth information maps at the same position as the actual image are shown as D0, D1, and D2 according to the viewpoint. Here, T0 and D0 are images obtained at View 0, T1 and D1 are images acquired at View 1, and T2 and D2 are images acquired at View 2, respectively. Each picture can be encoded into I (Intra Picture), P (Uni-prediction Picture), and B (Bi-prediction Picture). In Fig. 7, arrows indicate prediction directions. That is, the actual image and its depth information map are encoded / decoded depending on each other.

It may mean motion information of current block (only motion vector of the current block in the real image, or motion vector, reference picture number, unidirectional prediction, bi-directional prediction, inter-view prediction, temporal prediction or other prediction). The method for inferring is divided into temporal prediction and inter-view prediction. Temporal prediction is a prediction method using temporal correlation within the same time, and inter-view prediction is a prediction method using inter-view correlation at an adjacent time. Such temporal prediction and inter-view prediction can be used in combination with each other in one picture.

A merge method is used as a method of coding motion information in image encoding / decoding. At this time, the motion information is information including at least one of a motion vector, an index for a reference image, and a prediction direction (unidirectional, bidirectional, etc.). The prediction direction can be largely divided into unidirectional prediction and bidirectional prediction according to use of a reference picture list (Ref PictureList). The unidirectional prediction is divided into forward prediction (Pred_L0) using the forward reference picture list (LIST0) and backward prediction (Pred_L1; Prediction L1) using the reverse reference picture list (LIST1). The bidirectional prediction (Pred_BI) uses both the forward reference picture list (LIST 0) and the backward reference picture list (LIST 1), and both the forward prediction and the backward prediction are present. In addition, the forward reference picture list (LIST0) may be copied to the reverse reference picture list (LIST1), and two forward prediction may be included in the bidirectional prediction. The prediction direction can be defined using predFlagL0, predFlagL1. For example, in the case of unidirectional prediction and forward prediction, predFlagL0 may be '1' and predFlagL1 may be '0'. In case of unidirectional prediction and backward prediction, predFlagL0 may be '0' and predFlagL1 may be '1'. In case of bi-directional prediction, predFlagL0 may be '1' and predFlagL1 may be '1'.

The merging motion can be a merge motion in units of a coding unit (CU) and a merging motion in units of a prediction unit (PU). In the case of performing a merge movement in units of a CU or a PU (hereinafter, referred to as a 'block' for convenience of description), information on whether to perform a merge movement for each block partition and information on whether to perform a merge movement on neighboring blocks It is necessary to transmit information on which of the blocks adjacent to the left side of the block, the adjacent block on the upper side of the current block, the temporally neighboring block of the current block, etc., is to be merged.

The merge candidate list (List) represents a list in which the motion information is stored, and is generated before the merge movement is performed. Here, the motion information stored in the merged motion candidate list may be motion information of a neighboring block adjacent to the current block or motion information of a collocated block corresponding to the current block in the reference image. Also, the motion information stored in the merged motion candidate list may be new motion information created by combining motion information already present in the merged motion candidate list.

The merged motion candidate list is used for motion information of the current block in the neighboring blocks A, B, C, D, and E and the candidate block H (or M) And if it is available, the motion information of the block can be input to the merged motion candidate list. Also, if each neighboring block has the same motion information, the motion information of the neighboring block is not included in the merged motion candidate list. For example, when generating a merged motion candidate list for X block in FIG. 8, if neighboring block A is available and included in the merged motion candidate list, neighboring block B is not the same motion information as neighboring block A Only candidate merging motion candidate list. In the same way, the neighboring block C can be included in the merged motion candidate list only when it is not the same motion information as the neighboring block B. Can be applied to the peripheral block D and the peripheral block E in the same way. Here, the same motion information may mean that the same motion vector is used and the same reference picture is used and the same prediction direction (unidirectional (forward, reverse), bidirectional) is used. Finally, in FIG. 8, the merged motion candidate list for the X block may be added to the list in a predetermined order, for example, A → B → C → D → E → H (or M) block order.

In video coding, surrounding motion information can be used to efficiently encode motion information. The merged motion candidates derived from the neighboring motion information are inserted into the merged motion candidate list as shown in FIG. After that, the encoding is virtually attempted, and the candidate with the best coding efficiency is finally selected. The selected candidate is encoded using entropy encoding.

In this case, in order to increase the efficiency of entropy encoding, it is necessary to select a large number of candidates at the first position in the list and to encode the index of 0's at a large number of times. Therefore, there is a priority between the candidates, and the candidates are added to the list in the statistically most selected order. For example, in the current 3D-HEVC, MPI (inheritance from texture), IvMC (inter-view motion), A1 (left), B1 (top), B0 (top), IvDC (time difference) (Left), B2 (top left), ShiftIV (correction time difference), Col (same position), Bi (bi-directional) and Zero (zero vector) candidates are inserted into the merged motion candidate list in the order listed.

Table 1 is an example of a merged motion candidate selection ratio of a texture. Since MPI candidates are not used in textures, IvMC occupies the 0th index and is selected at the highest rate. 78% of the candidates are entropy-encoded with a value of 0, so that the coding efficiency is good. Subsequent candidates are also listed in order of increasing selection ratio, although they are not accurate.

There are many kinds of merging motion candidates. When a block using motion merging is encoded, a candidate index indicating which candidate to use in the list must be transmitted, which occupies a large portion in the compressed bitstream. Therefore, if the index transmission can be omitted by adaptively selecting the candidate index, the video encoding efficiency can be increased.

When constructing a merging candidate list in video coding, various kinds of candidates are used. At this time, for candidates used with a high frequency, there is a high correlation with motion candidate selection between the neighboring block and the current block.

In the present invention, when a neighboring block uses a specific candidate, a candidate used in neighboring blocks is used as a motion candidate of the current block without encoding an index of a candidate to be used as motion information on the motion candidate list when encoding the current block . A conceptual diagram of the proposed method is shown in FIG. It is not necessary to transmit the motion candidate index through the proposed method, so that the coding efficiency can be increased.

11 is an example of applying the proposed method to motion merging of 3D-HEVC. If the current block does not belong to the base view texture and satisfies one of the following conditions, the merged motion candidate is not encoded.

- Condition 1: The current picture is not a depth map, and the co-located block in the previous picture is the same as the current view, and the inter-view merging candidate (IVMC) is used.

- Condition 2: the current picture is a depth map, a co-located block in the texture corresponding to the current picture uses Motion Parameter Inheritance (MPI), and a motion inheritance candidate is available in the current block

Claims (4)

In the method of selecting one of the candidates in the motion candidate list and using it as the motion information of the current block, the motion candidate used by the neighboring block is used as the motion candidate of the current block and the transmission of the motion candidate index is omitted The method of claim 1, further comprising: using a motion candidate used in neighboring blocks spatially adjacent to the current block The method according to claim 1, further comprising the steps of: using a motion candidate used in a block temporally adjacent to the current block as a neighboring block The method according to claim 1, further comprising: using a motion candidate used as a neighboring block in a block adjacent to the current block

Images