KR100893930B1 - Time direct prediction method for encoding multi-view video - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a temporal direct prediction method for multiview video coding.

2. The technical problem to be solved by the invention

The present invention predicts a motion vector of a target block using a temporal correlation when the block (reference block) referred to by the anchor block to the target block to be encoded is the same view image. By predicting the motion vector of the block, it is possible to precisely predict the motion information of the current target block in the multiview image, thereby improving the compression efficiency, thereby providing a temporal direct prediction method for multiview video coding. There is a purpose.

3. Summary of Solution to Invention

According to the present invention, in the direct temporal prediction method for multi-view video encoding, a block (reference block) referenced by an anchor block for a target block to be encoded is located at a different view (View) on the same time axis as the anchor block. A reference block checking step of checking whether the image block is an image block; And a multi-view prediction step of predicting a motion vector of the target block using the motion vector of the anchor block if the reference block is an image block of another view.

4. Important uses of the invention

The present invention is used for temporal direct prediction for multi-view video encoding.

Multi-view video coding, temporal direct prediction, spatial direct prediction, motion vector, target block, target picture, anchor block, anchor picture, reference block, reference picture, temporal correlation

Description

Time-directed prediction for multi-view video coding {TIME DIRECT PREDICTION METHOD FOR ENCODING MULTI-VIEW VIDEO}

1 is an explanatory diagram of a conventional time direct prediction mode;

2 is an explanatory diagram of a conventional temporal direct prediction mode in multiview video encoding;

3 is a structural diagram of a picture group used in multiview video encoding;

4 is a diagram for explaining a direct temporal prediction method for multi-view video encoding according to the present invention;

5 and 6 are diagrams illustrating one embodiment of a motion vector of an anchor block in a direct temporal prediction method for multi-view video encoding according to the present invention;

7 is a flowchart illustrating a method for directly predicting time for multiview video encoding according to the present invention;

8 is a structural diagram of an embodiment of a picture group in a time direct prediction method applied to the present invention;

9A to 9C are comparison diagrams of a signal-to-noise ratio according to peak power measured according to a temporal direct prediction method and a conventional spatial direct prediction method for multi-view video encoding according to the present invention.

* Explanation of symbols on the main parts of the drawing

40: target picture 401: target block

41: anchor picture 411: anchor block

413, 415: reference block

The present invention relates to a temporal direct prediction method for multiview video encoding. More specifically, when a block (reference block) referenced by an anchor block for a target block to be encoded is the same view image, temporal correlation Predict the motion vector of the target block by using, but in particular, by using the motion vector of the anchor block if it is another viewpoint image, it is possible to accurately predict the motion information of the current target block in the multi-view image, thereby improving the compression efficiency. The present invention relates to a temporal direct prediction method for multiview video encoding.

The direct prediction mode refers to an encoding mode that predicts motion information of a block to be encoded using motion information of a block in which motion information is encoded. In the direct prediction mode, since the encoding can be performed without the number of bits required for encoding the motion information, the compression efficiency is improved. In the direct prediction mode, not only does it need to transmit residual data, but also because of the high frequency of the direct prediction mode, the encoding efficiency is increased in H.264 using entropy coding. The compression effect obtained by using the direct prediction mode depends on the precision when the motion information of the current block is predicted using the encoded block. Here, the precision means the degree to which accurate motion information can be predicted.

Such direct prediction modes include temporal direct prediction mode using motion information correlation in the temporal direction and spatial direct prediction mode using motion information correlation in the spatial direction (for example, horizontal and vertical two-dimensional space in a picture). Can be divided.

Looking at the two direct prediction modes provided in H.264 / AVC, the temporal direct prediction mode is used when the speed of the motion is constant in an image having different motions. In this case, the prediction effect in the temporal direct prediction mode is greater than in the spatial direct prediction mode. Since the spatial correlation between blocks is lost when different motions are included, it is difficult to predict in the spatial direct prediction mode.

Next, the spatial direct prediction mode is used when the speed of motion changes in an image having the same motion. In this case, the prediction effect is larger in the spatial direct prediction mode than in the temporal direct prediction mode. Since the temporal motion correlation disappears when the speed of motion changes, it is difficult to predict in the time direct prediction mode.

1 is an explanatory diagram of a conventional time direct prediction mode.

As shown in FIG. 1, in the conventional temporal direct prediction mode, the anchor picture 11 is the closest reference picture in the reverse direction of the target picture 10 to be currently encoded. A block at the same position as the target block 101 in the anchor picture 11 becomes the anchor block 111. If the motion vector of the anchor block 111 is set as the motion vector of the co-located block (MVCOL) 13, the image block 121 in the reference picture 12 indicated by the MVCOL 13 can be obtained. have.

In the conventional temporal direct prediction mode, it is assumed that the moving speed between the reference picture 12, the target picture 10, and the anchor picture 11 is constant. If the first and second motion vectors of the target block 101 are determined as MVL0 141 and MVL1 142, respectively, the magnitudes of MVL0 141 and MVL1 142 can be obtained using Equation 1 below. Can be.

MVL0 = MVCOL × TB / TD

MVL1 = -MVCOL X (TD-TB) / TD = MVL0-MVCOL

Here, MVCOL is the motion vector of the anchor block, MVL0 is the first motion vector of the target block, MVL1 is the second motion vector of the target block, TB is the time from the reference picture viewpoint to the target picture viewpoint, and TD is the anchor from the reference picture viewpoint. The time until the picture viewpoint is shown.

The sizes of the MVL0 141 and the MVL1 142 are obtained as in Equation 1 using the proportionality between the reference picture 12, the target picture 10, and the anchor picture 11. Accordingly, in the direct temporal prediction mode, the target block 101 may be encoded using the image blocks 122 and 112 corresponding to the MVL0 141 and the MVL1 142, respectively.

In the temporal direct prediction mode, the motion vector of the target block is determined by dividing the motion vector of the anchor block at the same position as the target block to be encoded by the distance between L0 and L1, respectively. Therefore, the prediction efficiency may be high in a panning image having relatively constant motion.

However, in multi-view video encoding in H.264 / AVC, only the spatial direct prediction mode is used among the two direct prediction modes. The conventional multi-view coding structure does not support the temporal direct prediction mode. This is because the reference picture used in the temporal direct prediction mode and the reference picture used in the multiview video are different. In other words, in order to use the temporal direct prediction mode, the reference picture must exist in a continuous picture taken by the same camera, but in the multi-view video, the anchor block refers to the image of another camera at the same time. Cannot be used as is.

2 is an explanatory diagram of a conventional temporal direct prediction mode in multiview video encoding.

As shown in FIG. 2, the case where the conventional temporal direct prediction mode is applied to multi-view video encoding is as follows.

Referring to the case where the target block 201 is encoded in the target picture 20 of the C1 camera by applying a temporal direct prediction mode in multi-view video encoding, the motion vector required for encoding is the anchor block 211 in the anchor picture 21. The MVCOL 222 which points to the reference block 221 in this reference picture 22 on the time axis.

However, the motion vectors of the actual anchor block 211 are MVC0L0 212 and MVC0L1 213. The MVCOL0 212 and the MVCOL1 213 are different from the motion vector (MVCOL) 222 necessary for encoding. Therefore, in the conventional time prediction mode, since the anchor block 211 refers to image blocks of different views on the same time axis, it is difficult to obtain a motion vector of the target block 201 using the conventional time direct prediction mode. have.

3 is a structural diagram of a picture group used in multiview video encoding.

As shown in FIG. 3, a group of pictures (GOPs) used in multiview video encoding includes frames of an image photographed by the cameras C0 to C7. In this group of pictures, a frame in which temporal direct prediction mode is enabled and a frame not available will be described.

In the case where the B9 frame is to be encoded in the picture group structure, only reference frames are I0 frames and I8 frames. In this case, like the conventional H.264 / AVC, the B9 frame can be encoded using the conventional time direct prediction mode. For example, the conventional encoding using the temporal direct prediction mode is equally applied to all the B frames B9 to B19 of the image photographed by the camera No.0. Since the frames B9 to B19 all have the reference picture in the time axis direction (horizontal direction), the conventional time direct prediction mode can be used. The B frames of cameras 2, 4, 6, and 7 also have most of the anchor pictures in the horizontal direction. Even in this case, the conventional time direct prediction mode can be used as it is.

On the other hand, the B45 frame has a P44 frame as an anchor picture. The reference picture of the P44 frame becomes the P20 frame to refer to the picture of another camera. In this case, there is a problem that the conventional time direct prediction mode cannot be used.

Also, an image taken by cameras 1, 3, and 5 may also exist in a reference picture of an anchor picture in an image captured by another camera on the same time axis. The anchor picture of the B34 frame is a B33 frame, and the reference picture of the B33 frame may be a B2 frame or a B32 frame, or a B9 frame or a B21 frame. When the reference picture of the B33 frame is a B9 frame or a B21 frame, the anchor block of the target picture to be encoded refers to a picture on another camera on the same time axis. Therefore, in this case, there is a problem that the conventional time direct prediction mode cannot be used.

The present invention has been proposed to solve the above problem, and if the block (reference block) referenced by the anchor block for the target block to be encoded is the same view image, the motion vector of the target block using temporal correlation Multi-view video encoding, which predicts the motion information of the current target block in the multi-view image by predicting the motion block of the anchor block, in particular, if it is another viewpoint image, thereby improving compression efficiency. The purpose of the present invention is to provide a direct time prediction method.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to the present invention for achieving the above object, in the time direct prediction method for multi-view video encoding, a block (reference block) referenced by an anchor block for a target block to be encoded is located on the same time axis as the anchor block. A reference block checking step of checking whether the image block is at another view; And a multi-view prediction step of predicting the motion vector of the target block using the motion vector of the anchor block if the reference block is an image block of another view.

The present invention may further include a same-view prediction step of predicting a motion vector of the target block using a temporal correlation when the reference block is an image block having the same view as the anchor block.

The present invention also provides a second motion vector prediction that predicts a motion vector of the target block by using a temporal correlation when the reference block is a video block at a different time than the anchor block and a video block at the same time. It further comprises a step.

The present invention also provides a second motion vector prediction that predicts a motion vector of the target block by using a temporal correlation when the reference block is a video block at a different time than the anchor block and a video block at the same time. It further comprises a step.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

4 is a diagram illustrating an embodiment of a direct temporal prediction method for multi-view video encoding according to the present invention.

As shown in Fig. 4, the anchor picture 41 is determined as the reference picture in the reverse direction closest to the target picture 40 to be encoded. The anchor block 411 is an image block of another view (View) on the same time axis as the target block 401 to be currently encoded in the anchor picture 41. When the motion vectors of the anchor block 411 are MVCOL0 412 and MVCOL1 414, the MVCOL0 412 and MVCOL1 414 refer to images of other cameras on the same time axis. That is, the anchor block 411 refers to two reference blocks 413 and 415 according to the motion vectors MVCOL0 and MVCOL1 412 and 414 of the anchor block 411.

If the anchor block 411 has both the motion vectors of the MVCOL0 412 and the MLCOL1 414, the MVCOL0 412 and the MLCOL1 414, which are the motion vectors of the anchor block 411, may be combined to form the target block 401. The motion vectors MVL0 402 and MVL1 404 can be predicted.

The object block 401 is encoded using the image blocks 403 and 405 indicated by the MVL0 412 and the MVL1 414 predicted in the temporal direct prediction mode for multiview video encoding.

5 and 6 are diagrams illustrating an embodiment of one motion vector of an anchor block in a temporal direct prediction method for multiview video encoding according to the present invention.

As shown in FIGS. 5 and 6, when the anchor picture is a B picture, the anchor picture may have only one motion vector by performing one direction prediction instead of the bidirectional prediction, and the reference picture may be a P picture. In this case, since the anchor block has only one motion vector, the anchor block also has only one motion vector of the target block that uses the motion vector of the anchor picture as it is. In this case, referring to Figures 5 and 6 as follows.

As shown in FIG. 5, a target block 501 is present in the target picture 50, and an anchor block 511 is present in the anchor picture 51. The motion vector 502 of the target block 501 is obtained using the motion vector 512 of the image block 513 indicated by the anchor block 511. The target block 501 may be encoded using the image block 503 indicated by the motion vector 502.

Similarly, as shown in FIG. 6, when referring to an image of a camera different from FIG. 5, an anchor block 611 in the anchor picture 61 is used to encode the target block 601 in the target picture 60. The motion vector 602 of the target block 601 is obtained using the motion vector 612 of the pointing image block 613. The target block 601 is encoded using the video block 603 indicated by the motion vector 602.

7 is a flowchart illustrating an embodiment of a direct temporal prediction method for multiview video encoding according to the present invention.

A direct temporal prediction method for multi-view video encoding according to the present invention will be described with reference to the embodiments illustrated in FIGS. 4 to 6.

The temporal direct prediction method for multiview video encoding first searches for an anchor block for a target block to be encoded (702). In the present invention, the closest video block among the video blocks to which the target block can be referred to is defined as an anchor block.

The motion vector of the anchor block obtained in the process "702" is calculated and the image block indicated by the calculated motion vector is found (704). The process "704" refers to a process of searching for a block (reference block) to which the anchor block refers. In the present invention, among the image blocks on the same time axis as the anchor block, the image block at the shortest distance from the anchor block is defined as the reference block.

In this case, it is checked whether the image block found in step 704, that is, the reference block, is an image block of a different view on the same time axis (706). In this case, the check result 706, the reference block may be an image block of different views on the same time axis, an image block of the same view, or both image blocks of the same view and different views. Hereinafter, the process "708" will be described with reference to FIGS. 4 to 6.

As a result of the check 706, if the reference block is an image block of different views on the same time axis, the motion vector of the target block is predicted using the motion vector of the anchor block (708). That is, the motion vector of the target block is predicted using the motion vector of the anchor block calculated in the process "704".

As shown in FIG. 4, when the motion vector of the anchor block refers to a bidirectional image, that is, when the anchor block has two motion vectors, the target block to be encoded is a target block by combining the motion vectors of the two anchor blocks. Predict the motion vector This is because, like the anchor block, the target block may have a reference picture up and down.

5 and 6, when the motion vector of the anchor block refers to one direction image, that is, when the anchor block has one motion vector or the reference picture is a P picture, the target to be encoded A block can have one vector. In this case, since the anchor block has only one motion vector, the motion vector of the anchor picture is designated as the motion vector of the target block.

As a result of the check 706, when the reference block is an image block of the same view on the same time axis, the motion vector of the target block is predicted using a temporal correlation (for example, a motion vector correlation in the time direction) (710). ). That is, when the anchor block refers to the image of the same view, the conventional time direct prediction method is applied.

On the other hand, when the check result 706, the reference block corresponds to both the anchor block and the image block of the different view and the image block of the same view, it is possible to predict the motion vector of the target block using the temporal correlation that is the process "710". . In addition, when the check result 706 corresponds to both the image block and the image block of the same view, the motion vector of the target block may be predicted using the motion vector of the anchor block, which is a process "708".

The target block is encoded using the motion vector of the target block predicted in the process 708 and 710.

8 is a structural diagram of an embodiment of a picture group in a time direct prediction method applied to the present invention.

The picture group shown in FIG. 8 uses a scalable video coding scheme and aligns and encodes an image using an alignment program. The picture group structure is the same as the structure shown in Fig. 8, and the picture group length is set to 12. The vertical axis of the picture group structure used in the multi-view video encoding refers to the camera axis, and the horizontal axis of the picture group structure refers to the time axis.

The shaded portions B33 to B43, B57 to B67, and B81 to B91 do not use the spatial direct prediction mode for comparison with the spatial direct prediction mode provided by the Joint Scalable Video Model (JSVM). This is a region to which a temporal direct prediction method for multi-view coding according to the present invention is applied. The unenclosed part is the same coded area as the conventional JSVM. That is, the image portions of cameras C1, C3, and C5 are regions to which the time direct prediction method for multi-view encoding according to the present invention is applied.

In this case, in the case of a frame such as a B12 or B13 frame or a B14 or B15 frame, the reference picture exists in the horizontal direction. That is, in the case of the B12 frame, the reference pictures are I0 and B10 frames, and in the case of the B27 frame, the reference pictures are B22 and B21 frames. In this case, since the motion vector is located on the picture in the same camera, the conventional time direct prediction mode (for example, the time direct prediction mode of H.264 / AVC) can be used as it is.

9A to 9C are comparison diagrams of a signal-to-noise ratio according to peak power measured according to a temporal direct prediction method and a conventional spatial direct prediction method for multi-view video encoding according to the present invention.

In order to compare the peak signal to noise ratio (PSNR) according to the peak power measured according to the direct time prediction method according to the present invention and the conventional spatial direct prediction method, an 'Exit' image, a 'Ballroom' image, ' Park_run 'image was used as an experimental image. The signal-to-noise ratio according to the peak power of each image is shown in sequence in FIGS. 9A to 9C. Comparing the results of the present invention with the conventional spatial direct prediction method, it can be seen that the difference in the signal-to-noise ratio according to the peak power is not large. Experimental images 'Exit' image, 'Ballroom' image, and 'Park_run' image are suitable for the spatial direct prediction method. In the future, when the time direct prediction method according to the present invention is applied to an image suitable for the time direct prediction mode, it is expected to exhibit better performance.

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form. Since this process can be easily implemented by those skilled in the art will not be described in more detail.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

As described above, when the anchor blocks on the same time axis refer to the multi-view image, the motion vector of the target block is predicted by using the motion vector of the anchor block. By using the prediction, it is possible to accurately predict the motion information of the current target block in the multiview image.

For example, the present invention can improve the compression efficiency because the accuracy of the motion information prediction of the current block is improved for an image having a relatively constant speed such as a panning image in an image including different movements. It has an effect.

Claims (8)

In the temporal direct prediction method for multi-view video encoding, A reference block checking step of checking whether a block (hereinafter referred to as a "reference block") referenced by the anchor block for the target block to be encoded is an image block of another view (View) on the same time axis as the anchor block; And The multi-view prediction step of predicting the motion vector of the target block by using the motion vector of the anchor block if the reference block is an image block of a different view. A direct temporal prediction method for multiview video encoding comprising a. The method of claim 1, As a result of the checking, if the reference block is an image block of the same view as the anchor block, the same view prediction step of predicting a motion vector of the target block using temporal correlation The direct temporal prediction method for multi-view video encoding further comprising. The method of claim 1, As a result of the checking, when one of the reference blocks is an image block at a different viewpoint than the anchor block, and the other is an image block at the same viewpoint, a first vector predicting the motion vector of the target block using the motion vector of the anchor block is performed. Motion vector prediction step The direct temporal prediction method for multi-view video encoding further comprising. The method of claim 1, As a result of the checking, if one of the reference blocks is an image block at a different viewpoint than the anchor block, and the other is an image block at the same viewpoint, a second motion vector prediction predicting a motion vector of the target block using temporal correlation. step The direct temporal prediction method for multi-view video encoding further comprising. The method according to any one of claims 1 to 4, The reference block check step, The method of claim 1, wherein the closest video block among the video blocks that can be referenced by the target block is defined as an anchor block. The method of claim 5, wherein The multi-view prediction step, And a video block located at the shortest distance from the anchor block among the image blocks on the same time axis as the anchor block as the reference block. The method of claim 6, The multi-view prediction step, If the motion vector of the anchor block is used, and if there is only one motion vector of the anchor block, a time direct prediction method for multi-view video encoding comprising designating the motion vector of the anchor block as the motion vector of the target block. . The method of claim 6, The multi-view prediction step, Multi-view video encoding is performed by using the motion vector of the anchor block, and predicting the motion vector of the target block by using an average value of the motion vectors of the anchor block if there are a plurality of motion vectors of the anchor block. Time for direct prediction method.

Images