KR20070116527A - A method and apparatus for decoding/encoding a video signal - Google Patents

Abstract

A method and an apparatus for decoding/encoding a video signal are provided to define flag information indicating whether to perform an inter-view weighted prediction. A weighted prediction for decoding a current block by using information about a picture located in a view different from a current picture is performed according to a slice type extracted from a video signal(S310,S320). The slice type includes a macro block to which an inter-view prediction is applied.

Description

A method and apparatus for decoding / encoding a video signal

1 illustrates a multi-view sequence encoding and decoding system to which the present invention is applied.

2 shows a prediction structure between pictures in multi-view video coding according to the present invention.

3 is a flowchart of performing weight prediction according to slice type in video signal coding to which the present invention is applied.

4 illustrates an embodiment of macroblock types allowed in a slice type in video signal coding to which the present invention is applied.

5A to 5B illustrate syntaxes for performing weight prediction according to a newly defined slice type according to an embodiment to which the present invention is applied.

6 illustrates a flowchart of performing weight prediction using flag information indicating whether to perform inter-view weight prediction in video signal coding according to the present invention.

FIG. 7 is a diagram for describing a weight prediction method based on flag information indicating whether weight prediction is to be performed using information of a picture that is different from a current picture, according to an embodiment to which the present invention is applied.

FIG. 8 illustrates a syntax for performing weight prediction according to newly defined flag information according to an embodiment to which the present invention is applied.

FIG. 9 is a flowchart to which weight prediction is performed according to a network abstraction layer (NAL) unit type according to an embodiment to which the present invention is applied.

10A to 10B illustrate syntaxes for performing weight prediction when the NAL unit type is the NAL unit type for multiview video coding according to an embodiment to which the present invention is applied.

The present invention relates to a method and apparatus for decoding / encoding a video signal.

The mainstream video broadcasting image is a single view image acquired with one camera. Even if the video is taken by multiple cameras, it is edited and treated as a single video. Multi-view video, on the other hand, is a three-dimensional (3D) image processing method that geometrically corrects images taken by more than one camera and provides users with various viewpoints in various directions through spatial synthesis. It is a field. Multi-view video can increase the freedom of view for the user, and has a feature that includes a larger area than the image area that can be acquired using a single camera.

Since such multi-view video images have a high correlation between viewpoints, prediction between viewpoints can be efficiently performed by removing redundant information through spatial prediction between viewpoints. However, since each view sequence of a multi-view video is obtained from different cameras, illumination differences may occur due to internal and external factors of the camera. For example, a camera heterogeneity, a difference in camera calibration, or a camera alignment is caused. Since such illumination differences significantly reduce the correlation between different views, thereby inhibiting efficient coding, a weighted prediction coding technique is required to compensate for this.

An object of the present invention is to define a slice type to which the inter-view prediction is applied, and to provide a weight prediction method accordingly.

Another object of the present invention is to define flag information indicating whether to perform inter-view weight prediction, and to provide a weight prediction method accordingly.

Another object of the present invention is to provide a weight prediction method according to a NAL unit type for multiview video coding.

It is another object of the present invention to efficiently compensate for lighting differences between viewpoints in a multiview image.

It is still another object of the present invention to improve the coding efficiency of a video image by effectively using the correlation between viewpoints.

In order to achieve the above object, the present invention provides a method for decoding a video signal, comprising: a weight for decoding a current block by using information about a picture at a different time point than a current picture according to a slice type extracted from the video signal. It provides a video signal decoding method comprising the step of performing the prediction.

The present invention also provides a method of extracting NAL unit type information from a video signal, and when the NAL unit type is a NAL unit type for multi-view video coding, the current block using information on a picture that is different from the current picture. It provides a video signal decoding method comprising the step of performing a weighted prediction for decoding.

The above objects and features of the construction will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In addition, the terms used in the present invention was selected as a general term widely used as possible now, but in some cases, the term is arbitrarily selected by the applicant, in which case the meaning is described in detail in the description of the invention, It is to be understood that the present invention is to be understood as the meaning of terms rather than the names of terms.

1 illustrates a multi-view sequence encoding and decoding system to which the present invention is applied. As shown in FIG. 1, the multiview image encoding system to which the present invention is applied includes a multiview image generator 10, a preprocessing part 20, and an encoding part 30. . In addition, the decoding system includes an extractor 40, an extractor 50, a decoding part 50, a post processing part 60, and a display part 70.

The multi-view image generator 10 includes an image obtaining apparatus (for example, cameras # 1 to #N) corresponding to the number of multi-views to acquire independent images for each viewpoint. When the multiview image data is input, the preprocessing unit 20 performs a function of increasing the correlation between the multiview image data through a preprocessing process while solving noise removal and imbalancing problems. The encoder 30 includes a motion estimator, a motion compensator, and a disparity estimator, a disparity compensator, an illumination compensator, a bit rate control, and a residual image encoder. do. The encoding unit 30 may apply a generally known method.

The extractor 40 functions to extract only a bitstream corresponding to a desired view from the transmitted MVC bitstream. The extractor 40 may view the header and selectively decode only the desired time point. In addition, view scalability may be implemented by extracting only a bitstream corresponding to a desired view using a view identifier that distinguishes a view of a picture. Since MVC must be fully compatible with H.264 / AVC, it is necessary to decode only a specific point in time that is compatible with H.264 / AVC. In this case, a view identifier that distinguishes a view of the picture may be used to decode only compatible views. The bitstream extracted through the extractor 40 is transmitted to the decoding unit 50. The decoder 50 includes a motion compensator, an illumination compensator, a weight predictor, a deblocking filter, and the like. The decoding unit 50 receives the bitstream encoded by the above-described method, and decodes it in reverse. In addition, the post-processing unit 60 performs a function of increasing the reliability and resolution of the decoded data. Finally, the display unit 70 provides the decoded data to the user in various ways depending on the function of the display, in particular, the ability to process a multi-view image. For example, it is a 2D display 71 that provides only a planar two-dimensional image, a stereo type display 73 that provides two views as a stereoscopic image, or a stereoscopic image of M views (2 <M). The display 75 may be provided.

2 shows a prediction structure between pictures in multi-view video coding according to the present invention.

As shown in FIG. 2, T0 to T100 on the horizontal axis represent frames according to time, and S0 to S7 on the vertical axis represent frames according to viewpoints, respectively. For example, pictures in T0 refer to images taken by different cameras in the same time zone (T0), and pictures in S0 refer to images in different time zones taken by one camera. In addition, the arrows in the drawing indicate the prediction direction and the order of each picture. For example, a P0 picture at S2 time point in the T0 time zone is a picture predicted from I0, and the P0 picture is also at S4 time point in the TO time zone. It becomes a reference picture of another P0 picture. It is also a reference picture of the B1 and B2 pictures in the T4 and T2 time zones at the time S2.

As described above, when the prediction coding is performed using information on a picture of a picture at a different point in time from the current picture, a weight prediction method according to the present invention to compensate for an illumination difference between the current picture and a picture at another point in time. This can be applied. Even when the weight prediction method is applied, an identifier for distinguishing a viewpoint between the pictures may be needed.

3 is a flowchart of performing weight prediction according to slice type in video signal coding to which the present invention is applied.

Weighted prediction is a method of scaling a sample of motion compensated prediction data within a P or B slice macroblock. The weight prediction method is an explicit mode in which weight prediction is performed on a current picture using weight coefficient information obtained from information on reference pictures or a weight coefficient obtained from distance information between a current picture and reference pictures. Implicit mode for performing weight prediction for the current picture using the information. This weight prediction method may be applied differently according to the slice type of the macroblock to be applied. For example, in the explicit mode, the weight coefficient information may vary depending on whether the current macroblock on which the weight prediction is performed is a macroblock of a P slice or a macroblock of a B slice. And, the weight coefficient in the explicit mode may be determined by the encoder and included in the slice header and transmitted. On the other hand, the weighting coefficient in the implicit mode may be obtained based on the relative temporal position of the List 0 and List 1 reference pictures. For example, if the reference picture is close to the current picture in time, a large weighting factor may be applied. If the reference picture is far from the current picture in time, a small weighting factor may be applied. Therefore, in the present invention, first, the slice type of the macroblock to which weight prediction is to be applied is extracted from the video signal (S310). According to the extracted slice type, weight prediction may be performed on the current macroblock (S320). Here, the slice type may include a macroblock to which inter-view prediction is applied. Inter-view prediction refers to predicting a current picture by using information of a picture located at a different viewpoint than the current picture. For example, the slice type may be a macroblock to which temporal prediction is applied using the information of a picture at the same point in time as the current picture, a macroblock to which the inter-view prediction is applied, and a temporal prediction to a viewpoint. The prediction may include macroblocks applied together. In addition, the slice type may include only macroblocks to which temporal prediction is applied, may include only macroblocks to which inter-prediction is applied, may include only macroblocks to which both predictions are applied, and the macroblock type. It can also include either or both types. This will be described in detail with reference to FIG. 4. As such, when the slice type including the macroblock to which the inter-view prediction is applied is extracted from the video signal, the weight prediction is performed by using information about a picture that is different from the current picture. Here, a viewpoint identifier for distinguishing a viewpoint of a picture may be used to use information about a picture at another viewpoint.

4 illustrates an embodiment of macroblock types allowed in a slice type in video signal coding to which the present invention is applied.

As shown in FIG. 4, when defining the P slice type by the inter-view prediction as VP (View_P), the P slice type by the inter-view prediction includes an intra macroblock (I) and a current view (current view). A macroblock (P) predicted from one picture in) or a macroblock (VP) predicted from one picture in a different view is allowed (410). When the B slice type based on the inter-view prediction is defined as VB (View_B), the B slice type based on the inter-view prediction includes an intra macroblock (I) and a macroblock predicted from at least one picture present at the current view. A macroblock (VP or VB) predicted from a picture at (P or B) or at least one or more other views is allowed (420). In addition, when the slice type predicted using each or both of the temporal prediction and the inter-view prediction is defined as Mixed, the mixed slice type includes an intra macroblock (I) and a macro predicted from at least one or more pictures at the current view. A macroblock (VP or VB) predicted from a block (P or B) or a picture at least one or more other views, or a macroblock predicted using both a picture at a current time and a picture at a different time is allowed. 430. In this case, in order to use a picture at another view, a view identifier that distinguishes a view of the picture may be used.

5A to 5B illustrate syntaxes for performing weight prediction according to a newly defined slice type according to an embodiment to which the present invention is applied.

As shown in FIG. 4, when the slice type is defined as VP, VB, or Mixed, the syntax for performing weight prediction of the existing (eg, H.264) may be changed as shown in FIGS. 5A to 5B. . For example, if the slice type is a P slice based on temporal prediction, the portion “if (slice_type! = VP ∥ slice_type! = VB)” is added (510), and when the slice type is a B slice based on temporal prediction The if statement can be changed to "if (slice_type == B | slice_type == Mixed)" (520). In addition, the VP and VB slice types are newly defined, and thus may be newly added in a format similar to that of FIG. 5A (530, 540). In this case, since the information about the viewpoint is added, the syntax elements include a "view" portion. Examples thereof include "luma_log2_view_weight_denom, chroma_log2_view_weight_denom", and the like.

6 illustrates a flowchart of performing weight prediction using flag information indicating whether to perform inter-view weight prediction in video signal coding according to the present invention.

In video signal coding to which the present invention is applied, more efficient coding becomes possible when using flag information indicating whether to perform weight prediction. Such flag information may be defined based on the slice type. For example, there may be flag information indicating whether the weight prediction is applied to the P slice and the SP slice, and there may be flag information indicating whether the weight prediction is to be applied to the B slice. As a specific example, the flag information may be defined as "weighted_pred_flag" and "weighted_bipred_idc", respectively. Weighted_pred_flag = 0 indicates that the weight prediction is not applied to the P slice and the SP slice, and weighted_pred_flag = 1 indicates that the weight prediction is applied to the P slice and the SP slice. And weighted_bipred_idc = 0 indicates that the default weighted prediction is applied to the B slice, weighted_bipred_idc = 1 indicates that the explicit weighted prediction is applied to the B slice, and weighted_bipred_idc = 2 implicit weights. Indicates that the prediction is applied to the B slice. In multi-view video coding, flag information indicating whether weight prediction is performed using information on pictures between views may be defined based on a slice type.

First, flag information indicating whether slice type and inter-view weight prediction is performed is extracted from the video signal (S610 and S620). In this case, the slice type may be, for example, a prediction using a macroblock to which a temporal prediction is performed using information of a picture at the same point in time as the current picture, and information of a picture at a different point in time from the current picture. It may include a macroblock to which the inter-view prediction is performed. A weight prediction mode may be determined based on the extracted slice type and flag information (S630). Weight prediction may be performed according to the determined weight prediction mode (S640). Here, the flag information may include flag information indicating whether to perform weight prediction using information of a picture that is different from the current picture in addition to the "weighted_pred_flag" and "weighted_bipred_idc" described above. This will be described in detail with reference to FIG. 7. As such, when the slice type of the current macroblock is a slice type including a macroblock to which inter-view prediction is applied, it is more efficient to use flag information indicating whether to perform weighted prediction using information on pictures of different views. Coding is possible.

FIG. 7 is a diagram for describing a weight prediction method based on flag information indicating whether to perform weight prediction using information on a picture that is different from a current picture, according to an embodiment to which the present invention is applied.

For example, flag information indicating whether to perform weight prediction using information of a picture that is different from the current picture may be defined as "view_weighted_pred_flag" and "view_weighted_bipred_idc". View_weighted_pred_flag = 0 indicates that weighted prediction is not applied to the VP slice, and view_weighted_pred_flag = 1 indicates that explicit weighted prediction is applied to the VP slice. If view_weighted_bipred_idc = 0, this indicates that a default weighted prediction is applied to the VB slice, and if view_weighted_bipred_idc = 1, it indicates that an explicit weighted prediction is applied to the VB slice. Applies to the slice. When implicit weight prediction is applied to a VB slice, the weight coefficient may be obtained by the relative distance between the current time point and another time point. In addition, when the implicit weight prediction is applied to the VB slice, the weight prediction may be performed by using a view identifier that distinguishes the viewpoints of the pictures, or a picture output order created by considering each viewpoint. , POC may be used to perform weight prediction. In addition, the flag information may be included in a picture parameter set (PPS). Here, the picture parameter set refers to header information indicating an encoding mode (for example, an entropy encoding mode, a quantization parameter initial value in picture units, etc.) of the entire picture. However, the picture parameter set is not attached to all the pictures. If there is no picture parameter set, the picture parameter set immediately existing is used as header information.

FIG. 8 illustrates a syntax for performing weight prediction according to newly defined flag information according to an embodiment to which the present invention is applied.

In multi-view video coding to which the present invention is applied, flag information indicating whether to perform weight prediction using a slice type including a macroblock to which inter-view prediction is applied and information of a picture that is different from the current picture is defined. If necessary, it is necessary to determine which weight prediction to perform according to which slice type. For example, as shown in FIG. 8, when the slice type extracted from the video signal is a P slice or an SP slice, weighted_pred_flag = 1 may be used for weight prediction, and when the slice type is a B slice, weighted_bipred_idc = 1. Weight prediction may be performed. In addition, when the slice type is a VP slice, weight prediction may be performed when view_weighted_pred_flag = 1, and when the slice type is VB slice, weight prediction may be performed when view_weighted_bipred_idc = 1.

FIG. 9 is a flowchart to which weight prediction is performed according to a network abstraction layer (NAL) unit type according to an embodiment to which the present invention is applied.

First, a NAL unit type (nal_unit_type) is extracted from a video signal (S910). Here, the NAL unit type means an identifier indicating the type of the NAL unit. For example, when nal_unit_type = 5, it indicates that the NAL unit is a slice of an IDR picture. An IDR (Instantaneous Decoding Refresh) picture refers to the first picture of the video sequence. Then, it is checked whether the extracted NAL unit type is a NAL unit type for multi-view video coding (S920). When the NAL unit type is a NAL unit type for multiview video coding, weight prediction is performed using information on a picture that is at a different time point than the current picture (S930). The NAL unit type may be a NAL unit type applicable to both scalable video coding and multiview video coding, or may be a NAL unit type for multiview video coding only. As such, in the case of the NAL unit type for multi-view video coding, since the weight prediction can be performed using information of a picture that is different from the current picture, a syntax needs to be newly defined. Hereinafter, the detailed description will be made with reference to FIGS. 10A to 10B.

10A to 10B illustrate syntaxes for performing weight prediction when the NAL unit type is the NAL unit type for multiview video coding according to an embodiment to which the present invention is applied.

When the NAL unit type is the NAL unit type for multi-view video coding, the syntax for performing weight prediction of the existing (eg, H.264) may be changed as shown in FIGS. 10A to 10B. For example, the 1010 portion of FIG. 10A corresponds to the syntax portion for performing the existing weight prediction, and the 1020 portion of FIG. 10A corresponds to the syntax portion for performing the weight prediction in multiview video coding. Accordingly, in 1020, weight prediction is performed only when the NAL unit type is the NAL unit type for multi-view video coding. In this case, since the information about the viewpoint is added, the syntax elements include a "view" portion. Examples thereof include "luma_view_log2_weight_denom, chroma_view_log2_weight_denom", and the like. In addition, part 1030 of FIG. 10B corresponds to a syntax part for performing weighted prediction, and part 1040 of FIG. 10B corresponds to a syntax part for performing weighted prediction in multiview video coding. Accordingly, in 1040, weight prediction is performed only when the NAL unit type is the NAL unit type for multiview video coding. In this case as well, since the information about the viewpoint is added, the syntax elements include a "view" portion. Examples thereof include "luma_view_weight_l1_flag, chroma_view_weight_l1_flag", and the like. As such, when a NAL unit type for multi-view video coding is defined, more efficient coding may be performed by performing weight prediction using information of a picture that is different from the current picture.

Each viewpoint image of the multi-view video is caused by a difference in illumination due to internal and external factors of the camera. This illumination difference significantly reduces the correlation between different viewpoints, which hinders effective coding. Therefore, in the present invention, flag information indicating whether the slice type including the macroblock to which the inter-view prediction is applied or whether to perform the inter-view weight prediction is newly defined and weighted prediction is performed using the same to enable more efficient coding. In addition, more efficient coding may be performed by defining a NAL unit type for multi-view video coding and performing weight prediction using the same.

Claims (7)

In the method for decoding a video signal, And performing weighted prediction for decoding the current block by using information about a picture that is different from the current picture according to the slice type extracted from the video signal. The method of claim 1, And the slice type comprises a macroblock to which inter-view prediction has been applied. The method of claim 1, wherein the video signal decoding method comprises: Extracting flag information indicating whether the inter-view weight prediction is performed; And performing weighted prediction according to the weighted prediction mode determined based on the slice type and the flag information. The method of claim 3, wherein And the weight prediction mode is a mode for performing weight prediction using coefficient information obtained from information about a picture that is different from the current picture. The method of claim 3, wherein And the weight prediction mode is a mode for performing weight prediction using a distance between the current picture and a reference picture at a different point in time. The method of claim 5, The distance between the current picture and a reference picture that is different from the current picture may be determined by using a view identifier indicating the view of the picture or by using output order display information of the picture that may distinguish the view of the picture. Video signal decoding method characterized in that it is obtained. Extracting a NAL unit type from the video signal; If the NAL unit type is a NAL unit type for multi-view video coding, performing weight prediction to decode the current block by using information about a picture that is different from the current picture. Video signal decoding method comprising a.

Images