KR101366289B1 - A method and apparatus for decoding/encoding a video signal - Google Patents

Abstract

The present invention provides a method of obtaining first flag information that distinguishes a coding scheme of a current NAL from a header of a video signal. Acquiring second flag information indicating; acquiring information for inter-view prediction based on the second flag information; and decoding the video signal using the information for inter-view prediction, The information for inter-view prediction provides a video signal decoding method comprising information indicating whether illumination compensation is applied in a current slice and inter-view variation vector information.

Multi-view, point-in-time prediction

Description

A method and device for decoding / encoding a video signal {A METHOD AND APPARATUS FOR DECODING / ENCODING A VIDEO SIGNAL}

The present invention relates to the coding of video signals.

Compression coding refers to a series of signal processing techniques that transmit digitized information through a communication line or store the data in a form suitable for a storage medium. The target of compression coding includes objects such as voice, image, and character. In particular, the technique of performing compression coding on an image is referred to as video image compression. The general feature of the video image is that it has spatial redundancy and temporal redundancy.

It is intended to efficiently code a video signal by defining viewpoint information that can identify a viewpoint of a picture.

It is intended to efficiently code a video signal by defining an interview prediction flag that indicates whether the coded picture of the current NAL unit is used for inter-view prediction.

It is intended to efficiently code a video signal by generating and managing a reference picture list using interview reference information indicating a dependency between viewpoints.

Random access between viewpoints can be efficiently performed by obtaining interview reference information based on the interview picture group identification information.

It is intended to improve coding efficiency by placing attribute information of a multiview video image in an appropriate place.

We want to set the level in various ways from time to time by telling whether the current slice corresponds to the reference time point.

The present invention provides a method of obtaining first flag information that distinguishes a coding scheme of a current NAL from a header of a video signal, and when multiview video coding is performed according to the first flag information, whether the current NAL is used for inter-view prediction. Acquiring second flag information indicating; acquiring information for inter-view prediction based on the second flag information; and decoding the video signal using the information for inter-view prediction, The information for inter-view prediction provides a video signal decoding method comprising information indicating whether illumination compensation is applied in a current slice and inter-view variation vector information.

The present invention may further include acquiring interview picture group identification information indicating whether a coded picture of a current NAL is an interview picture group, and acquiring the interview reference information according to the identification information. It provides a video signal decoding method characterized by decoding the video signal using the reference information.

The present invention may further include obtaining reference time flag information indicating whether the current NAL unit corresponds to a reference time point, and performing inter-view prediction when the current NAL unit corresponds to a reference time point according to the reference time flag information. And a method for decoding the video signal using the marked reference picture.

The present invention also provides a method of encoding a multiview video signal, the method comprising: generating interview reference information indicating a dependency relationship between viewpoints, determining priority information for each viewpoint according to the interview reference information,

Generating reference time flag information indicating whether a current NAL unit corresponds to a reference time point and generating a bitstream including the reference time flag information based on the priority information. A video signal encoding method is provided.

The present invention also provides a parser for acquiring second flag information indicating whether the current NAL is used for inter-view prediction according to first flag information for distinguishing a coding scheme of a current NAL. And an inter predictor to decode the video signal using information obtained for inter-view prediction based on the information, wherein the information for inter-view prediction includes information indicating whether illumination compensation is applied in a current slice and an inter-view variation vector. Provided is a video signal decoding apparatus comprising information.

According to the present invention, in coding a video signal, coding can be performed more efficiently by placing coding information at an appropriate place. In addition, by checking whether the current picture is used for inter-view prediction first, and transmitting information necessary for inter-view prediction only when it is used for inter-view prediction, the number of bits to be transmitted can be reduced and the decoded picture buffer By reducing the burden, coding speed can be improved. In addition, various levels between views may be set by indicating whether the current slice corresponds to a reference viewpoint. In addition, more efficient coding may be possible by using various attribute information of a multiview image.

The compression coding technique of video signal data considers spatial redundancy, temporal redundancy, scalable redundancy, and redundancy existing between views. In this compression encoding process, compression coding may be performed in consideration of mutual redundancy existing between views. The compression coding technique considering the inter-view redundancy is only an embodiment of the present invention, and the technical idea of the present invention may be applied to temporal redundancy, scalable redundancy, and the like. In addition, the term coding in this specification may include both the concepts of encoding and decoding, and may be flexibly interpreted according to the technical spirit and technical scope of the present invention.

1 is a schematic block diagram of a video signal decoding apparatus to which the present invention is applied.

The decoding apparatus includes a parsing unit 100, an entropy decoding unit 200, an inverse quantization / inverse transform unit 300, an intra prediction unit 400, a deblocking filter unit 500, a decoded picture buffer unit 600, Inter prediction unit 700 and the like.

 The parser 100 performs parsing on a network abstraction layer (NAL) basis in order to decode the received video image. In general, one or more sequence parameter sets and picture parameter sets are transmitted to the decoder before the slice header and slice data are decoded. In this case, various attribute information may be included in the NAL header area or the extension area of the NAL header. Multiview Video Coding (MVC) is an additional technology to the existing AVC technology, so it may be more efficient to add various attribute information only in the case of MVC bitstream rather than unconditionally adding. For example, flag information for identifying whether an MVC bitstream is included in the NAL header region or an extension region of the NAL header may be added. Attribute information about a multiview image may be added only when the input bitstream is a multiview image coded bitstream according to the flag information. For example, the attribute information may include view identification information, inter-view picture group identification information, inter-view prediction flag information, and reference view flag information. view flag information), and the like. This will be described in detail in FIG.

The parsed bitstream is entropy decoded through the entropy decoding unit 200, and coefficients, motion vectors, and the like of each macroblock are extracted. The inverse quantization / inverse transform unit 300 multiplies the received quantized value by a constant constant to obtain a transformed coefficient value, and inversely transforms the coefficient value to restore the pixel value. The intra prediction unit 400 performs intra prediction from the decoded samples in the current picture by using the reconstructed pixel value. Meanwhile, the deblocking filter unit 500 is applied to each coded macroblock in order to reduce block distortion. The filter smoothes the edges of the block to improve the quality of the decoded frame. The choice of filtering process depends on the boundary strength and the gradient of the image samples around the boundary. The filtered pictures are output or stored in the decoded picture buffer unit 600 for use as a reference picture.

The decoded picture buffer unit 600 stores or opens previously coded pictures in order to perform inter prediction. In this case, in order to store or open the decoded picture buffer unit 600, frame_num and POC (Picture Order Count) of each picture are used. Therefore, some of the previously coded pictures in MVC have pictures that are different from the current picture. Therefore, in order to utilize these pictures as reference pictures, not only the frame_num and the POC but also the view information for identifying the view point of the picture are included. It is available. In addition, information identifying a picture in a view direction having a concept similar to the frame_num may also be used.

The inter prediction unit 700 performs inter prediction using a reference picture stored in the decoded picture buffer unit 600. Inter-coded macroblocks can be divided into macroblock partitions, where each macroblock partition can be predicted from one or two reference pictures. The inter prediction unit 700 compensates for the movement of the current block by using the information transmitted from the entropy decoding unit 200. A motion vector of blocks neighboring the current block is extracted from the video signal, and a motion vector prediction value of the current block is obtained. The motion of the current block is compensated by using the obtained motion vector prediction value and the difference vector extracted from the video signal. In addition, such motion compensation may be performed using one reference picture or may be performed using a plurality of pictures. In multi-view video coding, when a current picture refers to pictures at different views, motion compensation is performed using information on a reference picture list for inter-view prediction stored in the decoded picture buffer unit 600. Can be done. In addition, motion compensation may be performed using viewpoint identification information for identifying the viewpoint of the picture.

Through the above process, the inter predicted pictures and the intra predicted pictures are selected according to the prediction mode to reconstruct the current picture. Hereinafter, various embodiments for providing an efficient decoding method of a video signal will be described.

FIG. 2 is an embodiment to which the present invention is applied and shows a structure of a NAL unit for describing attribute information for a multiview video image that may be added to a multiview video coded bitstream.

2 illustrates an example of a structure of a NAL unit to which attribute information of a multiview video image may be added. In general, the NAL unit may include a header of the NAL unit and a raw byte sequence payload (RBSP). The header of the NAL unit may include identification information nal_ref_idc indicating whether the NAL unit includes a slice of the reference picture and information nal_unit_type indicating the type of the NAL unit. In addition, the extended region of the NAL unit header may be limited. For example, when the information indicating the type of the NAL unit is related to multi-view video coding, the NAL unit may include an extended area of the NAL unit header. As a specific example, when nal_unit_type = 20 or 14, the NAL unit may also include an extended region of the NAL unit header. In addition, the extended region of the NAL unit header may include flag information svc_mvc_flag for distinguishing a coding scheme of a current NAL unit. For example, it may be identified whether the current NAL unit is a multiview video coded bitstream or a scalable video coded bitstream according to the flag information. In the case of a multiview video coded bitstream according to the flag information, attribute information of a multiview video image may be added.

First, view identification information refers to information for distinguishing a picture at a current view from a picture at a different view. When a video image signal is coded, a picture order count (POC) and frame_num are used to identify each picture. In the case of a multiview video image, since inter-view prediction is performed, identification information for distinguishing a picture at a current view from a picture at a different view is required. Therefore, it is necessary to define viewpoint identification information for identifying the viewpoint of the picture. The view identification information may be obtained from a header area of a video signal. For example, the header area may be an NAL header, an extended area of the NAL header, or a slice header. The view identification information may be used to obtain information of a picture that is different from the current picture, and the video signal may be decoded using the information of the picture at the other view.

Inter-view picture group identification information refers to information for identifying whether a coded picture of a current NAL unit is an interview picture group. The inter-view picture group means an encoded picture in which all slices refer only to slices in frames of the same time zone. For example, an encoded picture refers to only a slice at another viewpoint and no slice at the current viewpoint. In the decoding process of a multiview image, random access between views may be possible. As such, when the interview reference information is acquired based on the interview picture group identification information, random access between viewpoints can be performed more efficiently. This is because the inter-view reference relationship between the pictures in the interview picture group may be different from the inter-view reference relationship in the non-interview picture group. In this case, if the view identification information is used, more efficient random access may be possible.

Inter-view prediction flag information refers to information indicating whether a coded picture of a current NAL unit is used for inter-view prediction. The interview prediction flag information may be used in a part where temporal prediction or inter-view prediction is performed. In this case, the NAL unit may be used together with identification information indicating whether the NAL unit includes a slice of the reference picture. For example, when the current NAL unit does not include a slice of a reference picture according to the identification information but is used for inter-view prediction, the current NAL unit may be a reference picture used only for inter-view prediction. Alternatively, when the current NAL unit includes a slice of a reference picture according to the identification information and is used for inter-view prediction, the current NAL unit may be used for temporal prediction and inter-view prediction. In addition, even if the NAL unit does not include a slice of the reference picture according to the identification information, it may be stored in the decoded picture buffer. This is because, if the coded picture of the current NAL unit is used for inter-view prediction according to the interview prediction flag information, it needs to be stored. This will be described in detail with reference to FIG. 3.

The base view flag information may mean whether the current slice corresponds to the base view. For example, when the reference time flag is 1, the current slice corresponds to the reference time point. When the reference time flag is used, levels between time points can be set in more various ways. That is, when the reference view flag is used, a view level of 0 can be determined, and thus, the view level of another view can be defined more variously. This will be described in detail in FIG.

Priority identification information means information for identifying the priority of the NAL unit. This can be used to provide view scalability. For example, viewpoint level information may be defined using the priority identification information. Here, the view level information refers to information on a hierarchical structure for providing view expandability from a video signal. In a multi-view video image, it is necessary to define the levels of time and view in order to provide a user with images of various times and views. When defining the level information in this way, scalability with respect to time and time can be used. Accordingly, the user may view only an image of a desired time and time point, and may view only an image according to another constraint. This will be described in detail in FIG.

As another example, when the information indicating the type of the NAL unit is information indicating a subset sequence parameter set, the RBSP may include information about the subset sequence parameter set. As a specific example, when nal_unit_type = 15, the RBSP may include information about a subset sequence parameter set, information about a slice layer, and the like.

 In addition, the subset sequence parameter set may include an extended area of the sequence parameter set according to profile information. For example, when the profile information profile_idc is a profile related to multi-view video coding, the subset sequence parameter set may include an extended region of the sequence parameter set. Alternatively, the sequence parameter set may include an extended area of the sequence parameter set according to the profile information. The extended area of the sequence parameter set may include interview reference information and partial decoding information. Inter-view reference information may mean information used to indicate inter-view dependency. For example, in order to indicate the inter-view dependency, information about the number of interview reference pictures and viewpoint identification information on the interview reference picture may be required. The partial decoding information may mean information indicating that the picture of the corresponding reference view may not be fully decoded. This may apply in the case of a non-interview picture group. For example, for all non-interview picture groups at all time points, the pictures at the corresponding reference time points may be partially decoded instead of completely decoded. As a specific example, in the non-interview picture group, the intra macroblock may completely decode when necessary, and the inter macroblock may decode only residual information when necessary. Alternatively, macroblock type information and motion information may be decoded for the inter macroblock and used as motion skip information at different views.

The interview reference information refers to information capable of knowing how the inter-view images are predicted. The inter-view reference information may be determined using the number of reference pictures and the view information of the reference picture. For example, first, the number of all viewpoints may be obtained, and viewpoint identification information for distinguishing each viewpoint may be determined based on the number of all viewpoints. In addition, information about the number of interview reference pictures indicating the number of reference pictures with respect to the reference direction may be obtained at each viewpoint. View identification information of each interview reference picture may be acquired according to the number information of the interview reference picture. In this manner, the interview reference information may be obtained, and the interview reference information may be identified by being divided into an interview picture group and a non-interview picture group. This can be known using the interview picture group identification information indicating whether the coded slice currently in the NAL is an interview picture group. In addition, the interview reference information acquired according to the interview picture group identification information may be used to generate and modify a reference picture list.

3 is an embodiment to which the present invention is applied and shows an overall prediction structure of a multiview image signal for explaining a decoding process using an interview prediction flag.

As shown in FIG. 3, T0 to T100 on the horizontal axis represent frames over time, and S0 to S7 on the vertical axis represent frames according to viewpoints, respectively. For example, pictures in T0 refer to images taken by different cameras in the same time zone (T0), and pictures in S0 refer to images in different time zones taken by one camera. In addition, the arrows in the drawings indicate the prediction direction and the order of each picture. For example, a P0 picture at S2 time point in the T0 time zone is a picture predicted from I0, which refers to a P0 picture at S4 time point in the TO time zone. It becomes a picture. It is also a reference picture of the B1 and B2 pictures in the T4 and T2 time zones at the time S2. As such, an interview reference picture is required to perform the inter-view prediction, and whether the current picture is used for the inter-view prediction may be determined by the interview prediction flag information. Hereinafter, an embodiment in which the interview prediction flag is used will be described.

In another embodiment of the present invention, inter-view prediction flag information may refer to flag information indicating whether inter-view prediction is used in decoding a coded slice. For example, when the flag information is 1, it may mean that no inter-view prediction is used in the current slice. That is, it may mean that coding mode prediction, motion prediction, and / or residual prediction regarding inter-view prediction are not used in the current slice. In addition, even when the flag information is 0, inter-view prediction may or may not be used according to a coding scheme in a macroblock. This is because inter-view prediction may or may not be used in macroblock units.

In addition, the slice layer may include various pieces of information for inter-view prediction. For example, the information may include information indicating whether illumination compensation is applied in the current slice, global variation vector information in units of macroblocks between the current slice and slices at different views. The information may be obtained from a slice header. Alternatively, the information may include information indicating whether motion skip between views is applied. Such various information may not be transmitted by confirming the interview prediction flag.

As such, in multi-view video coding, by checking whether inter-view prediction is used in the header of the current NAL unit, when the current NAL unit is not used for inter-view prediction, it is necessary to transmit coding information used for inter-view prediction. There is no need to increase the coding efficiency.

4 is an embodiment to which the present invention is applied and shows a hierarchical structure of level information for explaining a decoding process in which a reference view flag is used.

Level information of each viewpoint may be determined in consideration of inter-view reference information. For example, since I and P pictures cannot be decoded without an I picture, level = 0 can be assigned to a base view where the interview picture group is an I picture, and a level when the anchor picture is a P picture. = 1 may be assigned, and level = 2 may be assigned when the interview picture group is a B picture. However, level information may be arbitrarily determined according to specific criteria.

In addition, the level information may be arbitrarily determined according to a specific criterion, or may be randomly determined without the criterion. For example, when the level information is determined based on a viewpoint, the viewpoint of pictures predicted by using view level 0, pictures at one viewpoint by setting the viewpoint of V0 as a reference viewpoint, view level 1, and a plurality of views The view of the predicted pictures may be set to view level 2 using the pictures at the view. In this case, at least one viewpoint image for compatibility with an existing decoder (eg, H.264 / AVC, MPEG-2, MPEG-4, etc.) may be required. This reference time point is again a reference of encoding among points, which may correspond to a reference time point for prediction of another time point. In multi-view video coding, an image corresponding to a reference view may be encoded by a conventional general image encoding scheme (MPEG-2, MPEG-4, H.263, H.264, etc.) to form an independent bitstream. The image corresponding to the reference time point may or may not be compatible with H.264 / AVC. However, an image of a viewpoint compatible with H.264 / AVC becomes a reference viewpoint.

Referring to FIG. 4, the prediction is performed by using the pictures at the view point V4 and the view point V4, which are views of the pictures predicted by using the pictures at the view point V2 and the view point V2, which are predicted using the pictures at the view point V0. A view level 1 may be set as a view level 1 based on a V6 view, which is the view of the pictures, and a V7 view, which is the view of the pictures predicted using the pictures at the V6 view. In addition, the view point V1, which is the view point of the pictures predicted using the pictures located at the view point V0 and the view point V2, and the view point V3 and V5 that are predicted in the same manner, may be set to view level 2. Therefore, when the decoder of the user cannot see the multiview video image, only the images of the view corresponding to the view level 0 are decoded. Alternatively, even when the user's decoder is limited by the profile information, only the limited view level information can be decoded.

In another embodiment, priority identification information may be used as a method of setting levels between viewpoints. As shown in FIG. 2, priority identification information means information for identifying a priority of the NAL unit. This can be used to provide view scalability. For example, viewpoint level information may be defined using the priority identification information. When defining the level information in this way, scalability with respect to time and time can be used. The level information may be set differently in various ways according to the reference condition. For example, it may be set differently according to the position of the camera, or may be set differently according to the arrangement of the camera. In addition, the level information may be determined in consideration of the interview reference information. For example, the level is assigned to 0 when the interview picture group is an I picture, and the level is assigned to 1 when the interview picture group is a P picture. When the interview picture group is a B picture, the level may be assigned to 2. As such, the level value may be assigned to the priority information. In addition, the level information may be arbitrarily set regardless of a special criterion.

As another embodiment to which the present invention is applied, a level between viewpoints may be set using a reference viewpoint flag. For example, when the reference view flag is used, a view level of 0 can be determined, and thus the view level of another view can be defined more variously. In this case, the methods described above may be applied.

As another embodiment to which the present invention is applied, the level information may vary depending on the position of the camera. For example, viewpoints V0 and V1 are images acquired by a camera located in the front, views V2 and V3 are images acquired by a camera located at the rear, and views V4 and V5 are viewed by a camera located on the left. When the V6 and V7 views are acquired by the camera located on the right side, the V0 and V1 views are view level 0, the V2 and V3 views are view level 1, and the V4 views are obtained. And V5 may be set to view level 2, and V6 and V7 may be set to view level 3. As another embodiment, the level information may vary depending on the arrangement of the cameras. In addition, the level information may be arbitrarily determined without resorting to special criteria.

The decoding / encoding device to which the present invention is applied may be provided in a multimedia broadcasting transmitting / receiving device such as digital multimedia broadcasting (DMB), and may be used to decode video signals and data signals. Also, the multimedia broadcasting transmitting / receiving apparatus may include a mobile communication terminal.

In addition, the decoding / encoding method to which the present invention is applied may be stored in a computer-readable recording medium that is produced as a program for execution on a computer, and multimedia data having a data structure according to the present invention may also be read by a computer. It can be stored in the recording medium. The computer-readable recording medium includes all kinds of storage devices in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted using a wired / wireless communication network.

Or more, preferred embodiments of the present invention described above, for the purpose of illustration, those skilled in the art, within the technical spirit and the technical scope of the present invention disclosed in the appended claims below, to further improve various other embodiments Changes, substitutions or additions will be possible. It is also possible to provide a medium for storing data applying at least one feature according to the present invention.

1 is a schematic block diagram of a video signal decoding apparatus to which the present invention is applied.

FIG. 2 is an embodiment to which the present invention is applied and shows a structure of a NAL unit for describing attribute information for a multiview video image that may be added to a multiview video coded bitstream.

3 is an embodiment to which the present invention is applied and shows an overall prediction structure of a multiview image signal for explaining a decoding process using an interview prediction flag.

4 is an embodiment to which the present invention is applied and shows a hierarchical structure of level information for explaining a decoding process in which a reference view flag is used.

Claims (10)

Obtaining first flag information identifying from the extended region of the NAL unit header of the video signal whether the NAL unit is a multiview video coded bitstream or a scalable video coded bitstream; Acquiring attribute information about a multiview image from the video signal when the multiview video is a bitstream according to the first flag information; wherein the attribute information includes interview prediction flag information and interview picture group identification information. And the interview prediction flag information is information indicating whether a coded picture of the NAL unit is used for inter-view prediction, and the interview picture group identification information is information for identifying whether a picture type is an interview picture group. The interview picture group is a picture in which all slices in the picture are encoded by referring to slices of the same time zone. Acquiring interview reference information from an extended region of the sequence parameter set of the video signal; wherein the interview reference information includes information about the number of interview reference pictures and viewpoint identification information of the interview reference picture, and the interview reference information includes: Obtained for each of the interview picture group and the non-interview picture group based on the interview picture group identification information, When a coded picture of the NAL unit is used for inter-view prediction according to the interview prediction flag information, generating a reference picture list for inter-view prediction using the interview picture group identification information and the interview reference information. ; And Performing inter prediction using the reference picture list for the inter-view prediction Video signal decoding method comprising a. Obtain first flag information identifying whether a NAL unit is a multiview video coded bitstream or a scalable video coded bitstream from an extension region of a NAL unit header of a video signal, and multiview according to the first flag information A parsing unit obtaining attribute information of a multi-view image from the video signal in the case of a video coded bitstream; wherein the attribute information includes interview prediction flag information and interview picture group identification information, and the interview prediction flag information includes: Information indicating whether a coded picture of a NAL unit is used for inter-view prediction, wherein the interview picture group identification information is information for identifying whether a picture type is an interview picture group, and the interview picture group includes all slices in the picture. Only refer to slices in the same time zone Is a coded picture, The parsing unit obtaining interview reference information from an extended region of the sequence parameter set of the video signal; wherein the interview reference information includes information about the number of interview reference pictures and viewpoint identification information of an interview reference picture, and the interview reference information includes: Obtained for each of the interview picture group and the non-interview picture group based on the interview picture group identification information, When the coded picture of the NAL unit is used for inter-view prediction according to the interview prediction flag information, decoding to generate a reference picture list for inter-view prediction using the interview picture group identification information and the interview reference information. A picture buffer unit; And An inter prediction unit performing inter prediction using the reference picture list for inter-view prediction Video signal decoding apparatus comprising a. delete delete delete delete delete delete delete delete

Images