KR20070013995A - Method for encoding a video signal using inter-layer prediction and decoding the encoded signal - Google Patents

Abstract

A method for decoding an image signal is provided to obtain residual data from the image restored from a lower layer when the image signal of plural layers is encoded and decoded, thereby enabling residual prediction. Information, which indicates the residual prediction on a target block within a picture of a first layer of each encoded bit stream, is checked. If the information indicates the residual prediction, a block having decoded data of a second layer of the encoded bit stream, which corresponds to the target block, is coded into residual data by using the target block or coding information of blocks including the target block. Residual data of the target block are obtained by adding the coded residual data to data of the target block.

Description

A method for encoding a video signal using the inter-layer prediction and decoding the encoded video signal {Method for encoding a video signal using inter-layer prediction and decoding the encoded signal}

1 is a diagram illustrating an operation of a decoder for performing residual prediction according to an embodiment of the present invention.

2 illustrates an example in which blocks of an enhanced layer and their block information are shrunk according to the present invention.

3 illustrates an example of recoding reconstructed image data into residual data using contracted block information according to the present invention.

4 is a diagram illustrating an operation of a decoder for performing residual prediction according to another embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

101, 111: inverse quantization and discrete cosine inverse transform unit

103, 112: temporal inverse decomposition performing unit 200: residual inverse predicting unit

The present invention relates to a method of encoding a video signal by an inter-layer prediction method and decoding the encoded video signal accordingly.

The scalable video codec (SVC) method encodes a video signal at the highest quality and decodes a partial sequence of the resulting picture sequence (a sequence of intermittently selected frames throughout the sequence). Even if it is used, it is a way to enable a low-quality video representation. The Motion Compensated Temporal Filter (MCTF) scheme is one of the proposed encoding schemes for use in the scalable image codec.

However, a picture sequence encoded by the scalable MCTF can be expressed in a low quality image only by receiving and processing only a partial sequence. However, when the bit rate is lowered, the picture quality is greatly deteriorated. In order to solve this problem, a separate auxiliary picture sequence for a low data rate, for example, a small picture and / or a low picture sequence per frame may be provided.

The auxiliary sequence is called a base layer, and the main picture sequence is called an enhanced (or enhanced) layer. However, since the base layer and the enhanced layer encode the same video signal source, there is redundant information (redundancy) in the video signals of both layers. Therefore, in order to increase the coding rate of the enhanced layer, macroblocks in the image frame of the enhanced layer based on an arbitrary image frame of the base layer are simultaneously referred to as image differences (also called 'residual'). The block coded by this method is also called an intra BL mode (intra_BL). In addition, even for a block having residual data coded in an inter mode by motion estimation, when the corresponding block of the base layer is coded in the inter mode, the difference data with the residual data (residual difference Also coded as)).

The latter coded method is called residual prediction. For the block of the enhanced layer coded as described above, residual_prediction_flag is set to 1 and transmitted to the decoder.

However, in order for the decoder to reconstruct the block coded by the residual prediction scheme to the pre-coding data, the residual data should be obtained from the stream of the received base layer. However, only reconstructed image data may be obtained from the base layer, and residual data before reconstruction may not be provided. For example, in the case of single-loop decoding, residual data before reconstruction to a video signal cannot be provided from the base layer.

In such a case, the decoder cannot restore the residual difference data of the residual predicted block to the original residual data.

Accordingly, an object of the present invention is to provide a method for acquiring residual data from a reconstructed image of a lower layer and enabling residual prediction in encoding and decoding a video signal received in a plurality of layers. .

The decoding method according to the present invention for achieving the above object, in receiving each of the encoded bit stream of the first layer and the second layer and decoded into a video signal, the register for the target block in the picture of the first layer A first step of confirming first information indicating whether or not dual prediction is performed; and if the first information indicates residual prediction, a block having decoded data of the second layer corresponding to the target block; And coding the corresponding block of the second layer into residual data using coding information of the target block or blocks including the target block, and encoding the coded residual data of the corresponding block into the target block. It is characterized by performing three steps of obtaining residual data of the target block by adding to the data of.

In addition, the encoding method according to the present invention, in encoding the input image pictures into a data stream of the first layer and the second layer, for the first and second image blocks in the first picture and the second picture among the picture pictures. A motion estimation / prediction operation is performed to code residual data, and for the block of the second layer decoded after encoding corresponding to the first image block, the first image block or blocks including the image block. Coding the corresponding block into residual data using coding information, subtract the coded residual data of the corresponding block from the data of the first image block, and encode the first image block into residual difference data, The second residual is obtained by subtracting encoded residual data of the block of the second layer corresponding to the second image block from the data of the second image block. Coding an image block with residual difference data, wherein the first information is set to a value indicating residual prediction for the first image block, and the second information is different from the first information for the second image block. Is set to a value indicating residual prediction.

In one embodiment according to the present invention, the first information is information distinguished from a flag (residual_prediction_flag) indicating whether the residual data is directly predicted from the residual data acquired at the time of encoding the second layer and coded with the residual difference. .

In one embodiment according to the present invention, if the flag (residual_prediction_flag) is directly predicted from the residual data obtained at the time of encoding of the second layer and is indicated as coded with a residual difference, steps 1 to 3 may be performed. If not, otherwise, the first step is performed, and steps 2 and 3 are performed according to the check result.

In another embodiment according to the present invention, if the type of the picture of the first layer is confirmed, and if a specific type, for example, a key (key), perform the steps 1 to 3, if not a specific type , Do not perform steps 1 to 3.

In one embodiment according to the present invention, in using the coding information, the data area is coded as residual data using the data area in the corresponding block, which is smaller than the data area to which the coding information is applied.

In one embodiment according to the present invention, when the coding information is used for the reduced data area in the corresponding block, the motion vector in the coding information is also reduced.

According to an embodiment of the present invention, the residual block is restored by adding the corresponding block coded as residual data or a part of the corresponding block to the data of the target block.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an operation mainly of a decoder for performing residual prediction according to the present invention. The decoder of FIG. 1 receives and decodes an encoded stream (texture information, macro block information, etc.) of an enhanced layer and a base layer, respectively. In the description of the present invention, two layers are described for convenience, but in the digital stream in which three or more layers are received, the present invention can be used as it is between two adjacent layers having one or more layers between or between adjacent layers. Can be applied.

The decoder of FIG. 1 performs inverse quantization (IQ) and discrete cosine inverse transform (IDCT) on the texture information in the received stream of the base layer (111). With respect to the data thus obtained, each macroblock is reconstructed into a pre-coded image using macroblock information in a received stream, and inverse temporal decomposition is performed on each picture (112). Restore The decoder of FIG. 1 is based on the assumption that the video stream of the base layer is coded by the MCTF scheme, but this is merely an example and may be other than the MCTF (eg, the AVC scheme). In this case, the decoder of FIG. 1 has a decoding block corresponding thereto and accordingly decodes the received signal into a video signal.

Meanwhile, the decoder performs inverse quantization and discrete cosine inverse transform on the texture information in the stream of the received enhanced layer (101). In addition, the received macroblock information first checks the macroblock of the 'intra BL mode', and for the macroblock, residual data of the block is obtained by using the reconstructed image of the macroblock of the base layer corresponding to the block. Reconstruct the image data before coding (102). In this case, the term 'corresponding block' refers to a block that is temporally co-temporally and spatially co-located within a frame (or slice). If the resolution of the frame of the enhanced layer and the base layer is different, the 'corresponding block' includes the macro layer of the enhanced layer when the frame of the base layer is enlarged to have the same resolution as that of the enhanced layer. Means block. In this case, when the resolution is different between the two layers, the restored image of the block of the corresponding base layer is enlarged (105), and the enlarged image is used to restore the residual data of the macro block of the intra BL mode to the image data. Use as a guide.

In addition, the decoder checks the residual predicted macroblock from the base layer and reconstructs the data before coding by the residual prediction for the block (200). For this purpose, specific information is received from the information of the received macroblock. For example, check the value of 'pseudo_residual_prediction_flag'. For example, if the value is 1, the residual data of the block corresponding to the current macro block is obtained from the image of the base layer reconstructed previously, and based on the residual data, the residual difference data of the residual predicted current block is obtained. Convert to This will be described in more detail below. In the following description, the resolutions of the enhanced layer and the base layer are described as having a difference of 2 times. However, when there is no difference (in this case, shrinking of macro block information, upsampling of data, etc.) Even if there is a difference or more, the same principle as that described in the following example can be applied without changing the ratio.

The decoder specifies a block of the base layer corresponding to the macro block of which the specific information, for example, 'pseudo_residual_prediction_flag' has a value of 1, and when the specified block is enlarged twice, 4 which is spatially co-located with the specified block. Obtain the block information (block mode, motion vector, subblock information, etc.) of the two blocks (where the ratio of the resolution is larger than 2, the number is different, and at least one of the blocks has a value of pseudo_residual_prediction_flag is 1.) The block information of the obtained four blocks is shrunk (201).

2 shows an example in which block information is shrunk according to the present invention. In the contraction of this block information, basically, blocks of four enhanced layers are converted into information about blocks of one base layer. First, each macro block of the enhanced layer is an 8x8 subblock in the block of the base layer. Information of each macro block is also interpreted as information of each 8x8 sub-block corresponding thereto. The subblocks in each block of the enhanced layer correspond to the subblocks in the block of the base layer in the same ratio, where the size that should correspond to the minimum subblock size in the macroblock, for example, 4x4 or less. The subblocks (eg, 4x8, 8x4, and 4x4) of the enhanced layer having all correspond to the 4x4 subblocks (A1 and A2 in FIG. 2). Then, in the contraction of the information, the motion vector of the enhanced layer is reduced by two times according to the resolution ratio.

2, the upper left block 21 is an intra mode block, and thus becomes an 8x8 subblock marked as not-predicted, and the right side having two 8x16 subblocks. The macro block 22 of the upper inter mode becomes two 4x8 subblocks of the upper right in the corresponding block of the base layer, and the macro block 23 of the inter mode of the lower left of the lower mode of the corresponding block corresponds to 8x8 sub of the lower left of the corresponding block. The bottom right macroblock 24, which is a block, has two 8x8, two 4x8 (A2) and also two 8x4 (A1) subblocks, the four 4x4 subblocks at the bottom right, as shown in the figure. Corresponds to.

For each subblock (except for non-predicted subblocks) of the base layer having the original video signal decoded, block information of the macroblock or subblock of the enhanced layer corresponding thereto and reduced motion vector information corresponding thereto. The prediction operation is performed by using the like (202). That is, each subblock of the corresponding base layer is coded as residual data using block information of the enhanced layer corresponding to the subblock and reduced motion vector information.

FIG. 3 shows an example of this process. In the example of FIG. 2, the residual data of the subblock B of the base layer corresponding to the macroblock 22 having the two 8x16 subblocks in the upper right is residual data. For example, the subblock B corresponds to a macroblock of an enhanced layer whose current pseudo_residual_prediction_flag is 1.

In the example of FIG. 3, the subblock 22a of the left 8x16 of the macroblock 22 is in the forward mode, so that the decoder is located at the left portion of the 4x8 of the subblock B of the base layer corresponding thereto. In operation 301, the reduced motion vector of the left subblock 22a and the reference picture value (ref_idx_10) of the omnidirectional direction are used as residual data. In addition, since the 8x16 subblock 22b of the right side of the macroblock 22 is a bi-directional mode, the decoder determines that the right portion of 4x8 of the subblock B of the base layer corresponding thereto corresponds to: In operation 302, the reduced motion vector of the right subblock 22b and the reference picture values ref_idx_l0 and ref_idx_l1 in the forward and backward directions are used as residual data, respectively (302).

If the subblock corresponding to the macroblock of the enhanced layer whose current pseudo_residual_prediction_flag is 1 is another subblock (except for the non-predicted subblock) in the example of FIG. 2, the subblock is shown in the example of FIG. 3. The same principle is used for residual coding.

Instead of coding only the subblocks corresponding to the macroblock of the enhanced layer with pseudo_residual_prediction_flag as 1, the entire macroblock (excluding the subblocks marked as unpredicted) including the subblocks is encoded as residual data. Can also be used.

In the above manner, if some or all of the macro blocks of the base layer corresponding to the macro blocks of the enhanced layer whose current pseudo_residual_prediction_flag is 1 are coded with residual data, the decoder upsamples all or some of the macro blocks. (up-sampling) to enlarge the size (203). In the enlarged macroblock, the original residual data is restored by adding residual data of an area corresponding to the current macroblock to data (residual difference data) of the current macroblock.

By the above-described process, the intra mode block is reconstructed into image data before coding (102), and the inter-layer residual predicted block is reconstructed from residual difference data into residual data (204) and then temporal inverse decomposition process ( The original video signal is recovered through 103. Since the temporal reverse decomposition process is not directly related to the present invention, description thereof is omitted.

On the other hand, in order to support the data reconstruction by the residual prediction using information shrinkage and the contracted information as described above, even in the encoder, in the case of a macroblock having a coding gain in the inter-layer residual prediction, The coding by residual prediction is performed in the same manner as described above.

In other words, the inverse quantization and discrete cosine inverse transform are performed on the encoded base layer video stream, which is then temporally inversely decomposed (in the case of MCTF encoding, and decoding according to other codec methods). After creation, the macroblock of the enhanced layer coded in the inter-mode is subjected to the same process as in step 200 of FIG. 1, and the residual of the macroblock of the base layer corresponding to the macroblock or the corresponding region in the block is performed. After the data is obtained, the gain of the coding of the current macroblock into the residual data and the coding into the difference data from the obtained residual data of the base layer, for example, the effect of reducing the amount of coding is compared. If coding to residual difference data is more advantageous, coding with difference data of residual data for the corresponding macro block, and as coding information for the block, as previously mentioned, transmits by setting pseudo_residual_prediction_flag to 1 Done.

On the other hand, instead of performing the inter-layer residual prediction in the same manner as in step 200 of FIG. 1, the decoder uses the residual data generated by the encoding process of the base layer to perform residual prediction on the macroblock of the enhanced layer. The video stream may be received from an encoder (or a server providing a stream encoded in such a manner) used directly in the. Residual predicted macroblocks using the residual data obtained in the encoding of the base layer directly are set with a flag other than the above-mentioned specific information, for example, pseudo_residual_prediction_flag, for example, residual_prediction_flag.

In addition, other encoders apply residual prediction coding differently according to the type of an image picture by coding using residual prediction according to the same procedure as 200 described above, and directly using residual data obtained when encoding the base layer. One video stream may be generated. For example, for a key picture or an L picture (low-pass subband picture), the pseudo_residual_prediction_flag is transmitted by coding by residual prediction using the same process as described above. For a key picture or an H picture (high-pass subband picture), residual_prediction_flag may be transmitted by performing residual prediction coding using the residual data of the base layer directly.

The above-mentioned 'key picture' may be an example of a picture coded into residual data by performing a prediction operation using only a quality-base picture. On the other hand, a picture coded as residual data using SNR enhancement picture data in addition to the basic picture is called a non-key picture. The definition of this key picture is only one example, and the present invention is not limited thereto.

Therefore, since a plurality of types of residual prediction information can be used, the decoder for performing the residual prediction according to the present invention can reconstruct both the residual prediction codings by the two methods as shown in FIG. 4. Has a configuration. The configuration of FIG. 4 is also shown mainly for operation.

The decoder of FIG. 4 restores data by directly using the residual data before the decoding of the image signal of the base layer, in addition to the data restoration operation 200 by the residual prediction using the reconstructed base layer image of the decoder of FIG. Action 401 may also be performed.

Therefore, the decoder of FIG. 4 corresponds to the macroblock in the data stream of the base layer subjected to inverse quantization and the discrete cosine inverse transform, to a macroblock in which residul_prediction_flag is set in the enhanced stream through the inverse quantization and the discrete cosine inverse transform. The original residual data is restored based on the residual data of the block to be generated (401). Of course, in this case, if the resolution between the layers is different, up-sampling according to the ratio is used to enlarge (402) the block of the base layer.

The decoder of FIG. 4 does not perform the checking operation of the pseudo_residual_prediction_flag for determining whether to perform the data reconstruction operation 200 by the residual prediction at a later stage for the block in which residual_prediction_flag is set. In addition, the decoder of FIG. 4 does not check the pseudo_residual_prediction_flag and does not perform the reconstruction operation 200 described above if the picture to be decoded is not a specific type, for example, if it is not a key picture.

Therefore, an encoder that performs residual prediction coding using the residual data directly does not need to transmit pseudo_residual_prediction_flag.

The decoder for performing the inter-layer residual prediction operation described above may be mounted in a mobile communication terminal or the like or in an apparatus for reproducing a recording medium.

It is to be understood that the present invention is not limited to the above-described exemplary preferred embodiments, but may be embodied in various ways without departing from the spirit and scope of the present invention. If you grow up, you can easily understand. If the implementation by such improvement, change, replacement or addition falls within the scope of the appended claims, the technical idea should also be regarded as belonging to the present invention.

As described above in a limited embodiment, the present invention, when decoding and outputting a video signal of a specific layer, even if the residual data before being reconstructed from the lower layer to the image is not received from the inter-layer residual The coding efficiency can be improved by enabling coding by prediction.

Claims (12)

A method of receiving each encoded bit stream of a first layer and a second layer and decoding the same into a video signal, A first step of confirming first information indicating whether or not a residual prediction for a target block in a picture of the first layer is performed; If the first information indicates residual prediction, for the block having decoded data of the second layer, corresponding to the target block, coding information of the target block or blocks including the target block is used. Coding the corresponding block of the second layer into residual data to form a residual data; And adding the coded residual data of the corresponding block to the data of the target block to obtain the residual data of the target block. The method of claim 1, And wherein the first information is information distinguished from second information indicating whether the residual data is directly predicted from residual data obtained at the time of encoding the second layer and coded with a residual difference. The method of claim 2, The method may further include checking a value of the second information. Steps 1 to 3 are performed according to the identified value, or is not performed. The method of claim 3, If the value of the second information is directly predicted from the residual data obtained at the time of encoding the second layer and is indicated as coded with the residual difference, the steps 1 to 3 are not performed. . The method of claim 1, The method may further include checking a type of the picture of the first layer. If the identified type does not belong to a predetermined type, steps 1 to 3 are not performed. The method of claim 5, And said predetermined type is a key picture. The method of claim 1, And the step (2), in using the coding information, codes the data area into residual data using the data area in the corresponding block, which is smaller than the data area to which the coding information is applied. The method of claim 7, wherein In the step 2, in using the coding information, the motion vector information in the coding information is reduced according to a reduction ratio of an area to which the coding information is applied. The method of claim 7, wherein In the step 3, the corresponding block coded as residual data or a part of the corresponding block is enlarged and added to the data of the target block. A method for encoding input picture pictures into data streams of a first layer and a second layer, the method comprising: Performing a motion estimation / prediction operation on the first and second image blocks in the first picture and the second picture among the picture pictures, and coding them into residual data; For the block of the second layer decoded after encoding corresponding to the first picture block, the corresponding block is encoded into residual data using coding information of the first picture block or blocks including the picture block. Two stages, Coding the first image block into residual difference data by subtracting the coded residual data of the corresponding block from the data of the first image block, and encoding the block of the second layer corresponding to the second image block. Subtracting the residual data from the data of the second image block and coding the second image block into residual difference data. First information is set to a value indicating residual prediction for the first image block, and second information different from the first information is set to a value indicating residual prediction for the second image block. Characterized in that the method. The method of claim 10, And wherein the first picture is a picture coded into a picture having low-pass component data, and the second picture is a picture coded into a picture having high-band component data. The method of claim 10, And wherein the first picture is a key picture and the second picture is a non-key picture.

Images