KR20070075293A - Inter-layer motion prediction method for video signal - Google Patents

Abstract

An inter-layer motion prediction method in encoding/decoding a video signal is provided to improve data coding rate irrespective of the picture type of a video signal when a plurality of layers is coded. For an inter-mode macroblock and an intra-mode macroblock of a base layer, which are vertically adjacent to each other, information on the motion of the inter-mode macroblock is set as information on the motion of the intra-mode macroblock, and then motion information on a pair of vertically neighboring macroblocks to be used for inter-layer motion prediction is obtained on the basis of the inter-mode macroblock and the intra-mode macroblock.

Description

Inter-layer motion prediction method for encoding / decoding video signal {Inter-layer motion prediction method for video signal}

1A and 1B illustrate a method of coding a plurality of layers from a single image source, respectively.

2A and 2B briefly illustrate a configuration of an image signal encoding apparatus to which the inter-layer prediction method according to the present invention is applied.

3A and 3B illustrate types of picture sequences encoding video signals of an interlaced scan method, respectively.

4A through 4F illustrate a virtual base layer to be used for inter-layer motion prediction of a field MB in an MBAFF frame using motion information of a frame macro block MB according to an embodiment of the present invention. It schematically shows a process of deriving the motion information of the field MB,

5A to 5B illustrate a derivation process of a reference index and motion information according to another embodiment of the present invention.

6A to 6C schematically illustrate a process of deriving motion information of a field MB of a virtual base layer using motion information of a frame MB according to an embodiment of the present invention.

7A to 7B illustrate a derivation process of a reference index and motion information according to another embodiment of the present invention.

8A to 8C schematically illustrate a process of deriving motion information of a field MB frame MB of a virtual base layer to be used for inter-layer motion prediction using motion information of field MB in an MBAFF frame according to an embodiment of the present invention. Represented by

9A and 9B illustrate a derivation process of a reference index and motion information according to other embodiments of the present invention, respectively.

10A to 10C schematically illustrate a process of deriving motion information of frame MB of a virtual base layer to be used for inter-layer motion prediction using motion information of field MB in a field picture according to an embodiment of the present invention. Will,

11 illustrates a derivation process of a reference index and motion information according to another embodiment of the present invention.

12A and 12B schematically illustrate a process of deriving motion information of a frame MB of a virtual base layer to be used for inter-layer motion prediction using motion information of a field MB in a field picture according to another embodiment of the present invention. Shown,

13A to 13D illustrate a process of deriving motion information of field MB of a virtual base layer to be used for inter-layer motion prediction using motion information of field MB according to the type of picture, according to an embodiment of the present invention. Are shown diagrammatically,

14A to 14H illustrate a method of performing inter-layer prediction when the inter-layer resolution is different according to an embodiment of the present invention, separately according to the type of picture.

<Description of the symbols for the main parts of the drawings>

20: EL encoder 21: BL encoder

The present invention relates to a method for predicting inter-layer motion when encoding / decoding a video signal.

The scalable video codec (SVC) method encodes a video signal at the highest quality, but decodes a partial sequence of the resulting picture sequence (a sequence of intermittently selected frames throughout the sequence). Even if it is used, it is a way to enable a low-quality video representation.

By the way, a picture sequence encoded in a scalable manner can display a low quality image only by receiving and processing only a partial sequence. However, when the bit rate is lowered, the picture quality is greatly deteriorated. In order to solve this problem, a separate auxiliary picture sequence for a low data rate, for example, a small picture and / or a low picture sequence per frame may be provided in a hierarchical structure as at least one layer.

Assuming two sequences, the auxiliary sequence (lower sequence) is called a base layer, and the main picture sequence (higher sequence) is called an enhanced (or enhancement) layer. However, since the base layer and the enhanced layer encode the same video signal source, redundancy information exists in the video signals of both layers. Therefore, in order to increase the coding rate of the enhanced layer, the image signal of the enhanced layer is coded using the coded information (motion information or texture information) of the base layer.

In this case, as shown in FIG. 1A, although coding may be performed from one image source 1 to a plurality of layers having different data rates, different scanning may be performed on the same content 2a as in FIG. 1B. A plurality of image sources 2b having a) scheme may be coded into each layer. However, even at this time, since both sources 2b are the same content 2a, an encoder coding an upper layer may increase coding gain when inter-layer prediction using coded information of a lower layer is performed.

Therefore, when coding to different layers from different sources, an inter-layer prediction method considering the scanning method of each image signal is required. In addition, when coding an interlaced image, it may be coded into even and odd fields, or may be coded as a pair of odd and even macroblocks in one frame. . Therefore, the type of the picture coding the interlaced video signal should also be considered in inter-layer prediction.

An object of the present invention is to provide a method for performing inter-layer motion prediction under a condition in which at least one of the two layers has interlaced scanning image signal components.

One of the inter-layer motion prediction methods according to the present invention includes information on the motion of the macro block of the inter mode with respect to both macro blocks of the vertically adjacent inter mode and the intra mode of the base layer. After setting the information on the motion of the macro block in the intra mode, motion information on a pair of vertically adjacent macro blocks to be used for inter-layer motion prediction is obtained based on the two macro blocks.

In another method of predicting inter-layer motion according to the present invention, information on motion is zero for an intra mode macro block among both macro blocks of vertically adjacent inter mode and intra mode of a base layer. After setting to the block of the inter mode, motion information for a pair of vertically adjacent macro blocks to be used for inter-layer motion prediction is obtained based on the above macro blocks.

Another method of the inter-layer motion prediction method according to the present invention derives motion information for a single macroblock from motion information of a vertically adjacent frame macroblock pair of a base layer, and converts the derived motion information into a current layer. It is used as the motion information of one field macroblock of or as prediction information of each motion information of a field macroblock pair.

Another method of the inter-layer motion prediction method according to the present invention is to perform two macroblocks from motion information of a single field macroblock of a base layer or motion information of a single field macroblock selected from a pair of vertically adjacent field macroblocks. Each motion information is derived, and the derived motion information is used as prediction information of each motion information of the frame macroblock pair of the current layer.

According to the present invention, an inter-layer motion prediction method having a different resolution of a picture may be performed by selectively applying a prediction method of converting a picture of a lower layer into frame macroblocks according to the type of the picture and the type of the macroblock in the picture. Inter-layer prediction method suitable for the type of frame macroblock in the upsampled frame picture and the macroblock in the picture of the higher layer after converting the frame picture to a resolution and upsampling the frame picture to be the same as the resolution of the upper layer. Apply.

In an embodiment according to the present invention, in inter-layer motion prediction, prediction is performed in order of a partition mode, a reference index, and a motion vector.

In another embodiment according to the present invention, prediction is performed in order of a reference index, a motion vector, and a split mode.

In one embodiment according to the present invention, motion information of a pair of field macro blocks of a virtual base layer to be used for inter-layer motion prediction is derived from a pair of frame information of frame macro blocks of a base layer.

In one embodiment according to the present invention, motion information of one field macro block in even or odd field pictures of a virtual base layer to be used for inter-layer motion prediction is derived from a pair of frame information of frame macro blocks of a base layer.

In one embodiment according to the present invention, a single macroblock is selected from a pair of field macroblocks of a base layer, and motion of a pair of frame macroblocks of a virtual base layer to be used for inter-layer motion prediction from the motion information of the selected macroblock. Derive information.

In one embodiment according to the present invention, motion information of a pair of frame macroblocks of a virtual base layer to be used for inter-layer motion prediction is derived from motion information of one field macroblock in an even or odd field picture of the base layer.

In another embodiment according to the present invention, a virtual field macro block is further configured by copying information of one field macro block in even or odd field pictures of the base layer, and a pair of field macro blocks configured as described above. Motion information of a pair of frame macroblocks of a virtual base layer to be used for inter-layer motion prediction is derived from the motion information of.

According to an embodiment of the present invention, in the inter-layer motion prediction method having different resolutions of a picture, the prediction method applied to converting a picture of a lower layer into a frame picture of the same resolution includes copying of macro block information and frame macro. Prediction of block macrofield pairs into field macroblock pairs, Prediction of frame macroblockpairs into single field macroblocks, Prediction of field macroblockpairs into frame macroblock pairs, Prediction of singlefield macroblocks into frame macroblock pairs One.

According to an embodiment of the present invention, in the inter-layer motion prediction method having a different resolution of a picture, the inter-layer prediction method applied based on a frame macro block in the upsampled frame picture is a field macro of a frame macro block pair. Prediction into block pairs, Prediction of frame macro block pairs into single-field macro blocks, Prediction of field macro block pairs into frame macro block pairs, Prediction of single-field macro blocks into frame macro block pairs, Fields of field macro blocks At least one of prediction to a macroblock and prediction to a frame macroblock of the frame macroblock.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2A is a block diagram of a configuration block of an image signal encoding apparatus to which an inter-layer prediction method according to the present invention is applied. The apparatus of FIG. 2A encodes an input video signal into two layers, but the principles of the present invention described later may be applied between layers even when coding into three or more layers.

The inter-layer prediction method according to the present invention is performed in the enhanced layer encoder 20 (hereinafter, abbreviated as 'EL encoder') in the apparatus of FIG. 2A, and the base layer encoder 21 (hereinafter, referred to as 'BL encoder'). Receives encoded information (motion information and texture information) from (), and performs inter-layer motion prediction based on the received information. Of course, the present invention may code the input video signal using the image source 3 of the base layer already coded as shown in FIG. 2B, and the inter-layer prediction method described below is equally applied. .

In the case of FIG. 2A, there may be two ways in which the BL encoder 21 encodes an interlaced video signal (or a coded method of the 2b encoded video source 3). That is, as shown in FIG. 3A, encoding is performed as a field unit and encoding a field sequence. As shown in FIG. 3B, each macroblock of both even and odd fields is paired and encoded into a frame sequence including a frame. If it is. In a frame coded as described above, a macro block above a pair of macro blocks is referred to as a 'top macro block' and a macro block below is referred to as a 'bottom macro block'. If the upper macro block is an image component of the even field (or even field), the lower macro block is an image component of the even field (or even field). The frame constructed as described above is called a MBAFF (MacroBlock Adaptive Frame Field) frame. However, the MBAFF frame may also include a macroblock pair in which each macroblock consists of a frame MB, in addition to the macroblock pair consisting of the odd field and the even field.

Therefore, when a macroblock in a picture is an image component of an interlaced scanning method (hereinafter, such a macroblock is called a 'field MB'. In contrast, a macroblock having a progressive component of a video component is called a 'frame'. The block may be in a field or may be in a frame.

Accordingly, if the type of the macroblock to be coded by the EL encoder 20 and the type of the macroblock of the base layer to be used for motion prediction of the macroblock are frame MB, field MB, and field MB, respectively, the field MB in the field. The motion prediction method should be determined by distinguishing whether it is a field MB in an MBAFF frame.

Hereinafter, each case will be described separately. First, it is assumed that the resolutions of the current layer and the base layer are the same. That is, assume that SpatialScalabilityType () is 0. The case where the current layer is higher than the resolution of the base layer will be described later. In addition, in description of the following description and drawings, the term "top" is used for the even (or even), and the term "bottom" is used for the radix (or the good) in parallel.

I. For Frame MB-> Field MB of MBAFF Frame

In this case, MB is coded as a field MB in an MBAFF frame in the current layer EL, and MB of the base layer BL to be used for motion prediction for the field MB is coded as a frame MB. The video signal components included in the upper and lower macro blocks (hereinafter referred to as 'macro block pairs', hereinafter referred to as 'pairs' to refer to blocks vertically adjacent to each other) may correspond to corresponding positions of the current layer ( Each of the colocated macroblock pairs is the same as the video signal component included in each.

Therefore, the macroblock partition mode obtained by merging (compressing by 1/2 in the vertical direction) the macroblock pair 410 of the base layer into one macroblock is used as the partition mode of the current macroblock. 4A shows this as a specific example. As shown, the corresponding macroblock pair 410 of the base layer is first merged into one macroblock (S41), and the split mode obtained by the merging is copied to the other (S42) to the macroblock pair 411. After the configuration, the split mode of the macro block pair 411 is applied to the macro block pair 412 of the virtual base layer, respectively (S43).

However, when merging the corresponding macro block pairs 410 into one macro block, a partition area not allowed in the split mode may be generated. In order to prevent this, the EL encoder 20 according to the following rule. Determine the partition type.

One). The two upper and lower 8x8 blocks (B8_0 and B8_2 in FIG. 4A) in the macro block pair of the base layer merge into one 8x8 block. If each of the corresponding 8x8 blocks is not subdivided, they are divided into two 8x4 blocks. Merge, and if any are subdivided, they merge into four 4x4 blocks (401 in FIG. 4A).

2). 8x16 blocks of the base layer are reduced to 8x8 blocks, 16x8 blocks to two horizontally adjacent 8x4 blocks, and 16x16 blocks to 16x8 blocks.

If at least one of the corresponding macro block pairs is coded in the intra mode, the EL encoder 20 performs the following process before the merging process.

If only one of them is in the intra mode, as shown in FIG. 4B, the motion information (MB division mode, reference index, and motion vectors) of the macro block in the inter mode is converted to the macro block in the intra mode. Or to a macroblock of intra mode as illustrated in FIG. 4C as a macroblock of 16x16 inter mode with a motion vector of zero and a reference index of zero, or for an intra mode MB as illustrated in FIG. 4D. After the reference index of the mode MB is set by copying and the motion vector has 0 motion vectors, the merge process described above is performed, and the reference index and motion vector derivation process described below are also performed.

The EL encoder 20 performs the following process to derive the reference index of the current macro block pair 412 from the reference index of the corresponding macro block pair 410.

If the 8x8 block pair of the base layer corresponding to the current 8x8 block is subdivided by the same number of times, the reference index of any 8x8 block in the pair (top or bottom) is determined as the reference index of the current 8x8 block, otherwise The reference index of the block having the smallest number of partitions is determined as the reference index of the current 8x8 block.

In another embodiment according to the present invention, the reference index of the smallest value among the reference indices designated in the 8x8 block pair of the base layer corresponding to the current 8x8 block is determined as the reference index of the current 8x8 block. This determination method is shown for the example of FIG. 4E.

refidx of curr B8_0 = min (refidx of base top frame MB's B8_0, refidx of base top frame MB's B8_2)

refidx of curr B8_1 = min (refidx of base top frame MB's B8_1, refidx of base top frame MB's of base B8_3)

refidx of curr B8_2 = min (refidx of base bottom frame MB ’s B8_0, refidx of base bottom frame MB ’s B8_2), and

refidx of curr B8_3 = min (refidx of base bottom frame MB ’s B8_1, refidx of base bottom frame MB ’s B8_3)

Becomes

The above-described reference index derivation process can be applied to both the upper field MB and the lower field MB. Then, the reference index of each 8x8 block determined in the manner described above is multiplied by 2 to be the final reference index value. This is because, when decoding, the field MB belongs to a picture divided into even and odd fields, so the number of pictures is twice as large as in the frame sequence. Depending on the decoding algorithm, the reference index of the lower field MB may be determined as the final reference index value by adding 1 after 2 times.

The process of deriving the motion vector of the macroblock pair of the virtual base layer by the EL encoder 20 is as follows.

Since the motion vector is determined based on the 4x4 block, as shown in FIG. 4F, the corresponding 4x8 block of the base layer is identified, and if the block is subdivided, the motion vector of the upper or lower 4x4 block is currently 4x4. The motion vector of the block is determined. Otherwise, the motion vector of the corresponding 4x8 block is determined as the motion vector of the current 4x4 block. For the determined motion vector, a vector obtained by dividing the vertical component by 2 is used as the final motion vector. This is because the size of the image is reduced to 1/2 on the vertical axis in the field image because the image component carried in the two frame MBs corresponds to the image component of one field MB.

When the motion information of the field MB pair 412 of the virtual base layer is obtained in the above manner, the motion information is used to predict the inter-layer motion of the target layer MB pair 413 of the current layer, that is, the enhanced layer. do. In the following description, if motion information for an MB or MB pair of a virtual base layer is obtained, the motion information is used for inter-layer motion prediction for a corresponding MB or MB pair of the current layer. Therefore, in the following description, even if it is not mentioned that the motion information of the MB or MB pair of the virtual base layer is used for the inter-layer motion prediction of the corresponding MB or MB pair of the current layer, this is of course applied.

5A illustrates a field MB pair 500 of a virtual base layer to be used for inter-layer prediction from motion information of a frame MB pair of a base layer corresponding to a current macroblock pair, according to another embodiment of the present invention. It is shown schematically to derive the motion information of. In this embodiment, as shown, the reference index of each upper 8x8 block of the field MB pair 500 of the virtual BL uses the reference index of the upper or lower 8x8 block of the upper MB of the frame MB pair of the BL, and each The reference index of the lower 8x8 block uses the reference index of the upper or lower 8x8 block of the lower MB of the BL. In the case of a motion vector, as shown, for each top 4x4 block of the field MB pair 500 of the virtual BL, the motion vector of the top 4x4 block of the top MB of the MB of the frame MB pair of the BL is equally used. For a 4x4 block, the motion vector of the third 4x4 block of the top MB of the frame MB pair of BL is the same, and for each third 4x4 block, the motion vector of the top 4x4 block of the bottom MB of the MB of the frame MB pair of the BL is equally used. For each fourth 4x4 block, the motion vector of the third 4x4 block of the lower MB of the frame MB pair of the BL is equally used.

However, as illustrated in FIG. 5A, the upper 4x4 block 501 and the lower 4x4 block 502 in the 8x8 block in the field macroblock pair 500 configured for use in inter-layer prediction are different from each other in the base layer. The motion vectors of the 4x4 blocks in the blocks 511 and 512 are used, which may be motion vectors using different reference pictures. That is, the different 8x8 blocks 511 and 512 may have different reference index values. Thus, in this case, to construct the field macroblock pair 500 of the virtual BL, the EL encoder 20 selects the motion vector of the second 4x4 block 502 of the virtual BL, as illustrated in FIG. 5B. The motion vector of the corresponding 4x4 block 503 selected for the upper 4x4 block 501 is equally used (521).

In the embodiment described with reference to FIGS. 4A to 4F, the EL encoder 20 configures motion information of a virtual base layer to predict motion information for a current macroblock pair. In this case, the embodiment described with reference to FIGS. 5A and 5B is based on the motion information of the corresponding macroblock pair of the base layer. A reference index and a motion vector are first derived for the macroblock pair, and finally, the split mode of the macroblock pair of the virtual base layer is determined based on the derived result. Finally, when determining the split mode, combine 4x4 block units with the same derived motion vector and reference index, and then designate the combined block type as the combined type if it belongs to the allowed split mode. Specified in partitioned form.

In the embodiment described with reference to FIGS. 5A and 5B, if the corresponding macro block pairs 410 of the base layer are all in intra mode, only intra base prediction is performed on the current macro block pair 413 (in this case, motion prediction is performed Of course, in the case of texture prediction, no macroblock pairs of the virtual base layer are constructed.) If only one is intra mode, texture prediction (intra base prediction and residual prediction) is not performed. 4B, the inter mode motion information is copied to the MB of the intra mode as shown in FIG. 4B, or the intra mode sets the motion vector and the reference index of the MB to 0 as shown in FIG. 4C, or the intra mode MB as shown in FIG. 4D. After setting the reference index of the intermode MB by copying and setting the motion vector to have a zero motion vector, as described above, Motion prediction is performed.

On the other hand, when performing texture prediction, the even lines of the macro block pair of the base layer are used to predict the upper field MB of the current macro block pair, and the odd lines are used to predict the lower field MB.

After the MB encoder of the virtual BL is configured as described above, the EL encoder 20 predicts and codes the motion information of the current field macro block pair 413 using the MB pair of motion information.

II. For frame MB-> field MB for field picture

In this case, the MB is coded as the field MB in the field picture in the current layer, and the MB of the base layer to be used for motion prediction for the field MB is coded as the frame MB, which is included in the macroblock pair in the base layer. The video signal component is the same as the video signal component included in the macroblock in the even or odd field of the corresponding position of the current layer.

Therefore, the macroblock partition mode obtained by merging the macroblock pairs of the base layer into one macroblock (compressed in half in the vertical direction) is used as the partition mode of the macroblock of the even or odd virtual base layer. do. 6A shows this as a specific example. As shown, the corresponding macroblock pair 610 of the base layer is first merged into one macroblock 611 (S61), and the division mode obtained by the merging is inter-layered in the current macroblock 613. It is applied to the macroblock 612 of the virtual base layer to be used for motion prediction (S62). The merging rule is the same as in the above case I, and the processing in the case where at least one of the corresponding macro block pairs 610 is coded in the intra mode is the same as in the case of the above case I.

In addition, the process of deriving the reference index and the motion vector is also performed in the same manner as described above for the case I. In the case of I, however, the macroblock is carried in a frame of even and odd numbers so that the same derivation process is applied to the upper MB and the lower MB. However, in the present case (II), the base layer is stored in the field picture to be coded. Since only one macro block corresponding to the macro block pair 610 exists, only the derivation process is applied to only one field MB, as illustrated in FIGS. 6B and 6C.

In the above embodiment, when the EL encoder 20 predicts motion information for a macro block of a virtual base layer, the EL encoder 20 performs partition mode, reference index, and motion vector based on the motion information of a corresponding macro block pair of the base layer. Induced.

In another embodiment according to the present invention, based on the motion information of the corresponding macroblock pair of the base layer, a reference index and a motion vector are first derived for the macroblock of the virtual base layer and then based on the derived result. Finally, the split mode of the macro block of the virtual base layer is determined. 7A and 7B diagrammatically show that a reference index and a motion vector of a field macroblock of a virtual base layer are derived according to the present embodiment, in which case the macroblock of the base layer for one macroblock at the top or bottom is derived. The operations applied to deriving only the derivation of the corresponding motion information by using the pair of motion information are the same as I described above with reference to FIGS. 5A and 5B.

Finally, when determining the split mode, combine 4x4 block units with the same derived motion vector and the same reference index, and then designate the combined type if the combined block is an allowed split mode, otherwise leave the split mode before combining. .

In the above embodiments, if the corresponding macroblock pairs of the base layer are all intra modes, only intra base prediction is performed on the current macro block (in this case, motion prediction is not performed and motion information of the macro block pair of the virtual base layer is performed). If only one of them is intra mode, texture prediction (intra base prediction and residual prediction) is not performed and motion prediction is performed as described above for this case. When performing texture prediction, when the current macro block 613 is the even field MB, the even lines of the macro block pair of the base layer are used for prediction, and in the case of the odd field MB, the macro block pair of the base layer macro block is used. Use odd lines for prediction.

III. Field MB for MBAFF frame-> Frame MB

In this case, MB is coded into a frame MB in the current layer, and the MB of the base layer to be used for motion prediction of the frame MB is coded into the field MB in the MBAFF frame, which is included in one field MB of the base layer. The video signal component becomes the same component as the video signal component included in the macroblock pair at the corresponding position of the current layer.

Therefore, the macroblock partition mode obtained by stretching the macroblock at the top or the bottom of one macroblock pair of the base layer (doubled in the vertical direction) is used as the partition mode of the macroblock pair of the virtual base layer. 8A shows this as a specific example. In the following description and drawings, the upper field MB is selected, but the following description may also be applied to the case where the lower field MB is selected.

As shown in Fig. 8A, the upper field MB in the corresponding macroblock pair 810 of the base layer is doubled to form two macroblocks 811 (S81), and the segmentation form obtained by the stretching is virtualized. The macroblock pair 812 of the base layer is applied (S82).

However, when the corresponding one field MB is vertically doubled, a partitioning form that is not allowed in the splitting mode of the macroblock may be generated. In order to prevent this, the EL encoder 20 uses the following rule. The split mode is determined according to the extended split form.

One). The 4x4, 8x4, and 16x8 blocks of the base layer are determined to be 4x8, 8x8, and 16x16 blocks that are vertically doubled.

2). The 4x8, 8x8, 8x16 and 16x16 blocks of the base layer are determined as two equally sized top and bottom blocks. As illustrated in FIG. 8A, the 8x8 B8_0 block of the base layer is determined to be two 8x8 blocks (801). The reason for not specifying one 8x16 block is that the split mode of the macroblock is not supported in this case because the extended block adjacent to the left or right side may not be the 8x16 divided block.

If one of the corresponding macroblock pairs 810 is coded in the intra mode, the EL encoder 20 selects the field MB at the top or the bottom of the inter mode rather than the intra mode to perform the above stretching process. The division mode of the macroblock pair 812 of the virtual base layer is determined.

If both are intra modes, only inter-layer texture prediction is performed, and the split mode determination through the above-described stretching process and the reference index and motion vector derivation process described below are not performed.

The EL encoder 20 derives the reference index of the 8x8 block B8_0 of the corresponding base layer to derive the reference index of the macro block pair 812 of the virtual base layer from the reference index of the corresponding one field MB. As illustrated in FIG. 8B, each reference index of the two upper and lower 8x8 blocks is determined, and divided by 2 for the determined reference index of each 8x8 block to be a final reference index value. This is because the number of reference pictures is specified for the field MB on the basis of pictures divided into even and odd fields, so that the picture number needs to be cut in half to be applied to the frame sequence.

When the EL encoder 20 derives the motion vector for the frame macro block pair 812 of the virtual base layer, as shown in FIG. 8C, the motion vector of the corresponding 4x4 block of the base layer is converted into the virtual base layer. A motion vector of a 4x8 block in the macroblock pair 812 is determined, and a vector obtained by multiplying two by a vertical component with respect to the determined motion vector is used as the final motion vector. This is because the image component carried in one field MB corresponds to the image component of two frame MBs, so the size of the frame image is doubled on the vertical axis.

In the above-described embodiment, when the EL encoder 20 predicts motion information for a macroblock pair of the virtual base layer, the partition mode, the reference index, and the motion vector are ordered based on the motion information of the corresponding field MB of the base layer. Induced.

In another embodiment according to the present invention, based on the motion information of the corresponding field MB of the base layer, refer to the motion information of the MB pair of the virtual BL to be used for inter-layer prediction of the current macroblock pair, as illustrated in FIG. 9A. The index and the motion vector are obtained first, and finally, based on the obtained result, each split mode of the macroblock pair of the virtual base layer is determined. Finally, when determining the split mode, combine 4x4 block units with the same derived motion vector and the same reference index, and then designate the combined form if the combined block is an acceptable split form, otherwise specify the split form before combining. do.

Referring to the embodiment of FIG. 9A in more detail, as shown, in order to derive a reference index and a motion vector for a frame MB pair of a virtual BL to be used for motion prediction of a current macroblock pair, The field MB of the inter mode is selected and the motion vector and reference index of the MB are used. If both macroblocks are inter, any one of the upper or lower MBs is selected (901 or 902) to use the motion vector and reference index information of the MB. For the reference index, as shown, copy the corresponding value of the upper 8x8 block of the selected MB to the reference index of the upper and lower 8x8 blocks of the upper MB of the virtual BL, and copy the corresponding value of the lower 8x8 block of the selected MB to the virtual BL. Copy to the reference index of the top and bottom 8x8 blocks of the bottom MB of. As for the motion vector, as shown, the corresponding value of each 4x4 block of the selected MB is equally used as the motion vector of both up and down adjacent 4x4 blocks of the macro MB pair of the virtual BL. In another embodiment according to the present invention, in order to derive the motion vector and the reference index of the frame MB pair of the virtual BL, the motion information of the MB pair of the corresponding BL may be mixed. 9B illustrates a process of deriving a motion vector and a reference index according to the present embodiment. From the above description of the derivation process of the motion information and the diagram of FIG. 9B, the reference radii of the respective subblocks (8x8 block and 4x4 blocks) in the MB pair of the virtual BL and the corresponding radiative relationship of the motion vector can be intuitively understood. Therefore, detailed description thereof will be omitted.

However, in the embodiment of FIG. 9B, since all the motion information of the field MB pair of the base layer is used, when either of the field MB pairs is intra mode, the motion information of the MB is used as the motion information of the other inter mode MB. To induce. That is, as illustrated in FIG. 4B, the motion vector and the reference index of the inter mode MB are copied to the corresponding information of the intra mode MB, or as illustrated in FIG. 4C, the intra mode MB is 0 and the motion vector of 0. Consider an inter mode MB having a reference index of or for an intra mode MB as illustrated in FIG. 4D, after setting the reference index of the inter mode MB to copy and setting the motion vector to have a motion vector of 0, FIG. 9B. As illustrated in FIG. 6, the motion vector and reference index information of the MB pair of the virtual BL may be derived. When the motion vector and the reference index information are derived, as described above, the block mode for the MB pair of the virtual BL is determined based on the derived information.

On the other hand, if the corresponding field MB pairs of the base layer are all intra modes, only intra base prediction is performed on the current macro block 813 (in this case, motion prediction is not performed). The reference data of the intra base prediction is a 16x32 block in which the even and odd lines of the corresponding field MB pair are alternately selected and interleaved. If only one of them is intra mode, intra base prediction is performed using the corresponding field MB of the intra mode, and residual prediction is performed using the corresponding field MB of the inter mode. Of course, in this case, the field MB to be used for prediction is upsampled on the vertical axis and then used for texture prediction.

IV. For field MB-> frame MB of a field picture

In this case, MB is coded as a frame MB in the current layer EL, and MB of the base layer BL to be used for motion prediction of the frame MB is coded as a field MB in the field picture. The video signal component included in the field MB is the same as the video signal component included in the macroblock pair at the corresponding position of the current layer.

Therefore, the division mode obtained by doubling one macro block in the even or odd field of the base layer in the vertical direction is used as the division mode of the macro block of the virtual base layer. 10A illustrates this as a specific example. The process illustrated in FIG. 10A differs from that in the case where the top or bottom field MB in the MBAFF frame is selected, in contrast to III above, one field MB 1010 corresponding to position in the even or odd field is naturally used. In addition, it is the same to extend the corresponding field MB 1010 and apply the divided form of the two macroblocks 1011 obtained by the expansion to the macroblock pair 1012 of the virtual base layer. In addition, when the corresponding field MB 1010 is vertically doubled, a partitioning form that is not allowed in the division mode of the macroblock may occur, so that the EL encoder 20 is extended to prevent this. The rules for determining the split mode according to the form are the same as those given in cases 1) and 2) of III.

If the corresponding macroblock is coded in the intra mode, the EL encoder 20 does not perform the split mode determination through the above-described decompression process and the reference index and motion vector derivation process described below. In other words, inter-layer motion prediction is not performed.

In addition, the process of deriving the reference index and the motion vector is also the same as described for the case III above. However, in the case of III, since the corresponding macroblock of the base layer is loaded in a frame of even and odd numbers, the derivation process is selected by selecting one of the upper MB and the lower MB, but in this case (IV), the current coding is performed. Since there is only one corresponding macro block of the base layer corresponding to the macro block to be executed, the macro block of the virtual base layer is extracted from the motion information of the corresponding one field MB, as illustrated in FIGS. 10B and 10C without selecting a macro block. The motion information for the pair 1012 is derived and then used for inter-layer motion prediction of the current macroblock pair 1013.

FIG. 11 is a diagram illustrating derivation of a reference index and a motion vector of a macroblock pair of a virtual base layer according to another embodiment of the present invention. In this case, motion information of one macroblock of an even or odd field of the base layer is derived. Only the derivation of the motion information of the macroblock pair of the virtual base layer is different from the above, and the operation applied to the derivation is the same as the case described with reference to FIG. 9A. However, since the upper and lower macro block pairs do not exist in one field of the corresponding base layer BL, the case where the motion information of the macro block pairs is mixed in the case illustrated in FIG. 9B is used in this case (IV). does not apply.

In the embodiment described with reference to Figs. 10A to 10C, when the EL encoder 20 predicts motion information for a macroblock pair of a virtual base layer, the split mode, based on the motion information of the corresponding field MB of the base layer, Reference indexes and motion vectors are derived in this order. However, in another embodiment according to FIG. 11, the reference index and the motion vector are first determined for the macroblock pair of the virtual base layer based on the motion information of the corresponding field MB of the base layer. After the derivation, the partition mode of the macroblock pair of the virtual base layer is finally determined based on the derived result. Finally, when determining the split mode, combine 4x4 block units with the same derived motion vector and the same reference index, and then designate the combined form if the combined block is an acceptable split form, otherwise specify the split form before combining. do.

In the above embodiments, only intra base prediction is performed for the current macro block if the corresponding field MB of the base layer is intra mode. Of course, in this case, the field MB to be used for prediction is upsampled on the vertical axis and then used for texture prediction.

In another embodiment according to the present invention, a virtual macro block is generated from a field MB in the odd or even field to form a macro block pair, and then the macro block pair of the virtual base layer is formed from the configured macro block pair. To derive motion information. 12A and 12B show an example of this.

In this embodiment, the reference index of the corresponding even (or odd) field MB of the base layer and the motion vector are copied (1201,1202) to create a virtual odd (or even) field MB to create a pair of macroblocks 1211. Next, the motion information of the macroblock pair 1212 of the virtual base layer is derived by mixing the motion information of the configured macroblock pair 1211 (1203 and 1204). One example of mixing motion information is, as illustrated in FIGS. 12A and 12B, in the case of a reference index, the upper 8x8 block of the corresponding upper macroblock is the upper macro of the macroblock pair 1212 of the virtual base layer. In the upper 8x8 block of the block, the lower 8x8 block in the upper 8x8 block of the lower macro block, the upper 8x8 block in the corresponding lower macro block is in the lower 8x8 block of the upper macro block of the macro block pair 1212 of the virtual base layer, The lower 8x8 block applies a reference index to the lower 8x8 block of the lower macro block (1203). And, in the case of a motion vector, correspondingly applied according to the reference index (1204), since this can be intuitively understood by Figs. 12A and 12B, description thereof is omitted.

In the case of the embodiment illustrated in FIGS. 12A and 12B, the split mode of the macro block pair 1212 of the virtual base layer is determined based on the derived reference index and the motion vector, and the method is the same as described above.

V. For Field MB-> Field MB

In this case, since the field MB belongs to the field picture or the MBAFF frame, it is divided into four cases as follows.

i) When the base layer and the enhanced layer are MBAFF frames

In this case, as shown in FIG. 13A, as shown, the motion information (dividing mode, reference index, and motion vector) of the macroblock pair of the corresponding base layer BL is used as the motion information of the macroblock pair of the virtual base layer. Copy and use However, copying is made between macro blocks of the same parity. That is, when motion information is copied, even field MB is copied to even field MB and odd field MB is copied to odd field MB.

ii) When the base layer is a field picture and the enhanced layer is an MBAFF frame

In this case, as shown in FIG. 13B, the motion information (segmentation mode, reference index, and motion vector) of one field macroblock of the corresponding base layer BL is shown as shown in FIG. 13B. Copy and use as motion information of a block. In this case, since the motion information of a single field MB is used for both the upper and lower field MBs, the same inter-pattern copying rule does not apply.

iii) the base layer is an MBAFF frame and the enhanced layer is a field picture.

In this case, as shown in FIG. 13C, as shown in FIG. 13C, the field MB of the same parity is selected among the macroblock pairs of the base layer BL corresponding to the current field MB, and motion information (partition mode, reference index, The motion vector) is copied into the motion information of the field macro block of the virtual base layer as it is.

iv) When the base layer and the enhanced layer are field pictures

In this case, as shown in FIG. 13D, as shown, the motion information (dividing mode, reference index, and motion vector) of the field macroblock of the corresponding base layer BL is used as the motion information of the field macroblock of the virtual base layer. Copy and use Even in this case, copying is made between macro blocks of the same parity.

Up to now, the case where the base layer and the enhanced layer have the same resolution has been described. Hereinafter, a case in which an enhanced layer has a higher resolution than a base layer (when SpatialScalabilityType () is greater than 0) will be described according to the picture type (sequential scan frame, MBAFF frame, interlaced field).

A). Base Layer: Sequential Scanning Frame-> Enhanced Layer: MBAFF Frame

Fig. 14A shows the processing method for this case. As shown in the drawing, the texture information and motion information of all macroblocks of the corresponding frame of the base layer are copied to create a virtual frame, and then upsampling is performed on the virtual frame. This upsampling is done at a rate such that the resolution (picture size) of the base layer becomes equal to the resolution (picture size) of the current layer. With this upsampling, the motion information of each macroblock of the upsampled picture is configured based on the motion information of each macroblock of the virtual frame. One of the well-known methods is used for this configuration method. The picture of the interim base layer configured as described above has the same resolution as that of the current enhanced layer picture. Therefore, the above-described inter-layer motion prediction can be applied.

In the case of Fig. 14A, since the base layer is a frame and the current layer is an MBAFF frame, each macro block in the picture becomes a frame MB and a field MB in the MBAFF frame. Therefore, in the above-described case, I is applied to perform inter-layer motion prediction. However, as mentioned above, the MBAFF frame may include not only the field MB pair but also the frame MB pair in the same frame. Therefore, when the macroblock pair in the picture of the temporary base layer and the corresponding macroblock pair of the current layer are not the field MB but the frame MB, the motion prediction method between the known frame MBs, that is, the prediction method of the frame_to_frame ( It is a copy of motion information.)

B). Base Layer: Sequential Scanning Frame-> Enhanced Layer: Interlaced Fields

Fig. 14B shows the processing method for this case. As shown in the drawing, the texture information and motion information of all macroblocks of the corresponding frame of the base layer are copied to create a virtual frame, and then upsampling is performed on the virtual frame. This upsampling is done at a rate such that the resolution of the base layer is equal to the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual frame. Since each macro block in the picture of the provisional base layer configured as described above is a frame MB, and each macro block of the current layer is a field MB in the field picture, inter-layer motion prediction is performed by applying II as described above.

C). Base Layer: MBAFF Frame-> Enhanced Layer: Sequential Scan Frame

Fig. 14C shows the processing method for this case. As shown, first, the corresponding MBAFF frame of the base layer is converted into a frame of a sequential scan method. In order to convert the field MB pair of the MBAFF frame into a sequential scanning method, the above-described method III is applied, and a known frame_to_frame prediction method is applied to the MB pair of frames. Of course, the method of III is to generate the motion information of the virtual frame and each macroblock in the frame by using the data obtained by the inter-layer prediction. It does not perform the operation of coding.

If a virtual frame is obtained, upsampling is performed on the virtual frame. This upsampling is done at a rate such that the resolution of the base layer is equal to the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual frame. Since each macro block in the picture of the provisional base layer configured as described above is a frame MB and each macro block of the current layer is also a frame MB, inter-layer motion prediction of a known frame MB to frame MB is performed.

D). Base layer: Interlaced fields-> Enhanced layer: Sequential scan frame

Fig. 14D shows the processing method for this case. As shown, first, the corresponding field of the base layer is converted into a frame of a sequential scan method. In order to convert an interlaced field into a sequential scan frame, upsampling and IV described above are applied. Of course, the method of IV is also used to generate motion information of a virtual frame and each macroblock within the frame using data obtained by inter-layer prediction. It does not perform the operation of coding.

If a virtual frame is obtained, upsampling is performed on the virtual frame to be equal to the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual frame. Since each macro block in the picture of the provisional base layer configured as described above is a frame MB and each macro block of the current layer is also a frame MB, inter-layer motion prediction of a known frame MB to frame MB is performed.

E). Base Layer: MBAFF Frame-> Enhanced Layer: MBAFF Frame

Fig. 14E shows the processing method for this case. As shown, first, the corresponding MBAFF frame of the base layer is converted into a frame of a sequential scan method. In order to convert an MBAFF frame into a sequential scanning method, the method in case III described above is applied to the field MB pair, and the prediction method of frame_to_frame is applied to the frame MB pair. Of course, the application of the method of III at this time is to generate the motion information of the virtual frame and each macroblock in the frame using the data obtained by the inter-layer prediction. It does not perform the operation of coding.

Once the virtual frame is obtained, upsampling is performed on the virtual frame to match the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual frame. Since each macro block in the picture of the provisional base layer configured as described above is a frame MB and each macro block of the current layer is a field MB in the MBAFF frame, inter-layer motion prediction is performed by applying the above-described method. However, as mentioned above, the MBAFF frame may include not only the field MB pair but also the frame MB pair in the same frame. Therefore, when the macroblock pair of the current layer corresponding to the macroblock pair in the picture of the temporary base layer is not a field MB but a frame MB, a motion prediction method between previously known frame MBs, that is, a prediction method of frame_to_frame (copy of motion information). ).

F). Base Layer: MBAFF Frame-> Enhanced Layer: Interlaced Fields

Fig. 14F shows the processing method for this case. As shown, first, the corresponding MBAFF frame of the base layer is converted into a frame of a sequential scan method. In order to convert an MBAFF frame into a sequential scanning method, the above-described method III is applied to the field MB pair, and the prediction method of frame_to_frame is applied to the frame MB pair. Of course, the method of III is also used to generate motion information of a virtual frame and each macroblock within the frame using data obtained by inter-layer prediction. It does not perform the operation of coding.

Once the virtual frame is obtained, upsampling is performed on the virtual frame to match the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual frame. Since each macroblock in the picture of the provisional base layer configured as described above is a frame MB and each macroblock of the current layer is a field MB in even or odd fields, inter-layer motion prediction is performed by applying the method of II described above.

G). Base layer: interlaced field-> enhanced layer: MBAFF frame Figure 14g shows the processing method for this case. As shown, first, the interlaced field of the base layer is converted into a progressive scan frame. In order to convert an interlaced field into a sequential scan frame, upsampling and IV described above are applied. Of course, the method of IV is also used to generate motion information of a virtual frame and each macroblock within the frame using data obtained by inter-layer prediction. It does not perform the operation of coding.

Once the virtual frame is obtained, upsampling is performed on the virtual frame to match the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual frame. Since each macro block in the picture of the provisional base layer configured as described above is a frame MB and each macro block of the current layer is a field MB in the MBAFF frame, inter-layer motion prediction is performed by applying the above-described method. However, as mentioned above, the MBAFF frame may include frame MB pairs in addition to the field MB pairs in the same frame. In the case of the frame MB, the motion prediction method between frame MBs known in advance, that is, the frame_to_frame prediction method, is applied instead of the aforementioned I prediction method.

H). Base Layer: Interlaced Fields-> Enhanced Layer: Interlaced Fields

Fig. 14H shows the processing method for this case. As shown, a virtual field is created by copying texture information and motion information of all macroblocks of a corresponding field of a base layer, and then upsampling is performed on the virtual field. This upsampling is done at a rate such that the resolution of the base layer is equal to the resolution of the current layer. One of the well-known methods together with this upsampling is used to configure the motion information of each macroblock of the upsampled picture based on the motion information of each macroblock of the virtual field. Since each macro block in the picture of the provisional base layer configured as described above is a field MB in the field picture, and each macro block of the current layer is also a field MB in the field picture, the inter-layer motion is applied by applying the above-described case of V). Make predictions.

The interlayer prediction operation performed by the EL encoder 20 in the apparatus of FIG. 2A or 2B has been described so far. However, the above description of all the inter-layer prediction operations may be equally applied to the decoder of the enhanced layer that receives the decoded information from the base layer and decodes the stream of the enhanced layer. In the encoding and decoding process, the above-described inter-layer prediction operation is performed in the same manner, except that the operations after the inter-layer prediction are different. For example, the encoder performs motion prediction and then codes the predicted information or the difference between the predicted information and the actual information to send to the decoder, but the decoder does not use the same inter-layer motion prediction as performed at the encoder. The only difference is that the actual motion information is obtained by applying the obtained information to the current macro block as it is or by additionally using the motion information of the actually received macro block. Thus, the teachings and principles of the invention described above in terms of encoding apply equally to decoders that decode the data streams of both layers received.

It is to be understood that the present invention is not limited to the above-described exemplary preferred embodiments, but may be embodied in various ways without departing from the spirit and scope of the present invention. If you grow up, you can easily understand. If the implementation by such improvement, change, replacement or addition falls within the scope of the appended claims, the technical idea should also be regarded as belonging to the present invention.

As a limited embodiment of the present invention, at least one embodiment of the present invention enables to perform inter-layer prediction even by using different types of video signal sources. The data coding rate can be increased irrespective of the scanning signal, the progressive scanning signal, the MBAFF frame picture, the field picture, etc.).

Claims (18)

In the method for performing inter-layer motion prediction during encoding / decoding of a video signal, For both vertically adjacent inter mode and intra mode macroblocks of the base layer, information about the motion of the macroblock of the inter mode is set as information about the motion of the macroblock of the intra mode. And the steps Based on the two macroblocks, deriving motion information for a pair of vertically adjacent macroblocks to be used for inter-layer motion prediction. The method of claim 1, The information about the set motion is information about a motion vector and a reference index. The method of claim 1, And the information about the set motion is information about a reference index. The method of claim 3, wherein The setting step further includes setting the macro block of the intra mode to have a motion vector of zero. The method of claim 1, And determining the division mode of both macroblocks using information on the motions of both macroblocks between the setting step and deriving the motion information. The method of claim 1, Wherein both macroblocks are macroblocks for an image of a field component. The method of claim 6, And said vertically adjacent macroblock pair to be used for inter-layer motion prediction is a macroblock pair for an image of a frame component. The method of claim 1, Wherein both macroblocks are macroblocks for an image of a frame component. The method of claim 8, And said vertically adjacent macroblock pair to be used for inter-layer motion prediction is a macroblock pair for a field component image. The method of claim 1, The deriving of the motion information may include deriving motion information of a pair of macroblocks of a virtual intermode from information about motions of both macroblocks; Deriving motion information of the vertically adjacent macroblock pair to be used for inter-layer motion prediction using the motion information of the macro block pair of the virtual inter mode. In the method for performing inter-layer motion prediction during encoding / decoding of a video signal, Setting the interlayer macro blocks in the intra mode and the intra mode to the inter mode in which the information on the motion is zero for the macro blocks in the intra mode; Deriving motion information for a pair of vertically adjacent macro blocks to be used for inter-layer motion prediction based on the two macro blocks. The method of claim 11, The information about the motion set to 0 is information about a motion vector and a reference index. The method of claim 11, And determining the division mode of both macroblocks using information on the motions of both macroblocks between the setting step and obtaining motion information. The method of claim 11, Wherein both macroblocks are macroblocks for an image of a field component. The method of claim 14, And said vertically adjacent macroblock pair to be used for inter-layer motion prediction is a macroblock pair for an image of a frame component. The method of claim 11, Wherein both macroblocks are macroblocks for an image of a frame component. The method of claim 16, And said vertically adjacent macroblock pair to be used for inter-layer motion prediction is a macroblock pair for a field component image. The method of claim 11, The deriving of the motion information may include deriving motion information of a macroblock pair of a virtual inter mode from information on the motions of both macroblocks; Deriving motion information of the vertically adjacent macroblock pair to be used for inter-layer motion prediction using the motion information of the macro block pair of the virtual inter mode.

Images