BRPI0706378A2 - interlayer prediction method for video signal - Google Patents

Abstract

PREDICTION METHOD INTERCEDED FOR VIDEO SIGNAL. The present invention relates to a method for performing interlayer texture prediction in video signal encoding or decoding. The present method builds a pair of frame macroblocks from a single field macroblock or two vertically adjacent field macroblocks from a base layer, and use texture information from the constructed pair of frame macroblocks in predicting the interlayer texture of a pair of frame macroblocks from a current layer.

Description

"INTERMEDIATE PREDICTION METHOD FOR VIDEO SIGNAL"

1. Technical field

The present invention relates to a method for interlayer prediction by encoding / decoding a due signal.

2. Background technique

Scalable video codec (SVC) encodes video in the highest quality image sequence while ensuring that part of the encoded image sequence (specifically, a partial sequence of intermittently selected frames from the full sequence of frames) can be decoded and used to represent video with poor picture quality.

Although it is possible to represent video with low image quality by receiving and processing part of a sequence of images encoded according to the resizable scheme, there is still a problem where the image quality is significantly reduced if the frame rate is reduced. bits is decreased. One solution to this problem is to provide an auxiliary image sequence for low bit rates, for example, a sequence of images having a telephoto size and / or a low frame rate, such as at least one layer in the hierarchical structure.

When it is assumed that two sequences are provided, the auxiliary (lower) image sequence is referred to as a base layer, and the main (upper) image sequence is referred to as an intensified or de-intensifying layer. Base and intensified layer video signals are redundant since the same source of video signals is encoded in two layers. To increase the coding efficiency of the enhanced layer, a signal enhanced layer signal is encoded using coded information (motion information or texture information) of the base layer.

Although a single video source 1 may be encoded in a plurality of layers with different transfer rates, as shown in Figure 1a, a plurality of video sources 2b in different scan modes that contain the same content 2a can be encoded in the respective layers, as shown in Figure Ib. Also in this case, an encoder encoding the upper layer can increase the coding gain by performing interlayer prediction using coded information from the lower layer since the two sources 2b provide the same content 2a.

Thus, it is necessary to provide a method for interlayer prediction by taking into account the video signal-locking modes when coding different sources in the respective layers. When interlaced video is encoded, it may be encoded in odd and even fields and may also be encoded in odd and even macroblock pairs in a frame. Therefore, the types of images for coding an interlaced video signal must also be considered for interlayer prediction.

Generally, the enhanced layer provides higher resolution images than those of the base layer. Therefore, if layer images have different resolutions when different sources are encoded in the respective layers, it is also necessary to perform interpolation to increase the image resolution (ie image size). Since the closer to the base layer image images for use in interlayer prediction are those of the enhanced layer image image for predictable coding, the higher the coding rate, it is necessary to provide a method for interpolation by taking considering the scan modes of the layered video signals.

3. Disclosure of the invention

It is an object of the present invention to provide a method for performing interlayer prediction on the condition that at least one of the two layers has interlaced video signal components.

It is another object of the present invention to provide a method for performing interlayer prediction of motion layered with images of different spatial resolutions (resizing) according to the types of the images.

It is yet another object of the present invention to provide a method for performing interlayer texture prediction of layers having images of different spatial and / or temporal resolutions (resizing).

An interlayer motion prediction method according to the present invention includes defining movement related information for an intramode macroblock for movement related information for an in-term macroblock, the intramode and intermediate macroblocks being two mats. vertically adjacent croblocks of a base layer; and then obtaining motion information from a pair of vertically adjacent macroblocks for use in interlayer motion prediction based on the two vertically adjacent macroblocks.

Another method of predicting interlayer motion according to the present invention includes defining an intramode macroblock, which is one of two vertically adjacent intramode and intermediate macroblocks of a base layer, in an intermediate block having zero motion related information; and then obtaining motion information from a vertically adjacent macroblock pair for use in predicting interlayer movement based on the two vertically adjacent macroblocks.

Another method of predicting interlayer motion according to the present invention includes deriving motion information from a single macroblock from motion information of a pair of quadruvertically adjacent macroblock of a base layer; and using derived motion information as motion information prediction information for a field macroblock in a current layer or respective motion information for a pair of field macroblock in the current layer.

Another method of predicting interlayer motion according to the present invention includes deriving respective motion information from two macroblocks from motion information from a single field-layer macroblock or motion information from a single macroblock. field selected from a pair of vertically adjacent field macroblocks of the base layer; and using the respective motion information derived as prediction information for respective motion information from a pair of frame macroblocks of a current layer.

An interlayer motion prediction method for layers of different resolution images according to the present invention includes transforming an image of a lower layer into a frame image of the same resolution by selectively using frame macroblock prediction methods to according to the image types and types of macroblocks in the images; sample the frame image substantially so as to have the same resolution as that of an upper layer; and then applying an appropriate interlayer prediction method for the frame macroblock types in the following sampled frame image and the macroblock types in an upper layer image.

Another method of interlayer motion prediction for layers of different resolution images according to the present invention includes identifying lower and upper layer image types and / or macroblock types included in the images; apply a method for predicting a pair of frame macroblocks from a single field macroblock in the lower layer image, according to the results identified to construct a virtual image having the same aspect ratio as that of the layer image higher; upwardly sample the virtual image; and then apply interlayer motion prediction to the upper layers using the upwardly sampled virtual image.

Another method of interlayer motion prediction for layers of different resolution images according to the present invention includes identifying lower and upper layer image types and / or macroblock types included in the images; apply a method for predicting a pair of frame macroblocks from a single field macroblock to the bottom layer image, according to the identified results to construct a virtual image having the same aspect ratio as that of the layer image higher; and applying interlayer motion prediction to the upper layer image using the constructed virtual image.

Another method of interlayer motion prediction for layers of different resolution images according to the present invention includes identifying lower and upper layer image types; copy motion information from a block in the lower layer image to construct a virtual image if the lower layer image type for field and the upper layer image type is progressive; upwardly sample the virtual image; and applying a macroblock motion prediction method to frame macroblock between the upwardly sampled virtual image and the top layer image. Another interlayer motion prediction method for layers of different resolution images according to the present invention , includes identifying lower and upper layer image types; copy motion information from a lower layer block to construct a virtual image if the lower layer image type is field and the upper layer image type is progressive; and using the virtual image to apply interlayer motion prediction to the upper layer image.

In one embodiment of the present invention, department modes, reference indices, and motion vectors are sequentially predicted in interlayer motion prediction.

In another embodiment of the present invention, reference indices, motion vectors, and partition modes are sequentially provided.

In another embodiment of the present invention, motion information of a pair of field-based bed macroblocks to be used for prediction of interlayer motion is derived from motion information of a pair of frame macroblocks of the base layer.

In another embodiment of the present invention, motion information of a field macroblock in an even or odd field base image of the virtual base layer to be used for prediction of interlayer motion is derived from motion information of a pair of macroblock. base layer pattern.

In another embodiment of the present invention, a macroblock is selected from a pair of base layer field macroblocks, and motion information from a pair of virtual base layer frame macroblocks to be used for prediction of interlayer motion is derived from one another. movement information from the selected macroblock.

In another embodiment of the present invention, motion information of a pair of frame macroblocks of the virtual base layer to be used for prediction of interlayer motion is derived from motion information of a field macroblock in an image. field has stopped odd from the base layer.

In another embodiment of the present invention, information of a field macroblock in an even or odd base layer field image is copied to further construct a virtual field macroblock and movement information of a pair of frame macroblocks. The virulent base layer to be used for prediction of interlayer motion is derived from motion information of a field macroblock constructed in this manner.

An interlayer texture prediction method according to the present invention includes constructing a field macroblock pair from a pair of vertically adjacent frame macroblocks of a base layer; and using respective texture information from the field macroblock constructed pair as predictive textural prediction information from a field macroblock pair of a current layer.

Another method of interlayer texture prediction according to the present invention includes constructing a single field macroblock from a pair of vertically adjacent frame macroblocks of a base layer; and use texture information from the single camp macroblock as texture prediction information from a field macroblock of a current layer.

Another method of interlayer texture prediction according to the present invention includes constructing a pair of frame macroblocks from a single field macroblock or a pair of vertically adjacent field macroblocks of a base layer; and using respective texture information from the pair of macroblock constructs as respective texture prediction information from a frame macroblock pair of a current layer.

Another interlayer texture prediction method according to the present invention includes constructing N frame macroblock pairs from a vertically adjacent field macroblock pair of a base layer, where N is an integer greater than 1; and using respective texture information from the N constructed frame macroblock pairs as their prediction information for the texture of N frame macroblock pairs located at different time positions in a current layer.

Another interlayer texture prediction method according to the present invention includes separating each frame from a lower layer into a plurality of field images to allow the lower layer to have the same temporal resolution as that of an upper layer. ; sampling each of the separate field images in a vertical direction to extend each of the separate field images in a vertical direction; and then using one of the upwardly sampled field images taken from the interlayer texture measurement of each upper layer frame.

Another method of interlayer texture prediction according to the present invention includes upwardly sampling each field image of a lower layer in a vertical direction to extend each field image in a vertical direction; and utilizing each of the upwardly sampled field images in interlayer texture prediction of each frame of an upper layer.

Another interlayer texture prediction method according to the present invention includes separating each frame from an upper layer into a plurality of field images; downwardly sampling lower layer images to reduce lower layer images in a vertical direction; and then using the descended sampled images in interlayer texture prediction of the separate upper layer field images.

A method of encoding a video signal using interlayer prediction according to the present invention includes determining whether to use interlayer prediction texture, respectively, of 2Nblock texture information constructed by alternately selecting 2Nblock lines. in an arbitrary image of the base layer and then arranging the selected lines in the selected order or respective texture information of 2N blocks constructed by interpolating a selected block from 2N blocks of the base layer; and incorporate information indicating the determination of encoded information.

A method for decoding a video signal using interlayer prediction according to the present invention includes checking whether or not specific indication information is included in a received signal; and determine, based on the result found, whether to use interlayer texture prediction, respective 2N block texture information constructed by alternately selecting 2N block lines in an arbitrary base layer image and then arranging the selected lines in the order. - must be selected or its 2Nblock texture information constructed by tweening a selected block from the 2N blocks of the base layer.

In one embodiment of the present invention, each frame of the upper or lower layer is separated into two field images.

In one embodiment of the present invention, if specific indication information is not included in the received signal, that case is considered to be equal as where a signal that includes the zero-defined indication information has been received and blocks whose respective texturess information is to be used. in interlayer prediction, are determined.

One method for utilizing an interlayer texture prediction base layer video signal according to the present invention includes separating a video interlaced signal from the base layer into even and odd field components; increasing each of the even field components and offset a vertical and / or horizontal direction; and then combining and using the even and odd field components increased by the interlayer texture measurement.

Another method for using an interlayer texture predictive base layer video signal according to the present invention includes separating a progressive video signal from the base layer into an even line group and an odd line group; increasing each of the odd and even line groups in a vertical and / or horizontal direction; ecombinate and use the increased even and odd line groups combined in interlayer texture prediction.

Another method for using an interlayer texture predictive base layer video signal in accordance with the present invention includes increasing an interlaced base layer video signal in a vertical / or horizontal direction to have the same resolution as that an upper layer progressive video signal; and performing interlayer texture prediction of the top layer video signal based on the augmented video signal.

Another method for utilizing an interlayer texture predictive base layer video signal in accordance with the present invention includes increasing a progressive base layer video signal in a vertical / or horizontal direction to have the same resolution as that of an interlaced video signal of an upper layer; and perform interlayer texture prediction of the upper layer video signal based on the augmented video signal.

In one embodiment of the present invention, video signal separation and augmentation are performed on a macroblock level (or on a macroblock basis).

In another embodiment of the present invention, video signal separation and enhancement are performed at an image level.

In another embodiment of the present invention, video signal separation and enhancement are performed if two-layer image formats to which interlayer texturing prediction is to be applied are different, that is, one layer includes progressive images and the other includes an image. interlocking edges.

In another embodiment of the present invention, video signal separation and enhancement are performed if the two two-layer images to which interlayer texture prediction is to be applied are interlaced.

4. Brief Description of Drawings

Figures 1a and Ib illustrate methods of encoding a single video source in a plurality of layers;

Figures 2a and 2b briefly show the configuration of a video signal coding apparatus to which an interlayer prediction method according to the present invention is applied;

Figures 2c and 2d show the types of image sequences for encoding an interlaced video signal;

Figures 3a and 3b schematically show a procedure in which a base layer image is constructed and unlocking filtration is performed for predicting interlayer texture according to one embodiment of the present invention;

Figures 4a to 4f schematically show a procedure in which motion information of a field macrob-block of a virtual base layer to be used for prediction of interlayer motion of a field-macroblock in an MBAFF frame is derived. movements of a frame macroblock according to one embodiment of the present invention;

Figure 4g shows schematically a procedure in which texture information from a pair of macroblock is used to predict texture of a pair of field macroblocks in an MBAFF frame according to a modality. of the present invention;

Fig. 4h illustrates a method for transforming a pair of frame macroblocks into a pair of field macroblocks according to one embodiment of the present invention;

Figures 5a and 5b illustrate a reference index and motion information derivation procedure according to another embodiment of the present invention;

Figures 6a-6c schematically illustrate the procedures in which motion information of a field macroblock in a virtual base layer is derived using motion information of a frame macroblock, in accordance with one embodiment of the present invention;

Fig. 6d schematically shows a procedure in which texture information from a pair of frame macroblocks is used to predict texture of a field macroblock in a field image according to one embodiment of the present invention;

Figures 7a and 7b illustrate a motion information derivation and reference index procedure according to another embodiment of the present invention;

Figures 8a-8c schematically show procedures in which motion information of a field macroblock frame macroblock of a basevirtual layer to be used for interlayer motion prediction is derived using motion information from a macroblock. field data in an MBAFF frame according to one embodiment of the present invention;

Fig. 8d schematically shows a procedure in which texture information of a pair of field macroblocks in an MBAFF frame is used to predict the texture of a pair of frame macroblocks according to one embodiment of the present invention;

Figure 8e shows a method for transforming a field macroblock pair into a frame macroblock pair according to one embodiment of the present invention;

Figures 8f and 8g schematically show procedures in which texture information of a pair of field macroblocks in an MBAFF frame is used in interlayer prediction of a pair of frame macroblocks when only one of the pair of macroblocks. Fig. 8h shows schematically a procedure in which texture information of a pair of field macroblocks in an MBAFF frame is used to predict the texture of a plurality. of frame macroblock pairs according to one embodiment of the present invention;

Figures 9a and 9b illustrate procedures for deriving motion information and reference index according to another embodiment of the present invention;

Figures 10a to 10c schematically show procedures in which motion information of a frame macroblock of a virtual base layer to be used for interlayer motion prediction is derived using motion information of a field macroblock in a field image according to one embodiment of the present invention;

Figure 10d schematically shows a procedure in which texture information of a field macroblock in a field image is used to predict the texture of a frame macroblock pair according to one embodiment of the present invention;

Fig. 11 illustrates a motion information and reference index deriving procedure according to another embodiment of the present invention;

Figures 12a and 12b schematically show the procedures in which motion information of a frame macrob-locus of a virtual base layer to be used for interlayer motion prediction is derived using motion information of a field macroblock in a motion picture. field, according to another embodiment of the present invention;

Figures 13a through 13d schematically show, separately, according to image types, procedures in which motion information of a field macroblock of a virtual base layer to be used for interlayer motion prediction is derived using motion information of a field macroblock according to one embodiment of the present invention;

Figures 14a-14k show, separately according to image types, methods for performing interlayer motion prediction when spatial layer resolutions are different according to a variety of embodiments of the present invention;

Figures 15a and 15b schematically show procedures in which an image of the base layer with a different spatial resolution is used to predict interlayer textures when the enhanced layer is progressive and the base layer is interlaced according to one embodiment. present invention;

Figures 16a and 16b schematically show the procedures in which, to use a layer image based on interlayer texture prediction, a pair of macroblocks in the image are separated into macroblocks and the separated macroblocks are enlarged by according to one embodiment of the present invention;

Figures 17a and 17b schematically show the procedures in which an image of the base layer with a different spatial resolution is used to predict interlayer textures when the enhanced layer is interlaced and the base layer is progressive according to one embodiment of the present invention;

Figure 18 schematically shows procedures in which an image of the base layer with a different spatial resolution is used in interlayer prediction when both the intensified and base layers are interlaced, according to one embodiment of the present invention;

Fig. 19a illustrates a procedure in which interlayer prediction is applied when the enhanced layer is a progressive frame sequence and the types of temporal resolution of the two layers are different, according to one embodiment of the present invention;

Fig. 19b illustrates a procedure in which interlayer prediction is applied when the enhanced layer is a progressive frame sequence and the two layers have different image types and the same resolution according to one embodiment of the present invention;

Fig. 20 illustrates a procedure in which interlayer prediction is applied when the base layer is a progressive frame sequence and the image types and temporal resolutions of the two layers are different, according to one embodiment of the present invention; and

Fig. 21 illustrates a procedure in which interlayer prediction is applied when the base layer is a progressive frame sequence and the two layers have different image types and the same resolution according to one embodiment of the present invention.

5. Modes for Carrying Out the Invention

Embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

Figure 2a schematically illustrates the building blocks of a video signal coding apparatus to which an interlayer prediction method according to the present invention is applied. Although the apparatus of FIG. 2a is implemented to encode a two-layer input video signal, principles of the present invention described below may also be applied to interlayer processes even when a video signal is encoded on three or more layers.

The interlayer prediction method according to the present invention is performed on an intensified layer encoder (EL) 20 in the apparatus of Figure 2a. Encoded information (motion information and texture information) is received in a base layer (EL) encoder 21. Interlayer texture prediction or motion prediction is performed based on the information received. If necessary, the information received is decoded and the prediction is executed based on the decoded information. Of course, in the present invention, an input video signal may be encoded using a base layer video source 3 which has already been encoded as shown in Figure 2b. The same interlayer prediction method, as described below, is applied in this case.

In the case of Fig. 2a, there may be two methods in which the BS encoder 21 encodes an interlaced video signal or in which the encoded video source 3 of Fig. 2b has been encoded. Specifically, in either method, the interlaced video signal is simply encoded in a field sequence on a field-by-field basis, as shown in Figure 3a, and in the other method, frames are encoded in a frame sequence. by constructing each frame of the sequence with two-field macroblock pairs (even and odd) as shown in Figure 3b. The upper macroblock of a pair of macroblocks in a coded frame of this manner is referred to as an "upper macroblock" and the lower one is referred to as "lower macroblock". If the upper macroblock is composed of an even (or odd) field image component, the lower macroblock is composed of an odd (or even) field image component. A frame constructed in this way is referred to as a MacroBlock Adaptable Frame Field (MABFF) frame. The MBAFF frame can include not only pairs of macroblocks, each including odd and even field macroblocks, but also pairs of macroblocks, each including two frame macroblocks.

Therefore, when a macroblock in an image has an interlaced image component, it can be a macroblock in a field and can also be a macroblock in a frame. Each macroblock having an interlaced image component is referred to as a field macroblock, while each macroblock having a progressive (scan) image component is referred to as a frame macroblock.

Thus, it is necessary to determine an interlayer prediction method by determining whether each type of macroblock to be encoded in the EL 20 encoder and a base layer macroblock to be used in macroblock interlayer prediction is a type of macroblock. frame or type of field macroblock. If the macroblock is a field macroblock, you must determine the interlayer prediction method by determining whether it is a field macroblock in a field or in an MBAFF frame.

The method will be described separately for each case. Prior to the description, it is assumed that the resolution of the current layer is equal to that of the base layer. That is, it is assumed that PatialScalabilityTypeO is zero. A description when the resolution of the current layer is higher than that of the base layer will be provided later. In the following description and drawings, the terms "upper" and "even" (or odd) are used interchangeably and the terms "lower" and "odd" (or even) are used interchangeably.

In order to perform interlayer prediction to encode or decode the enhanced layer using the base layer, it is first necessary to decode the base layer. Thus, the decoding of the base layer is first described as follows.

Not only is the layer movement information such as partition modes, reference indices, and motion vectors, but the base layer texture is decoded when the base layer is decoded.

When the base layer texture is decoded for interlayer texture prediction, all base layer image sample data is not decoded to reduce the decoder load. Intramode macroblock image sample data is decoded as residual data, which is error data between image sample data, intermodal macroblock is decoded without motion compensation with adjacent images.

In addition, the texture decoding of the base layer for interlayer texture prediction is performed on an image-by-image basis rather than on a macroblock macroblock base to construct base layer images that are temporally coincident with intensified bed. A base layer image is constructed from reconstructed image sample data from intramode macroblocks and decoded residual data from intermodal macroblocks, as described above.

Intramode or intermediate motion compensation and transformation such as DCT and quantization are performed on an image block basis, for example, on a 16x16 macrob-locus basis or on a 4x4 sub-block basis. This causes blocking artifacts on block boundaries to distort images. Unblock filtration is applied to reduce these lock artifacts. The unlock filter smooths the edges of image blocks to improve the quality of video frames.

Whether or not to apply unblocking filtration to reduce blocking distortion depends on the image block intensities at the boundaries and depixel gradients around the boundaries. The intensity or level of the unlocking filter is determined from a quantization parameter, an intramode, an intermediate, an image block partition mode indicating block size or the like, a motion vector, a pixel value before unlocking filters, and the like.

The interlayer prediction unlock filter is applied to an intramode macroblock in a layered image that is a basis for texture prediction of an enhanced intrabase (intraBL or intralayer mode) macroblock.

When two layers to be encoded, according to an interlayer prediction method, are all encoded in a field image sequence, as shown in Figure 2c, the two layers are all considered as frame format so that from the coding processes for the frame format, it is possible to easily enable coding / decoding processes including unlock filtering.

A method for performing unlocking filtration according to one embodiment of the present invention will now be described for the case where the base layer image format is different from that of the enhanced layer, i.e. hollow where the base layer is shaped. frame (or progressive) and the base layer is in field (or interlaced) format, the case where the base layer is in field format and the base layer is in frame format, or the case of one of the layers enhanced and base was encoded in one field image sequence and the other was encoded in an MBAFF frame although both the enhanced and base layers are in field format, as shown in figures 2c and 2d.

Figures 3a and 3b schematically illustrate procedures in which a base layer image is constructed to perform interlayer prediction unlock filtering according to embodiments of the present invention.

Figure 3a illustrates one embodiment wherein the intensified layer is in frame format and the base layer is in field format and Figure 3b illustrates one embodiment where the base layer is in field format and the base layer is in frame format.

In such embodiments, for interlayer texture prediction, the texture of an intermediate macroblock and an intramode macroblock of the base layer is decoded to construct a base layer image including image sample data and residual data, and the constructed image is The sample is descended according to the ratio of the resolution (or screen size) of the base layer to that of the tensified layer after applying an image unlocking filter constructed to reduce blocking artifacts.

The first method (method 1) in Figures 3a and 3b is a method in which a base layer is separated into two field images to perform unlock filtering. In this method, when an enhanced layer is created using a base layer encoded in a different image format, a base layer image is divided into an even line field image and an odd line field image and the two field images are divided. unlocked (ie filtered to unlock) and sampled upwards. The two images are then merged into a single image and predictive textureless prediction is performed based on the single image.

The first method includes the following three steps.

In a separation step (step 1), a base layered image is separated into an upper field (or odd field) image including even lines and a lower field (or even field) image including odd lines. The base layer image is a video image including residual data (intermediate data) and image sample data (intramode data) that is reconstructed from a base layer data stream by motion compensation.

In an unlock step (step 2), split field images in the separation step are unlocked through an unlock filter. Here, a conventional unlocking filter can be used as the unlocking filter.

When the resolution of the enhanced layer differs from that of the base layer, the images of unblocked camps are sampled upwards according to the ratio between the resolution of the enhanced layer and that of the base layer.

In a junction step (step 3), the upwardly sampled upper field image and the downwardly sampled lower field image are interlaced in an alternate mode to be joined into a single image. Subsequently, texture prediction of the enhanced layer is performed based on the single image.

In the second method (method 2) in Figures 3a and 3b, when an enhanced layer is created using a base layer encoded in a different image format, a base layer image is directly unlocked and sampled without even dividing it into two images. interlayer texture prediction and field is performed based on the resulting image.

In the second method, a base layer image, which corresponds to an enhanced layer image to be encoded by interlayer texture prediction, is immediately blocked without dividing it into upper and lower field images and is then sampled upward. you. Subsequently, texture prediction of the enhanced layer is performed based on the upwardly sampled image.

An unlock filter for application to a base layer image constructed for interlayer motion prediction is applied only to an area that includes image sample data decoded from intramode macroblocks, without being applied to an area that includes residual data.

In the case where the base layer was encoded in field format in figure 3a, ie where the base layer was encoded in a field image sequence as shown in figure 2c or in an MBAFF frame as shown In Figure 2d, to apply the second method, it is necessary to perform a process of alternately interlacing upper and lower field image lines to combine them into a single image (in the case of Figure 2c) or alternately interlacing image lines. upper and lower macroblocks from field macroblocks to combine them into a single image (in the case of figure 2d). This process will be described in detail with reference to Figures 8d and 8e. The upper and lower field images or upper and lower macroblocks to be interlaced are field or macroblock images including residual data (intermediate data) and reconstructed image sample data (intramode data) through image compensation. movement.

Moreover, in the case where upper and lower macroblocks of the field macroblock pairs (of the base layer) in an MBAFF frame, as shown in figure 2d, are of different ways and, from these macroblocks, intramode blocks are selected to be used for prediction. interlayer texture of enhanced layer macroblock pairs (in the case of Figure 8g described later), in the case of any frame (image) in the base layer encoded by field macroblock pairs in an MBAFF frame, as shown in Figure 2d, not temporally coincident with intensified layer image (in the case of figure 8h described later), or in the case where texture of a intensified layer of macroblock pairs is predicted from a field image field macroblock base layer , as shown in Fig. 2c (in the case of Fig. 10d described below), a macroblock selected from the field macroblock is sampled upwards to a temporary pair. and macroblocks ("841" in FIG. 8g and "851" and "852" in FIG. 8h) or two temporary macroblocks ("1021" in FIG. 10d) and an unlock filter is applied to intramodern macroblocks of these macroblocks.

The interlayer texture prediction, described in a variety of the following embodiments, is performed with only unlocked base layer images described in the fashion of Figures 3a and 3b.

An interlayer prediction method will now be described separately for each case classified depending on the type of macroblock in the current layer to be encoded and the type of base layer macroblock to be used for interlayer prediction of the current layer macroblocks. In the description, it is assumed that the spatial resolution of the current layer is equal to that of the base layer, as described above.

I. The Case of MB Frame -> HB Field MBAFF Frame

In this case, macroblocks in the current layer (EL) were encoded into field macroblocks in an MBAFF frame and macroblocks in the base layer to be used for interlayer prediction of the current layer macroblocks were encoded in frame macroblocks. Video signal components included in both the upper and lower macrolocks in the base layer are the same as those included in a pair of macroblocks co-located in the current layer. The upper and lower (upper and lower) macroblocks will be referred to as a pair of macroblocks and the term "even" will be used to describe a pair of vertically adjacent blocks in the following description. First, the interlayer motion prediction is described as follows.

The EL 20 encoder uses, as current macroblock partitioning modes, macroblock partition modes obtained by merging a pair of macroblock 410s from the base layer into a single macroblock (by compressing up to half the size in a vertical direction). . Figure 4a illustrates a detailed example of this process. As shown, first, a corresponding pair of macroblock 410 from the base layer is fused to a single macroblock (S41) and fusion macroblock partitioning modes are copied to each other to construct a pair of macroblock 411 (S42). Subsequently, respective partitioning modes of the macroblock pair 411 are applied to a macroblock pair 412 of a virtual base layer (S43).

However, a partition area that is not allowed in partition mode can be generated when the corresponding macroblock pair 410 is fused into a single macrob-locus. To avoid this, the EL 20 encoder determines the partition mode according to the following rules.

1) the upper and lower 8x8 blocks ("B8_0" and "B8_2" in Figure 4a) in a pair of basement layer macroblocks fused into a single 8x8 block. However, if any of the corresponding 8x8 blocks have not been subdivided, they are merged into two 8x4 blocks, and if any of the corresponding 8x8 blocks have been subdivided, they are merged into four 4x4 blocks ("401" in Figure 4a) .2) one block 8x16 of the base layer is reduced to an 8x8 block, a 16x8 block is reduced by two adjoining 8x4 blocks, and a 16x16 block is reduced to a 16x8 block.

If at least one of the corresponding macroblock pair has been encoded intramode, the EL 20 encoder first performs the following processes prior to the fusion process.

If only one of the two macroblocks is intramed, inter macroblock motion information such as macroblock partition modes, Reference Indices, and motion vectors are copied to the intra macroblock as shown in Figure 4b, or the macroblock intra is considered as an inter 16x16 macroblock with zero motion vectors and zero reference indices, as shown in figure 4c. Alternatively, as shown in Figure 4d, intra macroblock reference indices are defined by copying those from the inter macroblock to the intra macroblock and zero-movement vectors are allocated to the intra macroblock. Next, the aforementioned fusion process is performed and a motion vector and reference index derivation procedure is then performed as described below.

The EL 20 encoder performs the following process to derive reference indices from the current macroblock pair 412 from those of the corresponding macroblock pair 410.

If each of a pair of 8x8 blocks of the baseline layer corresponding to the current 8x8 block has been subdivided into the same number of parts, the reference index of one (the upper or lower block) of the 8x8 block pair is determined as the reference index of the 8x8 block. 8x8 current block. Otherwise, the reference index of one of the 8x8 block pair, which has been subdivided into the smallest number of parts, is determined as the reference index of the current 8x8 block.

In another embodiment of the present invention, the smallest reference indexes defined for a pair of base layer 8x8 blocks corresponding to the current 8x8 block is determined to be the current 8x8 block reference index. This method of determination in the example of Figure 4e can be expressed as follows:

CurrB8_0 refidx = min (top frame refidx MB's B8_0, top frame refidx MB's B8_2)

CurrB8_l refidx = min (top frame refidx base MB's B8_l, top frame refidx base MB's debase B8_3)

CurrB8_2 refidx = min (lower frame refidx base MB's B8_0, lower frame refidx base MB's B8_2), and

CurrB8_3 refidx = min (bottom frame refidx MB's B8_l, bottom frame refidx MB's B8_3) ·

The above reference index derivation procedure can be applied to both upper and lower field macroblocks. The reference index of each block 8x8 thus determined is multiplied by 2 and the multiplied reference index is determined as its final reference index. The reason for this multiplication is that, when decoded, the number of images is twice that of the frame sequence since field macroblocks are intended for images divided into even and odd fields.

Depending on the decoding algorithm, the final reference index of the lower field macroblock can be determined by multiplying its reference index by 2 and then adding 1 to the multiplied reference index.

The following is a procedure in which the EL 20 encoder derives motion vectors from a pair of macroblock loci of the virtual base layer.

Motion vectors are determined on a base 4x4 block and thus a corresponding 4x8 block of the base layer is identified as shown in figure 4f. If the corresponding 4x8 block was subdivided, a motion vector of its upper or lower 4x4 block is determined as that of the current 4x4 block. Otherwise, a motion vector of the corresponding 4x8 block is determined as that of the current 4x4 block. The determined motion vector whose vertical component is divided by 2 is used as a final motion vector of the current 4x4 block.

The reason for this division is that an image component included in two frame macroblocks corresponds to an image component of a field macroblock so that the size of the field image is halved in one direction. vertical action.

After determining field macroblock dopple motion information 412 of the virtual base layer in this manner, motion information is used for interlayer motion prediction of a target pair of field macroblock 413 of the enhanced layer. Also in the following description, after determining motion information of a macroblock or a pair of virtual base layer macroblocks, motion information is used to predict interlayer movement of a corresponding macroblock or a corresponding pair of macroblock of the current layer. In the following description, it is assumed that this process is applied even without mentioning that the motion information of a macroblock or a pair of virtual base layer macroblock is used to predict interlayer motion of a corresponding macroblock or a corresponding pair of macroblocks of the current layer.

Fig. 5a schematically illustrates how motion information of a pair of field macroblocks 500 of a virtual base layer to be used for interlayer prediction is derived from motion information of a pair of base layer frame macroblocks corresponding to the current pair of macroblocks according to another embodiment of the present invention. In this embodiment, as shown, a reference index of an upper or lower 8x8 block of the upper macroblock of the base layer macroblock pair is used for a reference index of an upper 8x8 block of each of the macroblock pair. of the virtual base layer 500 and a reference index of an upper or lower 8x8 block of the base layer lower macroblock is used for a reference index of a lower 8x8 block of each of the field macroblock pair 500. On the other hand, as shown, a upper 4x4 block motion vector of the upper macroblock of the base layer frame macroblock pair is commonly used for the upper 4x4 oblique of each of the virtual base layer field-500 macroblock pair, a third 4x4 block motion vector of the upper macroblock of the base layer frame macroblock pair is commonly used for the second 4x4 block of each of the field macroblock pair 500, a vector upper 4x4 block motion of the lower macroblock pair of macroblock frame pair is commonly used for the third 4x4 block of field macroblock pair 500, and a third vector block 4x4 movement vector of the lower macroblock of base layer frame macroblocks is commonly used for the fourth 4x4 block of each of the field macroblock pair 500.

As illustrated in Figure 5a, an upper 4x4 block 501 and a lower 4x4 block 502 in an 8x8 block in the field macroblock pair 500 constructed for use in inter-layer prediction utilize the different layer 4x4 block motion vectors 511 and 512 of the layer. base. These motion vectors can be motion vectors that use different reference images. That is, 8x8 blocks 511 and 512 may have different reference indices. Therefore, in that case, to construct the virtual base layer macroblock pair 500, the EL encoder 20 commonly uses a motion vector of a corresponding 4x4 block 503 selected for upper 4x4 block 501 as a motion vector of the second 4x4 block 502. of the virtual base bed, as shown in figure 5b (521).

In the embodiment described with reference to FIGS. 4a to 4f, to construct motion information from the virtual base layer to predict motion information of the current macroblock pair, the EL 20 encoder sequentially derives partition modes, reference indices, and motion vectors with based on motion information of a corresponding portion of the base layer macroblocks. However, in the embodiment described with reference to Figures 5a and 5b, the EL 20 coder first derives motion-evolving reference indices from a pair of basevirtual layer macroblocks based on motion information from a corresponding pair of macroblock layers. base and then finally determines partition modes of the virtual base layer macroblock pair based on the derived values. When department modes are determined, 4x4 block units with the same derived motion vectors and reference indices are. combined mode, and if the combined block mode is a permitted partition mode, the partition modes are set to the combined mode, otherwise the partition modes are set to those before the combination.

In the embodiment described above, if both macroblock of the corresponding pair of macroblock 410 of the base layer were intramode, intrabase prediction alone is performed on the current macroblock 413. In this case, motion prediction is not performed. Of course, a pair of macroblaces of the virtual base layer is not constructed in the case of texture prediction. If only one of the corresponding pair of macroblocks 410 of the base layer is intramode, inter macroblock motion information is copied to the intra macroblock as shown in Figure 4b, intra macroblock motion vectors and reference indices are set to zero, as shown in figure 4c, or intra macroblock reference indices are set by copying those from inter macroblock to the intra macroblock and intra macroblock motion vectors are set to zero, as shown in figure 4d. Next, motion information of the virtual base layer macroblock pair is derived as described above.

After constructing the virtual base layer macroblock pair for interlayer motion prediction as described above, the EL 20 encoder predicts and encodes motion information for the current field macroblock pair 413 using motion information from the constructed pair of ma-croblocos.

Prediction of interlayer texture will be described now. Figure 4g illustrates an example interlayer text prediction method in the case of "Frame MB -> MB MBAFF Frame Field". The EL 20 encoder identifies block modes of a corresponding pair of base layer frame macroblocks 410. If both of the corresponding frame macroblock pair 410 are intra or intermediate, the EL 20 coder converts (or transforms) the corresponding base layer macroblock 410 pair to a temporary field macroblock pair 421 to perform dopar intrabase prediction. field macroblock 413 (when both frame macroblocks 410 are intramode) or to perform their residual prediction in the manner described below (when both frame macroblocks 410 are intermediate). When both corresponding macroblock dopers 410 are intramode, temporary field macroblock 421 includes unblocked (i.e. filtered for unlocking) data after completing decoding in the case of intramode as described above. The same is true for a temporary pair of macroblocks derived from base layer macroblocks for use in texture prediction in the following description of a variety of embodiments.

However, interlayer texture prediction is not performed when only one of the two macroblocks is inter-mode. The base layer macroblock pair 410 for interlayer texture use has original unencoded image data (or decoded image data) if the macroblocks are intramode and have encoded residual data (or decoded residuals) if the macroblocks are of intermediate. The same is true for a couple of base layer macroblocks in the following description of texture prediction.

Figure 4b shows a method for converting a pair of frame macroblocks into an aser field macroblocks used for interlayer texture prediction. As shown, even lines from a pair of A and B frame macroblocks are sequentially selected to construct an upper-field macroblock A 'and odd lines from the AeB frame macroblock pair are sequentially selected to construct a lower-field macroblock B'. . When a field macroblock is filled with lines, it is first filled with odd (or odd) lines from the upper block (A_even or A_odd) and is then filled with odd (or even) lines from the lower block B (B_even or B_odd).

II. 0 frame MB case -> MB field in field image

In this case, macroblocks in the current layer were coded into field macroblocks in a field image and macroblocks in the base layer to be used for inter-layer prediction of macroblocks in the current layer were encoded in frame macroblocks. Video signal components included in a pair of macroblock in the base layer are the same as those included in a macroblock co-located in a campopar or odd in the current layer. First, the interlayer prediction is described as follows.

The EL 20 encoder uses, as partitioning methods of an even or odd macroblock of the virtual base layer, macroblock partition modes obtained by merging a pair of base layer macroblocks into a single macroblock (through compression to half of size in a vertical direction). Figure 6a illustrates a detailed example of this process. As shown, first, a corresponding pair of base layer macroblocks 610 is fused into a single macroblock 611 (S61) and partition modes obtained by this fusion are applied to a virtual base layer macroblock to be used for motion prediction. mats of the current ma-crobloco 613 (S62) Fusion rules are the same as those obtained in the previous case I. The processing method when at least one of the corresponding pair of intramodified macroblocks 610 is the same as that of the previous case I.

The procedure for deriving reference indices and motion vectors is also performed in the same manner as described above in the previous case I. In case I, the same derivation procedure is applied to the upper and lower macroblocks as pairs of Even and odd macroblocks are carried in a frame. However, this case II is different from case II where the derivation procedure is applied only to a field macroblock, as shown in figures 6b and 6c since only one macroblock corresponding to the base layer pair 610 is present in the image. current field to be encoded.

In the above embodiment, to predict moving information of a virtual base layer macroblock, the EL 20 coder sequentially derives macroblock partition modes, reference indices, and motion vectors based on motion information of a corresponding pair. base layer demacroblocks.

In another embodiment of the present invention, the EL'20 coder first derives motion-evolving reference indices from a virtual base layer macroblock based on motion information from a corresponding pair of base layer macroblocks, and then finally determines. in the macroblock block modes of the virtual base layer combines the derived values. Figures 7a and 7b schematically illustrate derivation of reference indices and motion vectors of a virtualized base layer field macroblock. Derivation operations in this case are similar to those in Case I described with reference to Figures 5a and 5b, except that upper or lower macroblock motion information is derived using motion information from a base layer macroblock pair.

When partition modes are finally determined, 4x4 block units with the same derived motion vectors and reference indices are combined, and if combined block mode is a permitted partition mode, partition modes are set to the combined mode, otherwise the partition modes are set to those before the combination.

In the embodiments described above, if both of the corresponding base layer macroblock parcel are intra-mode, motion prediction is not performed on virtual base layer macroblock pair motion information either, and if only one of the two macroblock -locations are intramode, motion prediction is performed as described earlier in this case.

The prediction of interlayer texture will now be described. Figure 6d illustrates an example interlayer text prediction method in the case of "Frame MB -> MB Field Image Field". The EL 20 encoder identifies block modes of a corresponding pair of base layer macroblocks 610. If both of the macroblock pair are in between or in between, the EL 20 encoder constructs a temporary field macroblock 621 from the single frame macroblock pair 610. If the current macroblock 613 belonged to an even field image, the EL 20 encoder constructs the temporary field macroblock 621 from matched lines of the corresponding pair of macroblocks 610. If the current macroblock 613 belongs to an odd field image, the EL 20 encoder constructs the temporary field macroblock 621 from odd lines. corresponding pair of macroblock 610. The construction method is similar to the method of constructing the single field macroblock A 'or B' in FIG. 4h.

After construction of the temporary field macroblock 621, the EL 20 encoder performs intrabase prediction of the current field macroblock 613 (when both of the corresponding pair of macroblock 610 are intramode) based on texture information in the field macroblock 621 or to perform their residual prediction (when the two of the corresponding pair of macroblock 610 are in between).

The EL 20 encoder does not perform interlayer text prediction if only one of the corresponding pair of macroblocks 610 is of intermediate.

III. 0 MB case in frame MBAFF -> MB in frame

In this case, macroblocks in the current layer were coded into frame macroblocks and macroblocks in the base layer to be used for interlayer prediction of current layer macroblocks were encoded in field macroblocks in an MBAFF frame. Video signal components included in a field macroblock in the basal layer are the same as those included in a pair of macroblock co-located in the current layer. First, the interlayer prediction is described as follows.

The EL 20 encoder uses, as partitioning modes of a macroblock pair in a virtual base layer, macroblock partition modes obtained by extending the upper or lower macroblock of the layered macroblock pair (twice in two layers). one vertical direction). Figure 8a illustrates a detailed example of this process. Although the upper field macroblock is selected in the following description and drawings, the same as described below applies when the lower field macroblock is selected.

As shown in Figure 8a, a superior field macroblock in a corresponding pair of base layer macroblock 810 is extended twice to construct two macroblocks 811 (S81) and partition modes obtained by extending are applied to one. pair of macroblock 812 of the virtual layer (S82).

However, a partition mode (or default) that is not allowed in macroblock partition modes can be generated when the corresponding field macroblock is extended twice in a vertical direction. To avoid this, the EL 20 encoder determines partition modes depending on extended partition modes according to the following rules.

1) 4x4, 8x4, and 16x8 blocks of the base layer after tensioning are determined to be 4x8, 8x8, and 16x16 blocks obtained by increasing them twice in one vertical direction.

2) each of the 4x8, 8x8 and 16x16 layers of the base layer after extension is determined to be two upper and lower blocks of the same size. As illustrated in Fig. 8a, an 8x8 B8_0 block of the base layer is determined as two 8x8 blocks (801). The reason that the 8x8B8_0 block after extension is not defined in an 8x16 block is that the adjacent extended sewer on the left or right side cannot be an 8x16 partition block and no macroblock partition mode is supported in this case.

If one of the corresponding pair of macroblocks 810 has been encoded intramode, the EL 20 encoder selects the upper or lower field macroblock from intramode rather than intramode and performs the above extension process thereon to determine partition modes of an 812 macroblock pair in the virtual base layer.

If both of the corresponding pair of macroblocks810 are intramode, the EL 20 encoder performs individually interlayer texture prediction without performing partition mode determination via the above extension process and a motion vector derivation process and reference described below.

To derive reference indexes from the virtual base layer macroblock pair from reference indices of a corresponding field macroblock, the EL 20 coder determines the reference index of a corresponding base layer 8x8 block B8_0 as being the reference index of each of the two upper and lower 8x8 blocks, as illustrated in Figure 8b, and dividing the determined reference index of each 8x8 block by 2 to obtain its final reference index. The reason for this division is that, for application in a frame sequence, it is necessary to reduce the image number in half since field macroblock reference image numbers are defined based on images divided into even and odd fields.

By deriving motion vectors from a pair of 812 frame macroblocks of the virtual base layer, the EL 20 encoder determines the motion vector of a corresponding 4x4 block of the base layer as the motion vector of a 4x8 block. in the macroblock pair 812 of the basevirtual layer, as illustrated in FIG. 8c, and uses, as a final motion vector, the motion vector determined whose vertical component is multiplied by 2. The reason for this multiplication is that a component of The image included in a field macroblock corresponds to an image component of two frame macroblocks so that the size of the frame image is increased twice in a vertical direction.

In the embodiment described above, to predict motion information for a pair of basevirtual layer macroblocks, the EL 20 encoder sequentially derives macroblock department modes, reference indices, and motion vectors based on motion information from a macroblock. corresponding field of the base layer.

In another embodiment of the present invention, by deriving motion information from a pair of macroblocks of the virtual base layer to be used for inter-frame prediction of the current pair of macroblocks, the EL 20 encoder first obtains reference indices and vector vectors. doping motion of the virtual base layer macroblocks based on movement information of a field macroblock corresponding to the base layer and then finally determining the block mode of each of the virtual base layer macroblock pair based on the values obtained as illustrated. in figure 9a. When all partitions are finally determined, 4x4 block units with the same derived motion vectors and reference indices are combined, and if block mode is a permitted partition mode, department modes are set to combined mode. Other modes and partition modes are set to those before the combination.

The following is a more detailed description of the embodiment of figure 9a. As shown, a base-layer midfield field macroblock is selected and selected macroblock motion indices and reference indices are used to derive reference indices and motion vectors from a pair of virtual base frame macroblocks to be used for prediction of dopar motion. current macroblock. If both macroblocks are deintermode, an arbitrary macroblock of the lower upper macroblocks is selected (901 or 902) and reference index and motion vector information of the selected macroblock is used. As shown, to derive reference indices, a corresponding value from the upper 8x8 block of the selected macroblock is copied to the upper and lower 8x8 block reference indices of the virtual base layer upper macroblock, and a corresponding value from the 8x8 lower block of the selected macroblock is copied to the reference indices. upper and lower 8x8 blocks of the lower macrob-locus of the virtual base layer. As shown, to derive motion vectors, a corresponding value of each 4x4 block of the selected macroblock is commonly used as the motion vectors of a corresponding pair of vertically adjacent 4x4 blocks in the virtual base layer macroblock pair. In another embodiment of the present invention, motion information of the corresponding pair of base layer macroblocks may be mixed and used to derive motion vectors and reference macroblock frames from the virtual base layer, which is different from the illustrated embodiment. in figure 9a. Figure 9 illustrates a procedure for deriving motion vectors and reference indices according to this embodiment. A detailed description of the association of copying reference indices and subblock motion vectors (8x8 blocks and 4x4 blocks) in the base layer macroblock pair. This virtual representation is omitted here as it can be understood intuitively from the description of the motion information derivation procedure described above and the drawing of figure 9b.

However, since the movement information of the two macroblocks of the base layer field macroblock pair is used in the embodiment of Figure 9b, if one of the base layer field macroblocks is intra-mode, motion information of the Intramode macroblock are derived using motion information from the other macroblock which is an intermediate macroblock. Specifically, reference index and doped motion vector information of the virtual base layer macroblock can be derived as illustrated in Figure 9b after motion vectors and intramode macroblock reference indices are constructed by copying corresponding information from the intermodal macroblock to the intramode macroblock, as shown in Figure 4b or after the intramode macroblock is considered as an intermediate macroblock having zero motion vectors and zero reference indices, as shown in Figure 4c, or after intramode macroblock reference indices are set by copying those from the Intermediate macroblock for intramode macroblock and motion vectors thereof are set to zero, as shown in Figure 4d. After the reference index and motion vector information of the virtual base layer macroblock pair is derived, block modes of the macroblock pair are determined based on the derived information as described above.

On the other hand, if both of the base layer field macroblock pair are intramode, motion prediction is not performed.

The prediction of interlayer texture will now be described. Figure 8d illustrates an example interlayer text prediction method in the case of "MB field in frameMBAFF -> MB of frame". The EL 20 encoder identifies block osmodes of a corresponding pair of base layer field 810 macroblocks. If both of the frame macroblock pair 810 are intra or intermediate, the EL 20 encoder converts the base layer matched field macroblock 810 pair to a temporary frame pair 821 to perform current dopar intrabase prediction. 813 frame macroblocks (when the two frame macroblocks 810 are intramode) or perform their residual measurement in the manner described below (when the two frame macroblocks 810 are intermediate). When two of the corresponding macroblock pair 810 are intra-mode, macroblock pair 810 includes data that has been coded, and an unlock filter is applied to the frame macroblock pair 821, as described above.

Figure 8e shows a method for converting a pair of field macroblocks into a pair of frame macroblocks. As shown, rows from a pair of A eB field macroblocks are alternately selected (A-> B -> A -> B -> A ->, ..) sequentially from the top of each macroblock are then sequentially arranged in the order selected from the top to construct a pair of frame A 'and B' frames. As the lines of the field macroblock pair are rearranged in this way, the upper frame macroblock A 'is constructed from the upper half lines of the field macroblock pair A and B and the lower frame macroblocks B' are constructed. from the lower half lines.

On the other hand, if only one of the corresponding pair of base layer field macroblocks 810 is inter-mode, a block is selected from the base layer macroblock pair 810, according to block modes of the current pair. macroblock frame 813 and the selected block is used for interlayer texture prediction. Alternatively, prior to determining block modes of the current pair of frame blocks 813, each method described below may be applied to perform interlayer prediction, and block modes of the pair of frames 813 may then be determined.

Figures 8f and 8g illustrate examples in which a block is selected to perform interlayer prediction. Where the current pair of frame macroblocks 813 are coded in intermediate (or where their intermediate prediction is performed), as shown in Figure 8f, an intermediate block 810a is selected from a pair of field macroblocks 810 of the layer. base and the selected block is sampled upwards in a vertical direction to create two corresponding macroblock 831. The two macroblock 831 are then used for residual prediction of the current pair of frame macroblock 813. In the case where the current pair of macroblock Table 813 is not coded in between (or where its intramode prediction is performed), as shown in Figure 8g, an intramode block 810b is selected from the base layer field macroblock pair 810 and the selected block is sampled upwards in a vertical direction to create two corresponding macroblock 841. After an unlock filter is applied to the two macroblock 841, the two macroblock 841 are used to intrabasal redemption current pair of frame 813 macroblocks.

The method illustrated in Figures 8f and 8g, in which a block is selected and sampled upwards to create a pair of macroblocks to be used for interlayer texture prediction, can also be applied when the layers have different image rates. When the image rate of the enhanced layer is higher than that of the base layer, some images of an enhanced layer image sequence may not have a correspondingly temporal image in the base layer. Interlayer texture prediction of a pair of frame macroblocks included in an enhanced layer image, which has no temporally matching image in the base layer, can be performed using one of a pair of spatially-located field macroblocks in a temporally anterior image in the base bed.

Figure 8h is an example of such a method in which case the image rate of the enhanced layer is twice that base layer.

As illustrated, the image rate of the enhanced layer is twice that of the base layer. Therefore, one of every two images of the enhanced layer as an image with an image order count (POC) of "n2" has no images with the same image order count (POC) in the base layer. Here, the same POC indicates the temporal coincidence.

When there is no time-matched image in the base layer (for example, when the current POC is n2), a lower-field macroblock 802 included in a pair of spatially co-located field macroblocations in a forward image (that is, a image with a lower POC than the current one at 1) is sampled vertically upward to create a temporary pair of 852 macroblock (S82) and a temporary block of 852 macroblock is then used to perform interlayer texture prediction of the current pair of ma-croblocks 815. When there is a temporally coincident image in the base layer (for example when the current POC is nl), an upper-field macroblock 801 included in a pair of spatially co-located ma-macroblocks at temporally coincident image is sampled vertically vertically to create a temporary pair of macroblock 851 (S82) and the temporary pair of macroblock 851 is then used to perform intercaural texture prediction. Current macroblock pair 814. When a pair of macroblock decoded from an intramode macroblock is included in the temporary pairs of 851 or 852 macroblocks created by upward sampling, the macroblock pair is used for interlayer texture prediction after a unblocking filter be applied to the macroblock pair.

In another embodiment of the present invention, when there is a temporally coincident image in the base layer (when the current POC is nl in the example of Fig. 8h), a frame macroblock pair may be created from a field macroblock pair, according to FIG. shown in Fig. 8d, instead of using the method illustrated in Fig. 8h, and can then be used to predict interlayer textures. In addition, when the current image has no temporally coincident image in the base layer (when the current POC is n2 in the example of figure 8h), prediction of interlayer textures can be performed as in figure 8h or alternatively interlayer texture prediction can will not run on macroblocks in the current image.

Accordingly, one embodiment of the present invention allocates a Λfield_base_fIag 'indicator to indicate interlayer texture measurement was performed according to the method illustrated in Fig. 8d or according to the method illustrated in Fig. 8h and incorporates the flag in coding information. . For example, this flag is set to λ0 'when texture prediction was performed, according to the method as in figure 8d and is set to Λ1' when texture prediction was performed, according to the method as in figure 8h . This indicator is defined as a set of sequence parameters in the enhanced layer, a sequence parameter in a scalable extension, a set of image parameters, a set of image parameters in a scalable extension, a slice header, a slice header in a scalable extension, a macroblock layer, or a macroblock layer in a scalable extension to be passed to a decoder.

IV. 0 case of field MB in field image -> frame MB

In this case, current layer macroblocks (EL) were coded into frame macroblocks and base layer macroblocks (BL) to be used for interlayer prediction of current layer frame macroblocks were encoded into field macroblocks in a field image. . Video signal components included in a field macroblock in the base layer are the same as those included in a macroblock pair located in the current layer. First, prediction of interlayer movement is described as follows.

The EL 20 encoder uses, as partitioning modes of a macroblock in a virtual base layer, department modes obtained by extending a macroblock in an odd stopped field of the base layer (twice in a vertical direction). Figure 10a illustrates a detailed example of this process. The procedure illustrated in Figure 10a is different from that of Case III, where the upper or lower field macroblock in an MBAFF frame is selected, where a spatially co-located field macroblock 1010 in a campopar or odd is naturally used and is similar. to that field III in which the co-located field macroblock 1010 is extended and two macroblock partition modes obtained through extension are applied to a pair of macroblock1012 of the virtual base layer. A partition mode (or pattern) that is not allowed in macroblock partition modes can be generated when the corresponding field macroblock 1010 is extended twice in a vertical direction. To avoid this, the EL 20 encoder determines partition modes depending on extended partition modes according to the same rules as rules 1) and 2) suggested in case III.

If the corresponding macroblock has been intramode encoded, the EL 20 encoder performs individually interlayer text prediction without performing partition mode determination through the above extension process and a motion vector and index induction process. reference described below. That is, the EL20 encoder does not perform interlayer motion prediction.

The procedure of derivation of motion vector and reference index is also similar to that described in the previous case III. However, this case IV is different from case III in the following respect. In Case III, since corresponding base layer macroblock is loaded into odd and even macroblock pairs in one frame, one of the upper and lower macroblock is selected and applied to the shunt procedure. In this case IV, since only one macroblock that corresponds to the current macroblock to be encoded is present in the base layer, motion information for a pair of virtual base layer 1012 macroblocks is derived from motion information of a ma corresponding field block without the macroblock selection procedure, as shown in Figures 10b and 10c, and derived motion information is used for interlayer motion prediction of the current pair of macroblock1013.

Figure 11 schematically illustrates derivation of reference indices and motion vectors of a pair of virtual base layer macroblock according to another embodiment of the present invention. In this case, moving information of a pair of virtual base layer macroblocks is derived from motion information of an even or odd base layer macroblock, which is different from the case described above with reference to Figure 9a. The same derivation operations as those in the case of figure 9a are applied in this case. However, the process of shuffling and using motion information from the macroblock pair in the case illustrated in Fig. 9b is not applied to this case since there is no upper and lower macroblock pair in a corresponding field of the base layer. .

In the embodiment described with reference to FIGS. 10a to 10c, to predict motion information of the virtual base layer macroblock pair, the EL 20 encoder sequentially degresses partition modes, reference indices, and motion vectors based on motion information. of a corresponding field macroblock of the layer is based. However, in another embodiment of FIG. 11, the EL 20 coder first derives reference indices and motion vectors from a pair of basevirtual layer macroblocks based on motion information from a corresponding pair of macroblock layers. base and then finally determines partition modes of the virtual base layer macroblock pair based on the derived values. When department modes are determined, 4x4 block units with the same derived motion vectors and reference indices are combined, and if combined block mode is a permitted partition mode, partition modes are set to combined mode otherwise. way the partition modes are set to those before the combination.

When texture prediction is performed in the modalities described above, if the corresponding field macroblock of the base layer is intramode, intrabase predictive coding is performed on the current macroblock. If the corresponding field macrob-block is from intermediate and if the current macrob-locus has been coded in intermediate, the interlayer residual predictive coding is performed. Here, of course, the field macroblock for prediction use is used for texture prediction after it is sampled upwards in a vertical direction.

In another embodiment of the present invention, a virtual macroblock is created from a field macroblock included in an odd or even field to construct a pair of macroblock, and then motion information from a pair of virtual base layer macroblock is derived from the pair. built of macroblocks. Figures 12a and 12b illustrate examples of this embodiment.

In this embodiment, reference indices and motion vectors of a corresponding (or odd) field-matched base layer macroblock are copied (1201 and 1202) to create a virtual odd (or even) field macroblock to construct a pair of macroblock 1211 and motion information of the macroblock constructed pair 1211 is mixed to derive motion information from a macroblock pair 1212 of the virtual base layer (1203 and 1204). In an example method of mixing and using motion information, as illustrated in Figures 12a and 12b, reference indices of the upper 8x8 block of the corresponding upper macroblock are applied to the upper 8x8 block of the virtual base layer macroblock 1212 upper macroblock, Lower 8x8 block reference indices are applied to the upper 8x8 block of the lower macroblock, reference 8x8 upper block indices of the corresponding lower macroblock are applied to the lower 8x8 block of the upper macroblock dopar 1212 of the virtual base layer, and lower 8x8 block reference indices are applied to the lower 8x8 block of the lower macroblock (1203). Motion vectors are applied according to reference indices (1204). A description of this process is omitted here as it can be understood intuitively from Figures 12a and 12b.

In the embodiment illustrated in Figures 12a and 12b, partition patterns of the basevirtual layer macroblock pair 1212 are determined based on derived reference indices and motion vectors using the same method as described above.

Prediction of interlayer texture will be described now. Figure 10b illustrates an example interlayer text prediction method for this case of "Field MB in Field Image -> Frame MB". The EL 20 encoder first upwardly samples a corresponding base layer camp macroblock 1010 to create two temporary macroblock 1021. If the corresponding field macroblock 1010 is intramode, the EL 20 encoder applies an unlock filter. in the two created temporary macroblocks1021 and then performs intrabase prediction of the current pair of frame macroblocks 1012 based on the two temporary macroblocks 1021. If the corresponding field macroblock 1010for is from intermediate, the EL 20 encoder performs residual prediction of the current pair of 1013 frame macroblocks based on the two created temporary macroblocks 1021.

V. 0 case of field MB -> field MB

This case is subdivided into the following four cases since field macroblocks are divided into field macroblocks included in a field image and field macroblocks included in an MBAFF frame.

i) The case where the intensified and base layers are MBAFF frames

This case is shown in figure 13a. As shown, motion information (partition modes, reference indices, and motion vectors) of a corresponding base layer macroblock pair is used as motion information for a virtual base layer macroblock pair. by directly copying the movement information from the corresponding macroblock pair to the virtual base layer macroblock pair. Here moving information is copied between macroblock of the same parity. Specifically, motion information from even field macroblock is copied to even field macroblock and odd field macroblock motion information is copied to odd field macroblock to build the virtual layer ma-croblock, which is used for prediction of moving the current layer macroblock.

A known method of interlayer texture prediction between frame macroblocks is applied when texture prediction is performed.

ii) the case where the base layer includes field images and the enhanced layer includes MBAFF frames

This case is shown in figure 13b. As shown, motion information (partition modes, reference indices, and motion vectors) of a base layer corresponding field macroblock is used as motion information for each of a pair of macroblocks of a virtual base layer for copying. directly the corresponding field macroblock moving information for each of the macroblock pair. Here, the parity copy rule is not applied since motion information from a single field macroblock is used for both upper and lower field macroblockers.

When texture prediction is performed, intrabase prediction is applied (when the corresponding base layer block is intramode) or residual prediction (when the base layer corresponding block is intermediate) is applied between macroblock of the base and intensified layers having same field attributes (even or odd).

iii) the case where the base layer includes MBAFF frames and the enhanced layer includes field images

This case is shown in figure 13c. As shown, a field macroblock of the same parity is selected from a pair of base layer macroblocks corresponding to the current field macroblock and motion information (partition modes, reference indices, and motion vectors) of the macroblock. Selected field data are used as motion information from a field macroblock of a virtual base layer by directly copying the moving information from the selected field macroblock to the virtual base layer field macroblock.

When texture prediction is performed, intrabase prediction is applied (when the corresponding base layer block is intramode) or residual prediction (when the corresponding base layer block is of intermediate) is applied between intensified and base layer macroblocks having same field attributes (even or odd).

iv) the case where intensified layers and basement field images

This case is shown in figure 13d. As shown, motion information (partition modes, reference indices, and motion vectors) of a base layer corresponding field macroblock is used as motion information of a field macroblock of a virtual base layer by directly copying. the movement information from the corresponding field macroblock to the virtual base layer field macroblock. Also in this case, the movement information is copied between the same parity macroblocks.

A known method of interlayer texture prediction between frame macroblocks is applied when texture prediction is performed.

The above description of interlayer prediction is provided for the case where the intensified and basaltic layers have the same resolution. The following description will be provided on how the type of an image (a progressive frame, an MBAFF frame or an interlaced field) of each layer and / or a macroblock in an image are identified when the resolution of the enhanced layer is more high basal layer (ie, when SpatialScalabilityTypeO is higher than zero) and an interlayer prediction method is applied according to the identified types. First, interlayer motion prediction is described.

BAD). Base layer (progressive board) -> Intensified layer (MBAFF board)

Figure 14a illustrates a processing method for this case. As shown, first, moving information from all macroblock of a corresponding frame in the base layer is copied to create a virtual frame.

The upward sampling is then performed. In this downward sampling, interpolation is performed using texture information from the base layer image at an interpolation rate that allows the image resolution (or image size) to be equal to that of the current layer. In addition, motion information for each macroblock in the image enlarged by interpolation is constructed based on the moving information of each virtual frame macroblock. One of a number of known methods is used for this construction. Images of an interim base layer constructed from this modulus have the same resolution as the current (intensified) layer images. Accordingly, the above-described interlayer motion prediction can be applied in this case.

In this case (Figure 14a), macroblocks in images in the base and current layers are frame macroblocks and field macroblocks in an MBAFF frame since the base layer includes frames and the current layer includes MBAFF frames. Therefore, the case I method described above is applied to perform interlayer motion prediction. However, not only a pair of field macroblocks but also a pair of frame macroblocks can be included in the same MBAFF framework as described above. Therefore, a known method for motion prediction between frame macroblocks (frame-by-frame prediction method), which includes simple copying of motion information, is applied when the type of a pair of macroblock of the current layer corresponds to the pair. of macroblocks in the baseinterin layer image was identified as the frame macroblock type rather than the field macroblock type.

M_B). Base layer (progressive frame) -> intensified layer (interlaced field)

Figure 14b illustrates a processing method for this case. As shown first, the motion information of all macroblocks of a corresponding frame in the base layer is copied to create a virtual frame. The upward sampling is then performed. In this ascending sample, interpolation is performed using texture information from the base layer image at a rate of interpolation that allows the image resolution to be equal to that of the current layer. In addition, motion information for each macroblock in the interpolated image is constructed based on movement information for each macroblock of the created virtual frame.

The Case II method described above is applied to perform interlayer motion prediction since each of the interim base layer image macroblock thus constructed is a frame macroblock and each current layer macroblock is a field macroblock in a cam image. -powder.

M_C). Base layer (MBAFF frame) -> Intensified layer (progressive frame)

Figure 14c illustrates a processing method for this case. As shown, first, a corresponding base layer MBAFF frame is transformed into a progressive frame. The case III method described above is applied for transforming MBAFF frame field macroblock pairs into the progressive frame, and the known frame to frame prediction method is applied for transforming MBAFF frame macroblock pairs. Of course, when the case III method is applied in this case, a virtual frame and motion information from each frame macroblock are created using data obtained through interlayer prediction without performing the operation to encode the difference between predicted data and layer data to actually be encoded.

After obtaining the virtual frame, the upstream sampling is performed on the virtual frame. In this ascending sampling, interpolation is performed at an interpolation rate that allows the resolution of the base layer to be equal to that of the current layer. In addition, movement information of each macroblock of the enlarged image is constructed based on movement information of each macroblock of the virtual frame using one of a number of known methods. Here, the known macroblock interlayer motion prediction method for known frame macroblock is performed since each macroblock of the inner base layer image constructed in this way is a frame macroblock and each actual layer macroblock is a frame macroblock.

M_D). Base layer (interlaced field) -> intensified layer (progressive frame)

Figure 14d illustrates a processing method for this case. In this case, the type of an image is the same as the macroblock type of the image. As shown first, a corresponding base layer field is transformed into a progressive frame. The transformed frame has the same vertical / horizontal ratio (aspect) as that of the current layer image. The bottom-up sampling process and case IV method described above are applied for transforming an interlaced field into a progressive frame. Of course, when the case IV method is applied in this case, virtual frame text data and frame macroblock motion information are created using data obtained through interlayer prediction without performing the operation to encode the difference between predicted data and data. layer to actually be coded.

After obtaining the virtual frame, the upstream sampling is performed on the virtual frame. In this upstream sampling, interpolation is performed to allow the resolution of the virtual frame to be the same as that of the current layer. In addition, motion information for each macroblock in the interpolated image is constructed based on motion information for each virtual frame macroblock. using a number of known methods. Here, the known macroblock interlayer motion prediction method for known frame macroblock is performed since each frame of the interim base layer frame constructed in this way is a frame macroblock and each current bed macroblock is a macroblock. of frame.

Figure 14e illustrates a method of processing for the above case (D) according to another embodiment of the present invention. As shown, this modality transforms an odd or even corresponding field into a progressive frame. To transform an interlaced field into a progressive frame, the upward sampling and the case IV method described above are applied, as shown in figure 14d. After obtaining a virtual frame, a method for predicting moving between frames having the same aspect ratio, which is one of a number of known methods, is applied to the virtual frame for predicting motion between the current layering image and the interim layer virtual frame. to perform predictable encoding of motion information for each macroblock progressive image of the current layer.

The method illustrated in Fig. 14e is different from that of Fig. 14d in that no temporary prediction signal is generated.

Figure 14f illustrates a method of processing for the above case (D) according to another embodiment of the present invention. As shown, this embodiment copies motion information from all macroblock of a corresponding base layer field to create a virtual image. Upward sampling is then performed. In this bottom sampling, texture information from a base layer image is used and different interpolation rates are used for vertical and horizontal interpolation so that the enlarged image is the same size (or resolution) as that of the current layer image. In addition, one of a number of known prediction methods (eg, Ex-tended Special Scalability (ESS)) is applied to the virtual image to construct a variety of syntax information and enlarged image movement information. Motion vectors constructed in this process are extended according to the rate of increase. After constructing the following sampled image of the interim base layer, this image is used to perform interlayer motion prediction of each macroblock in the current layer image to encode motion information for each macroblock of the current layer image. Here, the known frame-to-macroblock macroblock interlayer motion prediction method is applied.

Figure 14g illustrates a processing method for the above case (D) according to another embodiment of the present invention. As shown, this mode first copies motion information from all macroblock from a corresponding base layer field to create a virtual image. Subsequently, base layer image texture information is used to perform interpolation at different rates for vertical and horizontal interpolation. Texture information created through this operation is used for interlayer texture prediction. In addition, motion information in the virtual image is used to perform interlayer motion prediction of each macroblock in the current layer image. Here, one of a number of known methods (For example, Extended SpecialScalability (ESS) defined in the Joint Scalable Video Model (JSVM)) is applied to perform predictable encoding of the current layer image.

The method illustrated in Fig. 14g is different from that of Fig. 14f where no temporary prediction signal is generated.

ME). Base layer (MBAFF frame) -> Intensified layer (MBAFF frame)

Figure 14h illustrates a processing method for this case. As shown, first a corresponding MBAFF frame of the base layer is transformed into a quadrupressive. To transform the MBAFF frame into a quadrupressive, the case III method described above is applied to the MBAFF field macroblock pair transformation and the frame-to-frame prediction method is applied for pair transformation. of MBAFF frame quadrode macroblocks. Of course, when the case III method is applied in this case, a virtual frame and moving information from each frame macroblock are created using interlayer prediction data without performing the operation to encode the difference between predicted data and layer data to be η actually encoded.

After obtaining the virtual frame, the upstream sampling is performed on the virtual frame. In this upstream sampling, interpolation is performed at an interpolation rate that allows the resolution of the base layer to be equal to that of the current layer. In addition, movement information of each macroblock of the enlarged image is constructed based on movement information of each macroblock of the virtual frame using one of a number of known methods. The case I method described above is applied to perform interlayer motion prediction since each interlayer of the interim base layer image constructed in this way is a frame macroblock and each current layer macroblock is a frame macroblock. field in an MBAFF frame. However, not only a pair of field macroblocks but also a pair of frame macroblocks can be included in the same MBAFF frame as described above. Accordingly, a known method for motion prediction between frame macroblock (frame to frame prediction method), which includes copying motion information, is applied when the macroblock pair of the current layer corresponds to the macroblock in the image. of the interim base layer are frame macroblock locations rather than field macroblocks.

M_F). Base layer (MBAFF frame) -> Intensified layer (interlaced field)

Figure 14i illustrates a processing method for this case. As shown, first, a corresponding base layer MBAFF frame is transformed into a progressive frame. To transform the MBAFF frame into a progressive frame, the case III method described above is applied for MBAFF frame field block pair transformation and the frame-to-frame prediction method is applied for MBAFF frame frame pair transformation. Of course, also when the method of Case III is applied in this case, a virtual frame and movement information for each macroblock in the frame are created using data obtained through interlayer prediction without performing the operation to encode the difference between predicted data and data from the frame. layer.to be actually coded.

After obtaining the virtual frame, interpolation is performed on the virtual frame at an interpolation rate that allows the resolution to be the same as the current layer resolution. In addition, motion information for each macroblock of the enlarged image is constructed based on motion information for each virtual frame macroblock using one of a number of known methods. The method II described above is applied to perform interlayer prediction since each macroblock of the interim base layer image constructed in this way is a frame macroblock and each macroblock of the current layer is a field macroblock in a field. odd or even.

M_G). Base layer (interlaced field) -> Intensified layer (MBAFF table)

Figure 14 illustrates a processing method for this case. As shown, first, an interlaced field of the base layer is transformed into a progressive frame. The upward sampling and the case IV method described above are applied to transform the interlaced field into a progressive frame, of course, also when the case IV method is applied in this case, a virtual frame and movement information of each macroblock. Frame data are created using data obtained through interlayer prediction without performing the operation to encode the difference between predicted data and layer data to be actually encoded.

After obtaining the virtual frame, up-sampling is performed on the virtual frame to allow the resolution to be equal to the resolution of the current layer. In addition, motion information from each macroblock of the enlarged image is constructed using one of a number of known methods. The method of case I, described above, is applied to perform interlayer motion prediction since each macroblock of the interlayer base layer image in this way is a frame macroblock and each nia croblock of the current layer is a field macroblock. a fourth MBAFF. However, not only a pair of field macroblocks but also a pair of frame macroblocks can be included in the same MBAFF frame as described above. Therefore, a known method for motion prediction between frame macroblocks (quadrature-to-frame prediction method) rather than the prediction method of case I, described above, is applied when the current layer macroblock pair corresponds to the macroblock air in the The interim base bed image includes frame macroblocks instead of field macroblocks.

M_H). Base layer (interlaced field) -> Intensified layer (interlaced field)

Figure 14k illustrates a processing method for this case. As shown, firstly, moving information from all macroblocks of a field corresponding to the base layer is copied to create a virtual field and the ascending sample is then performed on the virtual field. This ascending sampling is performed at an ascending sample rate that allows the resolution of the layer is the same as that of the current layer. In addition, motion information from each macroblock of the enlarged image is constructed based on motion information from each virtual field macroblock created using one of a number of known methods. The method of case iv) of case V, described above, is applied to perform interlayer motion prediction since each interlayer image macroblock constructed in this way is a field macroblock in a field image and each current layer macroblock also is a field macroblock in a field image.

Although texture information of the virtual field or interim layer frame instead of textured background image of the base layer is used for down sampling in the description of the embodiments of figures 14a through 14k, texture information of the base layer image may also be used for up sampling. In addition, if not necessary, the interpolation process using texture information can be omitted in the upward sampling process described above by deriving interim layer image shift information to be used for interlayer motion prediction that is performed. at a subsequent stage.

On the other hand, although the prediction description of the text has been provided for the case where the base and intensified layers have the same special resolution, the two layers may have different spatial resolutions as described above. In the case where the resolution of the enhanced layer is higher than that of the base layer, firstly, operations to make the resolution of the layer image base equal to that of the enhanced layer image are performed to create a The base layer image having the same resolution as that of the enhanced layer and a texture prediction method corresponding to each of the I-V cases described above is selected based on each macroblocone image to perform predictable encoding. The procedure for making the base layer image resolution equal to that of the enhanced layer image will now be described in detail.

When two interlayer prediction layers are considered, the number of image format combinations (progressive and interlaced formats) to encode between the two layers is given as there are two video signal scanning methods, one progressive scan and the other scan. interlaced. Thus, a method for increasing the base layer image resolution to perform interlayer texture prediction will be described separately for each of the four cases.

OK) . The case where the intensified layer is progressive and the base layer is interlaced

Figure 15a illustrates one embodiment of a method for using a base layer image for predicting interlayer textures for that case. As shown, a base layer image 1501 corresponding to a temporal image 1500 of the current (enhanced) layer includes odd and even fields which are transmitted at different times. Therefore, firstly, the EL 20 encoder separates the base layer image into even and odd fields (S151). Intra-mode macroblocks of base layer image 1501 have original unencrypted image data (or decoded image data) that is used for intra-base prediction, and intermodal macroblocks thereof have residual data. encoded data (or decoded residual data), which is used for residual prediction. The same is true for base layer macroblocks or images when text prediction is described below.

After splitting the corresponding image 1501 into field components, the EL 20 encoder performs interpolation of the separate fields 1501a and 1501b in a vertical / or horizontal direction to create enlarged even and odd images 1502a and 1502b (S152). This interpolation utilizes one of a number of methods known as 6-lead filtration and binary-linear filtration. Vertical and horizontal ratios for increasing image resolution (i.e. size) by interpolation are equal to vertical and horizontal ratios of enhanced layer image size 1500 to base layer image size 1501. Vertical and horizontal ratios can be equal to each other. For example, if the resolution ratio between intensified and base layers is 2, interpolation is performed on separate even and odd fields1501a and 1501b to create one more pixel between each pixel in each field in vertical and horizontal directions.

Upon completion of interpolation, the enlarged even and odd fields 1502a and 1502b are combined to construct an image 1503 (S153). In this combination, increased odd and even field lines 1502a and 1502b are alternately selected (1502a-> 1502b -> 1502a-> 1502b-> ..) and are then arranged in the order selected to construct the 1503 combined image. Here, The block mode of each macroblock in the combined image 1503 is determined. For example, the block mode of a macroblock of the matching image 1503 is determined to be the same as that of a macroblock in the layered image 1501 that includes an area having the same image component. This method of determination can be applied in any case of enlarging images described below. Since the combined image 1503 constructed in this way has the same special resolution as that of the current intensified layer image 1500, texture prediction (for example, inter-macroblock texture prediction from frame to frame) of macroblock in the frame. current progressive image 1500 is matched to corresponding macroblocks of combined image 1503 (S154). Figure 15b illustrates a method of using an interlayer texture prediction base layer image according to another embodiment of the present invention. As shown, this mode performs interpolation of a base layered image including even and odd fields that are transmitted at different times in a vertical and / or horizontal direction directly, without separating the base layer image into an attribute base. (Parity) (S155) to build an enlarged image of the same resolution (i.e. size) as that of the enhanced layer image. The enlarged image constructed in this way is used to perform interlayer texture prediction of the current progressive layer intensified image (S156).

Figure 15a shows, at an image level, the procedure for interpolating an image having even and odd fields by separating it on a field attribute basis. However, the EL 20 encoder can achieve the same results as illustrated in Fig. 15a by performing the procedure illustrated in Fig. 15a at a macroblock level. More specifically, when the base layer having odd even fields was encoded in MBAFF, a pair of vertically adjacent macroblocks in image 1501, which are co-located with a pair of macroblocks in the enhanced layer image currently being subjected to predictable texture coding, may include video signals from odd and even components such as in figure 16a or 16b. Fig. 16a illustrates a frame MB pair mode in which odd-even field components are interleaved on each of a pair of macroblock Ae B and Fig. 16b illustrates a field-pair MB mode in which each of a pair of macroblock AeB includes lines. devideo of the same field attribute.

In the case of Fig. 16a, to apply the method illustrated in Fig. 15a, even lines of each of the pair of macroblock loci AeB are selected to construct a field block even A 'and odd lines thereof are selected to construct a pair. odd field block B ', thereby separating the opar from macroblocks having even and odd interleaved field components in each macroblock in both blocks A' and B 'respectively having odd and even field components. Interpolation is performed on each of the two macroblocks A 'and B' thus separated to construct an enlarged block. Texture prediction is performed using data in an enlarged noblock area that corresponds to an intra_BL or residual_prediction macroblock in the enhanced layer image currently being subjected to predictable text encoding. Although not shown in Fig. 16a, the combination of individually enlarged blocks on a field assignment basis partially builds up the odd and even increased images 150a and 1502b in Fig. 15a and therefore the enlarged odd and even images 1502a and 1502b in Fig. 15a. can be constructed by repeating the above operations for each pair of macroblocks.

In the case where a macroblock pair is split on a field assignment basis to construct each macroblock, as in Figure 16b, the separation procedure described above is a process of simply copying each macroblock from the macroblock pair to construct two separate macroblocks. . The subsequent procedure is similar to that described with reference to figure 16a.

ALSO) . The case where the intensified layer is interlaced and the base layer is progressive

Figure 17a illustrates one embodiment of a method of using a base layer image for predicting interlayer textures for this case. As shown first, the EL 20 encoder builds two images for a current layer image 1700 (S171). In an exemplary method for constructing two images, even lines of a corresponding image 1701 are selected to construct an image 1701a and odd lines thereof are selected to construct another image 1701b. The EL 20 encoder then performs interpolation of these two constructed images 1701a and 1701b in a vertical and / or horizontal direction to create two enlarged images 1702a and 1702b (S172). Such interpolation uses one of a number of methods known as 6-lead filtering and binary-linear filtering as in the case T_A). The reasons for increasing the resolution are also the same as those described in the case T_A).

Upon completion of interpolation, the two augmented fields 1702a and 1702b are combined to construct an image 1703 (S173). In this combination, rows of the two enlarged fields 1702a and 1702b are alternately selected (1702a-> 1702-> 1702a-> 1702-> ..) and are then arranged in the selected order to construct the combined image 1703. Once the combined image 1703 constructed modulate the same spatial resolution as that of the current layer 1700 of the enhanced layer, texture prediction (for example, frame-frame inter-macroblock texture prediction or texture prediction described with reference to figure 4g) of macroblock in the image. The current interlaced 1700 is executed based on corresponding macroblocks of the merged image 1703 (S174).

Figure 17b illustrates a method of using an interlayer texture predictive base layer image according to another embodiment of the present invention. As shown, this mode performs interpolation of a base layered image in a vertical and / or horizontal direction directly, without separating the base layer image into two images (S175) to construct an enlarged image of the same resolution ( that is, size) than that of the enlarged layer image. The enlarged image constructed in this way is used to perform interlayer texture prediction of the current interlaced enhanced layer image (S176).

Although the description of Fig. 17a has been provided at an image level, the EL 20 encoder can perform the process of image separation at a macroblock level, as described above in case T_A). The method of Fig. 17b is similar to the separation and interpolation procedure illustrated in Fig. 17a when single image 1701 is considered to be a pair of vertically adjacent macroblocks.A detailed description of this procedure is omitted here as it can be understood intuitively. from Figure 17a.T_C). The case where both intensified base layers are interlaced

Figure 18 illustrates one embodiment of a method of using a base layer image for predicting interlayer textures for this case. In this case, as shown, the EL 20 encoder separates a base layer image 1801 which is corresponding to a current layer image 1800 in odd and even fields (S181) in the same way as in the case T_A). The EL 20 encoder then performs interpolation of separate fields 1801a and 1801b in a vertical and / or horizontal direction to create enlarged even and odd images 1802a and 1802b (S182). The EL 20 encoder then combines the enlarged odd and even fields 1802a and 1802b to construct an image 1803 (S182). The EL 20 encoder then performs interlayer texture prediction (e.g., frame-to-frame inter-frame texture prediction or texture prediction, described with reference to Figure 4g) of macroblock (MBAFF-encoded frame macroblock pairs) in the image. current interlaced frame 1800 based on matching macroblocks of the combined image 1803 (S184).

Although the two layers have the same image format, the EL 20 encoder separates the base1801 layer image into a field attribute base (S181) and individually enlarges the separate fields (S182) and then combines enlarged images (S183). Since if the image 1801 combining the odd and even fields is interpolated directly when it has a feature that the odd and even field video signals vary greatly, the enlarged image may have a distorted picture (for example, an image having wide boundaries). ) compared to interlaced image 1800 of even and odd interleaved tendon fields of the enhanced layer. Therefore, even when the two layers are interlaced, the EL 20 encoder uses the base layer image after stopping it on a field attribute basis to obtain two fields and individually enlarge the two fields and then combine the augmented fields. in accordance with this invention.

Of course, instead of using the method illustrated in figure 18 always when images of the two layers are interlaced, the method can be selectively used depending on the video characteristics of the images.

Figure 18 shows, at an image level, the procedure for separating and enlarging an image having odd and even fields and a field attribute base in accordance with the present invention. However, as described above in T_A), the EL 20 encoder can achieve the same results, as illustrated in Figure 18 by performing at a macrob-locus level the procedure illustrated in Figure 18 which includes macroblock-based separation and interpolation processes. -co, described with reference to Figures 16a and 16b (specifically, separating a pair of frame macroblocks into odd and even line blocks and individually increasing the separated blocks) and the combination and interlayer texture prediction processes (specifically, alternately selecting li -nails of the augmented blocks to construct a pair of augmented blocks and perform texture prediction of a pair of frame macroblocks of the current layer using the constructed pair of augmented blocks).

T_D). The case where the layers, both intensified as a base, are progressive

In this case, a base layer image is enlarged to the same size as that of the enhanced layer image and the enlarged image is used to predict the interlayer texture of the current enhanced layer image having the same image format.

Although the above texture prediction modalities have been described when the base and intensified layers have the same temporal resolution, the two layers may have different temporal resolutions, that is, different image rates. If layer images are of different image scan types even when the layers have the same temporal resolution, the images may contain video signals with different output times even if they are POC images (ie, images temporally corresponding between each other). si). A method for predicting interlayer texture in this case will now be described. In the following description, it is assumed that the two layers initially have the same spatial resolution. If the two layers have different spatial resolutions, the methods described below are applied after upward sampling of each base layer image to make the spatial resolution equal to that of the intensified layer as described above.

a) In the case where the intensified layer includes progressive frames, the base layer includes MBAFF frames, and temporal resolution of the intensified layer is twice as high.

Figure 19a illustrates an interlayer texture prediction method for this case. As shown, each base layer MBAFF frame includes odd and even fields with different output times, and thus the EL 20 encoder is split into odd and even fields MBAFF (S191). The EL 20 encoder separates even field components (eg even lines) and odd field components (eg odd lines) from each MBAFF frame into an even field and an odd field, respectively. . After splitting an MBAFF frame into two fields in this way, the EL 20 encoder interpolates each field in a vertical direction so that it has twice as high resolution (S192). This interpolation utilizes one of a number of methods known as 6-drift filtration, binary-linear filtration, and simple linefilling. Upon completion of interpolation, each frame of the intensified layer has a temporally matched image in the base layer and thus the EL 20 encoder performs known interlayer texture prediction (e.g., frame-to-frame inter-block prediction) in frame-intensified layer macroblocks (S193). ).

The above procedure can also be applied to the interlayer movement prediction. Here, by separating an MBAFF frame into two fields, the EL 20 encoder copies motion information from each of a pair of MBAFF frame field macroblocks as motion information from a macrob-Ioco of the same field attribute (parity ) to use the same for interlayer motion prediction. Use of this method can create a temporally coincident image according to the method described above for performing interlayer prediction even when there is no temporally coincident image in the base layer (in the case of tl, t3, ...).

The method described above can be directly applied when the resolution of one of the two layers is twice as high as that of the other as in the example of Fig. 19a and even when it is N (three or more) times as high. For example, when the resolution is three times as high, one of two separate fields can be further copied to build and use three fields, and when the resolution is four times as high, each of the two separate fields can be copied once more to construct. and use four fields. Clearly, with any difference in temporal resolution, those skilled in the art can perform interlayer prediction simply by applying principles of the present invention without any creative thinking. Thus, it is natural that any method for prediction between different time-resolution layers not described in this descriptive report is within the scope of the present invention. The same is true for other cases described below.

If the base layer was encoded in Picture Adaptive Field & Frame (PAFF) instead of MBAFF frames, both layers can have the same temporal resolution as in figure19b. Thus, in this case, interlayer texture prediction is performed after constructing an image having the same temporal resolution as that of the current layer by interpolating a frame directly without the process of separating the frame into two fields.

b) In the case where the enhanced layer includes four MBAFF frames, the base layer includes progressive frames, and the temporal resolution of the enhanced layer is half that base layer.

Figure 20 illustrates an interlayer text prediction method for this case. As shown, each MBAFF frame of the enhanced layer includes odd and even fields with different output times, and thus the EL20 encoder separates each MBAFF frame into even and odd fields (S201). EL 20 decoder separates even field components (eg even lines) and odd field components (eg odd lines) from each MBAFF frame into an even field and an odd field, respectively. The EL 20 encoder performs sub-sampling of each base layer frame in a vertical direction to construct a half-resolution (S202) image. This sub-sampling may use line sub-sampling or one of several other known downward sampling methods. In the example of Figure 20, the EL 20 encoder selects even lines of images with even image indexes (images of tO, t2, t4, ...) to obtain half-size images and selects odd image lines. with odd image indexes (tl, t3, ... images) for reduced size images by the whole. Frame separation (S201) and sub-sampling (S202) can also be performed in reverse order. Upon completion of the two processes S201 and S202, fields 2001 separated from the frames of the intensified layer have images, which are temporally fields 2001 and have the same spatial resolution as those fields 2001 at the base layer and thus the EL 20 encoder performs known interlayer texture prediction (for example, frame-to-frame inter-macroblock prediction) on macroblocks in each field (S203).

The above procedure can also be applied to the interlayer movement prediction. Here, by obtaining a reduced size image from each base frame by sub-sampling (S202), the EL 20 encoder can obtain motion information of a corresponding macroblock from motion information of each of a pair of vertically adjacent macroblocks according to an appropriate method (for example, a method of adopting motion information from a block that has not been fully divided) and then can use the motion information obtained for interlayer motion prediction. .

In this case, enhanced layer images are PAFF-coded to be transmitted since interlayer prediction is performed on each separate field image2001 from MBAFF frames.

c) In the case where the enhanced layer includes four MBAFF, the base layer includes progressive frames, and both layers have the same temporal resolution.

Fig. 21 illustrates an interlayer texture prediction method for this case. As shown, each enhanced layer MBAFF frame includes odd and even fields with different output times and thus the EL 20 encoder for each MBAFF frame in odd and even fields (S211). The EL 20 coder separates even field components (e.g. even lines) and odd field components (e.g. odd lines) from each MBAFF frame into an even field and an odd field, respectively. The EL 20 encoder sub-samples each frame of the base layer in a vertical direction to construct a half-resolution (S212) image. This sub-sampling may use line sub-sampling or one of several other known downward sampling methods. Frame separation (S211) and sub-sampling (S212) can also be performed in reverse order.

The EL 20 encoder may also construct a field (e.g., an even field image) from an MBAFF frame, rather than splitting an MBAFF frame into two fields. This is because the two layers have the same temporal resolution and so only one of the two separate field images of a frame (rather than both) has a corresponding frame in the base layer that can be used for interlayer prediction.

Upon completion of both processes S211 and S212, the EL 20 encoder performs known interlayer texture prediction (e.g., quad-to-frame inter-macroblock prediction) only on even (odd) fields, between fields separated from the intensified layer frames, image combining corresponding sub-samples in the base layer (S213).

Also in this case, interlayer motion prediction can be performed in separate fields of the intensified layer, for which interlayer texture prediction is performed, as described in case b).

Although above descriptions of interlayer prediction operations performed by the EL 20 encoder of Figure 2a or 2b have been provided, all descriptions of interlayer prediction operations can commonly be applied to an EL decoder that receives decoded information from the base layer and decodes it. intensified layer streams. In encoding and decoding procedures, the interlayer prediction operations described above (including operations to separate, enlarge and combine video signals in images or macroblocks) are performed in the same way as operations subsequent to interlayer prediction are performed in different modes. An example of the difference is that, after performing motion texture prediction, the coder encodes the predicted information or difference information between the predicted information and actual information for transmitting it to the decoder, while the decoder obtains information from the decoder. effective texture and motion information by directly applying information obtained by executing the same interlayer texture prediction as that performed at the encoder, the current macroblock, or additionally utilizing effectively received macroblock coding information. The details and principles of the present invention described above from the coding point of view are directly applied to the decoder which decodes received data streams from the two layers.

However, when the EL encoder transmits MBAFF frame-intensified layer in a PAFF mode after separating the intensified layer into field sequences and performing interlayer prediction, as described with reference to Figures 20 and 21, the decoder does not perform the above procedure for separating frames. MBAFF in field images at the effectively received layer.

In addition, the decoder then decodes, from a received signal, the _bfield_base_fIag 'indicator which identifies whether the EL 20 encoder predicted interlayer textures between macroblocks, as shown in Figure 8d or as shown in Figure 8h. Based on the decoded indicator value, the decoder determines whether the macroblock prediction was performed as shown in Fig. 8d or as shown in Fig. 8h and obtains texture prediction information according to the determination. If the Λfield_base_fIag 'indicator is not received, the EL decoder assumes that the indicator having a value of "0" has been received. That is, the EL decoder assumes that texture prediction between macroblocks was performed according to the method shown in Figure 8d. and get prediction information from the macroblock paratual to reconstruct the current macroblock or pair of macroblock.

At least one of the limited embodiments of the present invention described above may perform interlayer prediction even when using video signal sources of different formats (or modes). Thus, by encoding a plurality of layers, it is possible to increase the data encoding rate regardless of the video signal image types such as interlaced signals, progressive signals, MBAFF frame images, and field images. In addition, when one of the two layers is a source of interlaced video signals, an image of the image for prediction use may be constructed to be more similar to that of the original image for predictable encoding, thereby increasing the data decoding rate.

While the present invention has been described with reference to preferred embodiments, it will be apparent to those skilled in the art that various improvements, modifications, substitutions, and additions may be made to the invention without departing from the scope and spirit of the invention. Accordingly, the invention is intended to cover the refinements, modifications, substitutions and additions of the invention, provided that they are within the scope of the appended claims and their equivalents.

Claims (23)

Method for performing interlayer prediction to encode or decode a video signal, the method being characterized by comprising the steps of: a) constructing a pair of field macroblocks from a pair of vertically adjacent frame macroblocks of a base layer; and b) use respective texture information from the field macroblock as the respective texture prediction information for a pair of field macroblocks of a current / layer. Method according to claim 1, characterized in that step a) includes having even lines of an upper macroblock of the frame macroblock pair in an upper portion and even lines of a lower macroblock of the pair. of frame macroblocks in a lower portion to construct a higher field macroblock of the field macroblock pair and arrange odd lines of the upper macroblock of the frame macroblock in an upper portion and odd lines of the lower macroblock of the frame macroblock in a lower portion. lower portion to construct a lower field macroblock from the pair of field macroblocks. A method according to claim 1, characterized in that step b) includes using the respective texture information of the constructed pair of field macroblocks as intra base prediction information both of the frame macroblock pair are. intra-mode and using the respective texture information of the field macroblock pair as the individual prediction information if both of the frame macroblock pair are intermodal. Method for performing interlayer prediction to encode or decode a video signal, the method being characterized by comprising the steps of: a) constructing a single field macroblock from a pair of vertically adjacent frame macroblocks of a base layer; and b) use texture information from the single field macroblock constructed as text prediction information of a field macroblock of a current layer. A method according to claim 4, characterized in that step a) includes arranging even lines of an upper macroblock of the frame macroblock pair in an upper portion and even lines of a lower macroblock of the pair. of frame macroblocks in a lower portion to construct the single field macroblock if a field attribute of the current layer field macroblock is even and arrange odd lines of the upper macroblock of the frame macroblock in an upper portion and odd lines of the lower macroblock of the frame macroblock pair in a lower portion to construct the single field macroblock if the field attribute of the current camera field macroblock is odd. Method according to claim 4, characterized in that step b) includes using the texture information of the constructed single-field macroblock as intra base prediction information if the two frame macroblock dopers are of the same manner. intra and use texture information from the constructed single-field macroblock as residual prediction information if the two frame macroblock dopers are of the inter mode. Method for performing interlayer prediction to encode or decode a video signal, the method being characterized by comprising the steps of: a) constructing a pair of frame macroblocks from a single field macroblock or a pair of vertically field macroblocks adjacent to a base layer; and b) use respective texture information from the frame macroblock as the respective texture prediction information of a pair of frame macroblocks of a current layer. A method according to claim 7, characterized in that step a) includes alternatively selecting upper and lower macroblock lines from the field macroblock sequence sequentially line by line and arranging the selected lines in the order selected for each one. construct the pair of frame macroblocks. A method according to claim 8, characterized in that step b) includes using the respective texture information of the constructed pair of frame macroblocks as intra baseline prediction information if both of the field macroblock pairs are and use the respective texture information of the frame macroblock pair as residual preset information if both of the field macroblock pair are of the inter mode. A method according to claim 7, characterized in that step a) includes interpolating single field macroblocks in a vertical direction to construct the pair of frame macroblocks. A method according to claim 10, characterized in that step b) includes using the respective texture information of the constructed pair of frame macroblocks as intra prediction information if the single field macroblock is intra-mode. and use the respective texture information of the frame macroblock pair as residual prediction information if the single field macroblock is of the inter mode. A method according to claim 7, characterized in that step a) includes selecting one from the field macroblock pair and interpolating the selected field macroblock in a vertical direction to construct frame macroblock pair if the macroblock pair field is an intra-mode field macroblock and an inter-mode field macroblock. A method according to claim 12, characterized in that step b) includes using the texture information of the constructed pair of frame macroblocks as intra base prediction information if the frame macroblock is constructed from the macroblock. field-mode block in step a) and use texture information from the built-in pair of quadrocode macroblocks as residual prediction information if the frame-macroblock pair is constructed from the field-mode macroblock in step a). A method according to claim 7, characterized in that the pair of campovertically adjacent macroblock is included in a frame in which an interlaced video signal has been encoded in a MBAFF scheme. A method according to claim 7, characterized in that the macroblock-constructed texture information used as the texture prediction information is image data that has been decoded. 16. Method for performing interlayer prediction to encode or decode a video signal, the method being characterized by comprising the steps of: a) constructing N frame macroblock pairs from a vertically adjacent field macroblock pair of a base layer where N is an integer greater than 1; and (b) use respective texture information of the Npars constructed of frame macroblocks as the respective texture prediction information of N frame macroblock pairs located at different time positions in a current layer. Method according to claim 16, characterized in that N is 2 and step a) includes interpolating an upper macroblock of the field macroblock pair in a vertical direction to construct one of the N frame macroblock pairs and interpolating one. lower macroblock of the field macroblock pair in a vertical direction to construct another of the N frame macroblock pairs. 18. Method for performing interlayer prediction to encode or decode a video signal, the method being characterized by the steps of: a) separating each frame from a lower layer into a plurality of field images to allow the lower layer to have the same temporal resolution of that of an upper layer b) interpolating each of the separate field images in a vertical direction to extend each of the separate field images in a vertical direction; and c) using each of the interlayered field images in interlayer prediction of each upper layer frame. Method according to claim 18, characterized in that the interlayer prediction in step c) is texture prediction and / or motion prediction. A method for performing interlayer prediction to encode or decode a video signal, the CHARACTERIZED method comprises the steps of: a) interpolating each field image of a lower layer in a vertical direction to extend each field image in a vertical direction; and b) use each of the interleaved field images in interlayer prediction of each frame of an upper layer. 21. Method for performing interlayer prediction to encode or decode a video signal, the method being characterized by the steps of: a) separating each frame from an upper layer into a plurality of field images, b) downstream sampling of a lower layer to reduce lower layer images in a vertical direction; and c) use the descended sampled images in interlayer prediction of the upper layer separate field images. A method according to claim 21, characterized in that step a) includes separating each frame from the upper layer into a plurality of field images to allow the lower layer to have the same temporal resolution as that of the upper layer. . A method according to claim 21, characterized in that step c) includes applying interlayer prediction only for field images constructed using even lines of each frame of the upper layer between the separate field images.