JP2008539646A - Video coding method and apparatus for providing high-speed FGS - Google Patents

Abstract

The present invention relates to a method for reducing the amount of computation of a multi-layer based PFGS algorithm, and a video coding method and apparatus using the method.
The video encoding method includes a step of obtaining a predicted image for a current frame using a motion vector estimated with a predetermined accuracy, and quantizing a residual between the current frame and the predicted image, and then inversely quantizing the current frame. Generating a reconstructed image of the FGS layer, using the estimated motion vector to motion compensate the reference frame of the FGS layer and the reference frame of the base layer, and the motion compensated reference frame of the FGS layer and the motion compensated base The step includes a step of obtaining a difference from the reference frame of the hierarchy, a step of subtracting the restored image and the difference from the current frame, and a step of encoding the subtraction result.

Description

  The present invention relates to a video coding technique, and more particularly to a method for reducing the amount of computation of a multi-layer based PFGS algorithm, and a video coding method and apparatus using the method.

  With the development of information communication technology including the Internet, not only text and voice but also image communication is increasing. Existing character-centric communication methods cannot satisfy the diverse needs of consumers, and multimedia services that can accept various forms of information such as characters, video, and music are increasing. Multimedia data requires an enormous amount and a large capacity storage medium, and requires a wide bandwidth during transmission. Therefore, in order to transmit multimedia data including characters, video, and audio, it is essential to use a compression coding technique.

  The basic principle of compressing data is a process of removing duplicate elements of data. Spatial overlap in which the same color and object are repeated in the image, temporal overlap in which the adjacent frame is hardly changed in the video frame and the same sound is repeated in the audio, insensitive to frequencies with high human visual and perceptual ability The data can be compressed by removing perceptual duplication taking account of this. In a general video coding method, temporal overlap is removed by temporal filtering based on motion compensation, and spatial overlap is removed by spatial transformation.

  After removing data duplication, a transmission medium is required to transmit the generated multimedia, but the performance varies depending on the transmission medium. Currently used transmission media have various transmission speeds such as an ultrahigh-speed communication network capable of transmitting data of several tens of megabits per second and a mobile communication network having a transmission speed of 384 kbits per second. A scalable data coding method that can provide transmission media of various speeds or transmit multimedia at a transmission rate suitable for the transmission environment may be suitable for the multimedia environment. Absent.

  Extensibility also means that the bitstream cannot be completely decoded. It also includes spatial expansion, meaning video resolution, SNR scalability for video quality level, temporal scalability for frame rate, and the like.

  Currently, JVT, a joint working group of MPEG and ITU, Standardization work for realizing expandability in multiple layers based on H.264 is in progress. JVT has adopted existing FGS technology to provide SNR extensibility.

FIG. 1 is a diagram for explaining a conventional FGS technique. The FGS-based codec performs coding in a base layer and an FGS layer. In this specification, a prime (') code indicates an image generated through quantization / inverse quantization, that is, a restored image, not an original image. Specifically, in the current original frame 12, a certain block O includes a corresponding block M _B ′ of the restored base layer frame 11 on the left side by a motion vector and a corresponding block N of the restored base layer frame 12 on the right side. It becomes difference block R _B is subtracted the predicted block P _B from _{B '.} Thus, _{R B} is represented by the following equation (1).

_{R B = O-P B =} O- (M B '+ N B') / 2 ··· (1)

Difference block R _B is, R _{B ^Q,} also becomes difference block R _{B 'restored} through the inverse quantization process after being quantized by the quantization step QP _B base layer. Thereafter, in the FGS layer, the difference block R _B that is not quantized and the restored difference block R _B ′ are subtracted, and the resulting block Δ is quantized by a quantization step QP _F that is smaller than the quantization step of the base layer. (The smaller the quantization step, the lower the compression ratio). The quantized △ is represented by △ ^Q. After all, data to be transmitted to the decoder stage is △ ^Q of the base layer R _B ^Q and FGS layers.

FIG. 2 is a diagram for explaining a conventional PFGS technique. Compared to reducing the amount of data in the FGS layer using the quantized difference R _B ′ of the base layer where the existing FGS technology is restored, the PFGS technology is the FGS layer, and the right and left reference frames are also improved by the FGS technology. There using that improved, the newly updated using the left and right reference frames to calculate a new difference block R _F, quantizes the difference between the block R _{B 'which} has been quantized in this and the base layer that To increase performance. The _RF is represented by the following formula (2).

R _F = O-P _F = O- (M _F '+ N _F ') / 2 (2)

Here, M _F ′ is a region corresponding to the motion vector in the restored left reference frame 21 of the FGS layer, and N _F ′ corresponds to the motion vector of the restored right reference frame 23 in the FGS layer. It is an area to do.

  The advantage of the PFGS technology compared to the FGS technology is that the amount of data in the FGS layer can be reduced by increasing the quality of the left and right reference frames. However, since motion compensation is separately required even in the FGS layer, there is a disadvantage that the amount of calculation increases. As described above, the PFGS has an advantage that the performance is improved as compared with the existing FGS. However, the prediction signal must be generated by motion compensation for each FGS layer, and a residual signal corresponding to this must be generated. . Recent video codecs perform motion compensation by interpolating to 1/2 pixel or 1/4 pixel unit. If motion compensation is performed in units of 1/4 pixel, an image having a size four times the corresponding resolution must be generated.

  Further, H.C. H.264 based on H.264. In the case of H.264SE, the ½ pixel interpolation filter is a 6-tap filter, and its calculation is very complicated and occupies most of the amount of motion compensation. Therefore, the encoding process and the decoding process become complicated, which requires higher system resources, and is particularly problematic in fields where real-time encoding and decoding are required, such as real-time broadcasting and image conferencing.

  An object of the present invention is to provide a method capable of reducing the amount of calculation required at the time of motion compensation while maintaining the performance of the PFGS algorithm, and an apparatus using the method.

  Further, the present invention is not limited to the above objects, and further objects not mentioned can be clearly understood by those skilled in the art by the following.

  An FGS-based video encoding method for achieving the above object includes a step of obtaining a prediction image for a current frame using a motion vector estimated with a predetermined accuracy, and a residual between the current frame and the prediction image is quantized. And generating a restored image of the current frame by inverse quantization, motion-compensating the reference frame of the FGS layer and the reference frame of the base layer using the estimated motion vector, and the motion Determining a difference between a compensated FGS layer reference frame and the motion compensated base layer reference frame, subtracting the restored image and the difference from the current frame, and encoding the subtraction result Including.

  An FGS-based video encoding method for achieving the above object includes a step of obtaining a prediction image for a current frame using a motion vector estimated with a predetermined accuracy, and a residual between the current frame and the prediction image is quantized. And generating a restored image of the current frame by inverse quantization, and performing motion compensation on the reference frame of the FGS layer and the reference frame of the base layer using the estimated motion vector, Generating a prediction frame and a base layer prediction frame, obtaining a difference between the FGS layer prediction frame and the base layer prediction frame, and subtracting the restored image and the difference from the current frame. Encoding the subtraction result; Including.

  An FGS-based video encoding method for achieving the above object includes a step of obtaining a prediction image for a current frame using a motion vector estimated with a predetermined accuracy, and a residual between the current frame and the prediction image is quantized. And generating a restored image of the current frame by inverse quantization, determining a difference between a reference frame of the FGS layer and a reference frame of the base layer, and using the estimated motion vector Subtracting the difference from motion, subtracting the restored image and the motion compensated result from the current frame, and encoding the subtraction result.

  In order to achieve the above object, an FGS-based video decoding method includes a step of extracting base layer texture data, FGS layer texture data, and a motion vector from an input bitstream, and the base layer texture. Reconstructing a base layer frame from data, using the motion vector to perform motion compensation of a reference frame of an FGS layer and a reference frame of the base layer, and the motion compensated reference frame of the FGS layer and the motion compensated. A step of obtaining a difference from the reference frame of the base layer, and a step of adding the base layer frame, the texture data of the FGS layer, and the difference.

  In order to achieve the above object, an FGS-based video decoding method comprises: extracting base layer texture data, FGS layer texture data, and motion vectors from an input bitstream; and A step of restoring a base layer frame from the data, and a step of generating a prediction frame of the FGS layer and a base layer prediction frame by performing motion compensation on the reference frame of the FGS layer and the reference frame of the base layer using the motion vector And calculating the difference between the predicted frame of the FGS layer and the predicted frame of the base layer, and adding the texture data, the restored base layer frame and the difference.

  In order to achieve the above object, an FGS-based video decoding method comprises: extracting base layer texture data, FGS layer texture data, and motion vectors from an input bitstream; and Restoring a base layer frame from data; obtaining a difference between a reference frame of the FGS layer and a reference frame of the base layer; motion compensating the difference using the motion vector; and texture of the FGS layer Adding data, the restored base layer frame and the motion compensated result.

  In addition, the specific matter of embodiment is contained in detailed description and drawing.

  According to the present invention, the amount of computation can be greatly reduced in the implementation of PFGS, and the decoding process is changed accordingly. It can also be applied to H.264 SE standardized documents.

  Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can be realized in various different forms. This embodiment is provided to complete the disclosure of the present invention and to inform those having ordinary knowledge in the technical field to which the present invention pertains the scope of the invention. Defined only by. The same reference numerals denote the same components throughout the specification.

[First Embodiment]
FIG. 3 illustrates a high-speed PFGS method according to the first embodiment of the present invention.

  As in FIG. 2, the value quantized in the FGS hierarchy by the PFGS method is Δ, which is expressed by the following mathematical formula (3).

Δ = R _F −R _B ′ (3)

Here, R _F is the above equation (2), and R _B ′ is represented by the following equation (4).

_{R B '= O'-P B} = O' - (M B '+ N B') / 2 ··· (4)

Here, O ′ means an image restored by performing inverse quantization after quantizing the original image O by the quantization step QP _B of the base layer.

If the above formula (3) is arranged using R _F represented by the above formula (2) and R _B ′ represented by the above formula (4), Δ is represented by the following formula (5).

_{△ = O- (M F '+} N F') / 2- [O '- (M B' + N B ') / 2] ··· (5)

On the other hand, referring to FIG. 3, is the difference in the hierarchy between the reference frames △ _M and △ _N is represented by the following equation (6).

_{△ M = M F '-M B} '
_{△ N = N F '-N B} ' ··· (6)

  If the above formula (5) is arranged using the formula (6), Δ is expressed by the following formula (7).

△ = O-O '- ( △ M + △ N) / 2 ··· (7)

According to Equation (7), the encoder stage is quantized with QP _B from the original image O, and then the inversely quantized base layer image, that is, the restored base layer image O ′ and the inter-layer reference frame. It can be seen that Δ can be obtained by subtracting the average of the differences ((Δ _M + Δ _N ) / 2). Similarly, in the decoder stage, the image O can be restored by adding the restored images O ′ and Δ of the base layer and the average of the differences of the inter-layer reference frames.

  In the existing PFGS, motion compensation is performed using a motion vector having a pixel or sub-pixel accuracy (1/2 pixel, 1/4 pixel, etc.) generated by motion vector search. Recently, in order to increase the compression efficiency, it is common to perform motion vector search and motion compensation with high accuracy such as 1/2 pixel or 1/4 pixel. In the existing PFGS, for example, after a predicted image generated by motion compensation with 1/4 pixel accuracy is packed in units of integer pixels, the original image and the predicted image are subtracted and quantized. Packing is a method of selecting one pixel for every four pixels, for example, in the process of returning the reference image interpolated four times to the original size when performing a motion vector search in units of 1/4 pixel. Done in

However, since the data Δ in the FGS layer quantized by the high-speed PFGS according to the present invention is expressed as the above equation (7), the compression performance is not greatly affected even if the motion vector search is not performed with high accuracy. . The first term O and the second term O ′ on the right side of the equation (7) are portions to which motion vector search and motion compensation are not applied, so that there is no problem. However, although motion vector search and motion compensation are applied only to the third term ((Δ _M + Δ _M ) / 2), since this term is represented by a difference between layers, motion vector search is performed with high accuracy. And motion compensation is not very effective. This is because the image is relatively insensitive to pixel accuracy because the image compensated for motion with a predetermined pixel accuracy in the basic layer is subtracted from the image compensated for motion with the pixel accuracy in the enhancement layer. . Therefore, motion vector search and motion compensation can be performed with lower pixel accuracy than existing PFGS.

[Second Embodiment]
The above formula (5) in the first embodiment can also be described by the difference between the prediction signals as in the following formula (8). Here, _{P F} is the _{(M F '+ N F'} ) / 2, P B is the _{(M B '+ N B'} ) / 2.

Δ = O−O ′ − (P _F −P _B ) (8)

There are the following differences between the first embodiment and the second embodiment. After calculating the difference △ _M between layers of the reference _image, the △ _N previously in the first embodiment, dividing it in two. After calculating the predicted image P _F -P _B in each layer above the second embodiment obtains a difference between the prediction image. However, this is a difference in the realization of the algorithm, and the calculation results Δ are the same.

[Third Embodiment]
In the first embodiment and the second embodiment, after performing motion compensation first, a difference between images is obtained. However, it is also possible to perform motion compensation after changing the order and calculating the difference between the layers of the reference image first. Thus, according to the third embodiment, since motion compensation is performed on the differential signal, the influence of boundary padding is very small. Therefore, the boundary padding process can be omitted. Boundary padding means that pixels in the boundary portion are copied around the boundary in consideration of block matching being limited at the boundary portion of the frame during motion vector search.

  The difference Δ according to the third embodiment is expressed by the following mathematical formula (9). Here, mc (.) Is a function for performing motion compensation.

△ = O-O '- [ (mc (M F' -M B ') + mc (N F' -N B ')] / 2
... (9)

In the existing PFGS, when obtaining R _F or R _B in the above formula (3), the prediction results are subtracted or subtracted in the above three embodiments compared to direct prediction (motion vector search and motion compensation). In order to predict, the performance does not change so much even by the interpolation for improving the accuracy of the motion vector, that is, it has a feature insensitive to the interpolation.

Therefore, 1/4 pixel interpolation or 1/2 pixel interpolation can be omitted. In addition, H.M. In place of the H.264 1/2 pixel interpolation filter, a bilinear filter having a relatively small amount of calculation can be used. For example, a bilinear filter is applied to the third term of the mathematical formulas (7), (8), and (9). As a result, the performance degradation is reduced as compared with the case where the bilinear filter is directly applied to the prediction signal for obtaining R _F and R _B as in the existing PFGS.

[Fourth Embodiment]
The first to third embodiments are based on the mathematical formula (3). In other words, is the basic assumption that the values to be coded is further subtracted value and the difference R _B obtained from the difference R _F and base layer obtained from FGS layer. However, when the difference obtained from the FGS layer is very small, that is, when the temporal relevance is very large, such an approach may decrease coding performance. In this case, it is better to code only the difference obtained from the FGS layer. That is, only R _F is coded in the above equation (3).

  In this case, the above formulas (7) to (9) can be changed to the following formulas (10) to (12).

_{△ = O-P B - (} △ M + △ N) / 2 ··· (10)

Δ = O−P _B − (P _F −P _B ) (11)

_{△ = O-P B - [} (mc (M F '-M B') + mc (N F '-N B')] / 2
(12)

Eventually, Equations (10) to (12) show that the base layer image O ′ restored by Equations 7 to 9 is replaced with the predicted image P _B for the base layer image. Of course, the interpolation itself can be omitted for the third term of the above formulas (10) to (12), or the interpolation by the bilinear filter having a relatively small amount of calculation can be applied.

Although there are two P _B in the above formula (11), according to the present invention, the two are not exactly the same value. Although it accepts the motion vector estimated in the motion compensation process to generate a first P _B, 2 th P _B and lower accuracy than the motion vectors in the motion compensation process is the estimation for generating a P _F Motion vectors can be used. Alternatively, a filter that requires a small amount of calculation (for example, a bilinear filter) can be applied.

[Fifth Embodiment]
In PFGS, both restored reference frames are used to restore the current frame, so that a drift phenomenon occurs in which the degradation of the image quality of both reference frames is cumulatively reflected in the current frame. Leaky prediction is used to reduce this, and this is a method using a prediction image generated by polymerization between prediction images obtained from both reference frames and prediction images obtained from the base layer.

  According to leaky prediction used in the existing PFGS, a value coded in the FGS layer is expressed by the following equation (13).

Δ = O− [αP _F + (1−α) P _B ] (13)

  If this formula is arranged by the fifth embodiment, it is expressed by the following formula (14).

_{△ = O-P B -α (} P F -P B) ··· (14)

According to the equation (14), it can be understood from the equation (11) that the weighting factor (α) may be simply applied to the difference between predictions. Therefore, the present invention can also be applied to leaky prediction. That is, the interpolation itself may be omitted from (P _F −P _B ) or the bilinear filter may be applied, and the result may be multiplied by α.

FIG. 4 is a block diagram showing a configuration of the video encoder 100 according to the first embodiment of the present invention. The description of FIGS. 1 to 3 is based on a block which is a unit of motion vector search. However, the following description will be made on a frame basis including the block. In order to unify the expression, the identifier of the block is indicated by a subscript “F” character representing a frame. For example, a frame including a block of _{R B} is represented by _{F RB.} Of course, in the following, the prime (') display indicates data restored through quantization / inverse quantization.

The input current frame _FO is input by the motion vector search unit 105, the difference unit 115, and the difference calculation unit 170.

  The motion vector search unit 105 obtains a motion vector MV by performing a motion vector search for the current frame with reference to surrounding frames. The peripheral frame referred to in this way is referred to as a “reference frame”. In general, a block matching algorithm is widely used for such motion vector search. That is, while moving a given block in units of pixels or sub-pixels (1/2 pixel, 1/4 pixel, etc.) within a specific search region of the reference frame, the displacement when the error is minimized is estimated as a motion vector. To do. Although a fixed block can be used for motion vector search, a hierarchical method based on a hierarchical variable size block matching method (HVSBM) can also be used.

  If the motion vector search process is performed in units of subpixels, the reference frame must be upsampled or interpolated. When it is performed in units of 1/2 pixel, double upsampling or interpolation is required, and when it is performed in units of 1/4 pixel, upsampling or interpolation is required 4 times.

However, if the encoder 100 is in the open-loop codec like, the reference original peripheral frame F _M in the _frame, the F _N is used as it is, if turned closed loop codec shape, reconstructed base layer in the reference frame Peripheral frames F _MB ′ and F _NB ′ are used. Although the following description will focus on a closed loop codec, the present invention is not limited to this.

The motion vector MV obtained by the motion vector search unit 105 is provided to the motion compensation unit 110. The motion compensation unit 110 performs motion compensation on the reference frames F _MB ′ and F _NB ′ using the motion vector MV, and generates a predicted image _FPB for the current frame. If bi-directional reference is used, the predicted image can be calculated by averaging motion compensated reference frames. And if unidirectional reference is used, the predicted image may be the same as the motion compensated reference frame. In the following, the case of using a bi-directional reference in motion vector search and motion compensation will be described.

The differentiator 115 provides a residual signal F _RB which is calculated by subtracting the predicted image from the current frame to the converter 120.

The transform unit 120 performs a spatial transform on the residual signal F _RB to generate a transform coefficient F _RB ^T. As such a spatial transformation method, DCT, wavelet transformation or the like is used. When using DCT, the transform coefficient is a DCT coefficient, and when using wavelet transform, the transform coefficient is a wavelet coefficient.

  The quantization unit 125 quantizes the transform coefficient. The quantization means a process of expressing the transform coefficient represented by an arbitrary real value as a discontinuous value. For example, the quantization unit 125 can perform quantization by dividing the transform coefficient represented by an arbitrary real value by a predetermined quantization step and rounding the result to an integer value. The quantization step is applied to the base layer and generally has a larger value than the FGS layer.

The result of quantization by the quantization unit 125, that is, the quantization coefficient F _RB ^Q is provided to the entropy encoding unit 140 and the inverse quantization unit 130.

  The inverse quantization unit 130 inversely quantizes the quantization coefficient. Such an inverse quantization process is a process of restoring a value matched with an index generated in the quantization process using the same quantization step as that used in the quantization process.

The inverse transform unit 135 receives the inversely quantized result and performs inverse transform. Such an inverse transform is performed in the inverse process of the transform process of the transform unit 120, and specifically, an inverse DCT transform, an inverse wavelet transform, or the like is used. The adder 140 adds the result of the inverse transformation and the predicted image _FPB used in the motion compensation process of the motion compensation unit 110 to generate a restored image F _O ′ of the current frame.

The buffer 145 stores the result provided from the adder 140. Therefore, the buffer 145 can store not only the restored image F _O ′ of the current frame but also the reference frames F _MB ′ and F _NB ′ of the base layer restored in advance.

The motion vector changing unit 155 receives the motion vector MV and changes the accuracy of the motion vector. For example, if the accuracy of the motion vector MV is 1/4 pixel unit, the motion vector MV has one of 0, 0.25, 0.5, and 0.75 as a decimal value. Can do. As described above, according to the embodiment of the present invention, there is no significant difference in performance even if the high accuracy of the high-precision motion vector obtained in the base layer is not maintained at the time of motion compensation in the FGS layer. is there. Accordingly, the motion vector changing unit 155 changes the 1/4 pixel unit motion vector to a motion vector MV ₁ with a lower accuracy such as 1/2 pixel unit or pixel unit. Such a change process is performed by a simple method of truncating or rounding a portion exceeding the accuracy unit changed by the original motion vector.

The buffer 165 temporarily stores the reference frame of the FGS layer. Although not shown in detail, as the reference frame of the FGS layer, the restored frames F _MF ′ and F _NF ′ of the FGS layer are used, or original frames around the current frame are used.

The motion compensation unit 160 uses the changed motion vector MV ₁ to restore the reference frames F _MB ′ and F _NB ′ of the base layer provided from the buffer 145 and the reference of the FGS layer provided from the buffer 165. The frames F _MF ′ and F _NF ′ are motion-compensated, and as a result, the mc (F _MB ′), mc (F _NB ′), mc (F _MF ′), and mc (F _NF ′) are provided to the difference calculation unit 170. . Here, F _MF ′ is a forward reference frame in the FGS layer, F _NF ′ is a backward reference frame in the FGS layer, F _MB ′ is a forward reference frame in the base layer, and F _NB ′ is a backward reference frame in the base layer. Respectively.

When interpolation is necessary for motion compensation of the motion compensation unit 160, an interpolation filter of a form different from the interpolation filter used in the motion vector search unit 105 or the motion compensation unit 110 can be used. For example, when a motion vector MV _{1 in} units of 1/2 pixel is used during the motion compensation, H.264 is used for the interpolation. Instead of the H.264 6-tap filter, a bilinear filter with a small amount of calculation can be used. However, since the difference between the layers between the frames after motion compensation is obtained, the compression efficiency is not greatly affected.

The difference calculation unit 170 performs the motion-compensated FGS layer reference frames mc (F _MF ′) and mc (F _NF ′) and the motion-compensated base layer reference frames mc (F _MB ′) and mc (F _Find the difference from _NB '). _{That, △ M (= mc (F} MF ') -mc (F MB')), and _{△ N (= mc (F NF} ') -mc (F NB')) determined. Of course, in the case of unidirectional reference, only one difference can be obtained.

Then, the difference calculation unit 170, the difference △ _M, △ Average look of _N, the current from the frame F _O subtracting the mean of the restored image F _{O 'and} the differential. Of course, in the case of unidirectional reference, the process of obtaining the average is not required.

The result F _Δ subtracted from the difference calculation unit 170 is spatially transformed F _Δ ^T by the transformation unit 175, quantized through the quantization unit 180, and the quantized result F _Δ ^Q is transmitted to the entropy coding unit 150. Is done. The quantization step used in the quantization unit 180 generally has a smaller value than the quantization step used in the quantization unit 125.

The entropy encoding unit 150 losslessly encodes the motion vector MV estimated by the motion vector search unit 105, the F _RB ^Q provided from the quantization unit 125, and the F _Δ ^Q provided from the quantization unit 180. To generate a bitstream. As such a lossless coding method, Huffman coding, arithmetic coding, variable length coding, and other various methods are used.

  On the other hand, the configuration of the video encoder according to the second embodiment of the present invention can also be shown as in FIG. However, in the second embodiment, there is a difference in that a prediction frame is calculated first for each layer before obtaining a difference between layers. That is, there is a difference only in the operation of the difference calculation unit 170.

If according to the second embodiment, the difference calculation unit 170, the motion compensated FGS layer reference frames mc _{(F MF} ') to produce a predicted frame _{F PF} of FGS layer from, mc _{(F NF'),} the motion A base layer prediction frame F _BF is generated from the compensated base layer reference frames mc (F _MB ′) and mc (F _NB ′). The process of generating a prediction frame can be easily determined by averaging two motion compensated reference frames. Of course, in the case of unidirectional reference, the motion-compensated frame becomes the prediction frame as it is.

Then, the difference calculation unit 170, predicted frame _{F PF,} obtains the difference _F PF _{-F PB} between hierarchy from _{F PB,} subtracting the restored image _{F O} 'and the difference _F PF _{-F PB} the current from the frame _{F O} .

  FIG. 5 is a block diagram showing a configuration of a video encoder 300 according to the third embodiment of the present invention. In the first embodiment and the second embodiment, after performing motion compensation first, the difference between images is obtained. However, in the third embodiment, this order is changed, and the difference between the layers of the reference image is determined first. After the calculation, motion compensation is performed. In order to avoid an overlapping description with FIG.

The subtractor 390 subtracts the base layer restored reference frames F _MB ′ and F _NB ′ provided from the buffer 345 from the reference frames F _MF ′ and F _NF ′ of the FGS layer provided from the buffer 365, and The results F _MF ′ −F _MB ′ and F _NF ′ −F _NB ′ are provided to the motion compensation unit 360. Of course, there is only one difference in the case of unidirectional reference.

The motion compensation unit 360 uses the changed motion vector MV ₁ provided from the motion vector change unit 355 and uses the difference F _MF ′ −F _MB ′, F _NF ′ of the inter-layer reference frame provided from the differentiator 390. -F _NB 'is motion compensated. When a motion vector MV _{1 in} units of 1/2 pixel is used during the motion compensation, H.264 is used for the interpolation. Instead of the H.264 6-tap filter, a bilinear filter with a small amount of calculation is used, and the compression efficiency is not so much affected.

The difference calculation unit 370 calculates an average of the motion-compensated differences mc (F _MF ′ −F _MB ′) and mc (F _NF ′ −F _NB ′), and calculates the restored image F _O ′ from the current frame F _O and Subtract the average of the differences. Of course, in the case of unidirectional reference, the process of obtaining the average is not required.

6 and 7 are block diagrams showing the configuration of the video encoders 400 and 600 according to the fourth embodiment of the present invention. The difference between the fourth embodiment and the first to third embodiments are simply the current frame F _O from reconstructed frame F _{O 'rather} than the base layer by the difference calculation section, predicted frame F _PB of the base layer is subtracted It is a point.

6 corresponds to FIG. 4 (first embodiment), and FIG. 7 corresponds to FIG. 5 (third embodiment). Referring to FIG. 6, the difference calculation unit 470 is provided with a reference image _FPB of the base layer from the motion compensation unit 410 instead of the image F _O ′ of the base layer restored in the current frame F _O of FIG. Is shown. Therefore, the difference calculation unit 470 calculates the F _△ by subtracting the average of the difference △ _M, △ _N between the predicted image _{F PB} and hierarchy currently from the frame _{F O.}

Similarly, the difference calculation unit 670 in FIG. 7, the predicted current from the frame _{F O} image _{F PB} and motion compensated differential _{mc (F MF '-F MB'} ), mc of _{(F NF '-F NB')} Find F _Δ by subtracting the average.

On the other hand, the fourth embodiment corresponding to the second embodiment is not shown, but has the same configuration as the configuration diagram of FIG. However, there are some differences in the operation of the difference calculation unit 470. In the fourth embodiment corresponding to the second embodiment, the difference calculation unit 470 calculates the FGS layer predicted frame F _PF from the motion-compensated FGS layer reference frames mc (F _MF ′) and mc (F _NF ′). The base layer prediction frame F _BF is generated from the motion compensated base layer reference frames mc (F _MB ′) and mc (F _NB ′). Then, the difference calculation unit 170, predicted frame _{F PF,} obtains the difference _F PF _{-F PB} between hierarchy from _{F PB,} subtracting the restored image _{F O} 'and the difference _F PF _{-F PB} the current from the frame _{F O} determine the F _△ by.

If, by applying the herein leaky prediction (fifth exemplary embodiment), the difference calculation unit 170, the multiplying weighting factor to the difference _F PF _{-F PB} between layers (alpha), the said recovery current from the frame _{F O} F _Δ is obtained by subtracting the image F _O ′ and the multiplied result α × (F _PF −F _PB ).

  FIG. 8 is a block diagram showing a configuration of the video decoder 700 according to the first embodiment of the present invention.

  The entropy decoding unit 701 performs lossless decoding on the input bitstream, and extracts base layer texture data, FGS layer texture data, and motion vectors. Lossless decoding is a process that is performed in reverse of the lossless encoding process in the encoder stage.

The extracted base layer texture data F _PB ^Q is provided to an inverse quantization unit 705, the extracted FGS layer texture data F _Δ ^Q is provided to an inverse quantization unit 1045, and a motion vector MV is The motion compensation unit 720 and the motion vector change unit 730 are provided.

The inverse quantization unit 705 inversely quantizes the base layer texture data F _PB ^Q output from the entropy decoding unit 701. Such an inverse quantization process is a process of restoring a value matched with an index generated in the quantization process using the same quantization table used in the quantization process.

  The inverse transform unit 710 performs inverse transform on the inversely quantized result. Such inverse transformation reverses the conversion process of the encoder stage, and specifically, inverse DCT transformation, inverse wavelet transformation or the like is used.

As a result of the inverse transformation, the restored residual signal F _RB ′ is provided to the adder 715.

The motion compensation unit 720 generates a prediction image _FPB by performing motion compensation on the reference frames F _MB ′ and F _NB ′ restored in advance using the extracted motion vector MV and stored in the buffer 725. This is provided to the adder 715.

In the case of bi-directional prediction, the prediction image F _PB can be calculated by the average of the motion-compensated reference frames F _MB ′ and F _NB ′. In the case of uni-directional prediction, the motion-compensated reference frame becomes the prediction image F _PB as it is. obtain.

The adder 715 outputs the restored image F _O ′ of the base layer by adding the input F _RB ′ and _{FP B} , and the buffer 725 stores the restored image F _O ′.

Meanwhile, the inverse quantization unit 745 inversely quantizes the texture data F _△ ^Q of FGS layer, by the inverse transform unit 750 performs inverse transform on the inversely quantized result F _△ ^{T ',} it was restored F _Δ (F _Δ ′) is obtained and provided to the frame restoration unit 755.

The motion vector changing unit 730 receives the extracted motion vector MV and reduces the accuracy of the motion vector. For example, if the accuracy of the motion vector MV is 1/4 pixel unit, the motion vector MV has one of 0, 0.25, 0.5, and 0.75 as a decimal value. Can do. The motion vector changing unit 155 changes the motion vector in ¼ pixel units to a motion vector MV ₁ having a lower accuracy than ½ pixel unit, pixel unit, or the like.

The motion compensation unit 735 uses the changed motion vector MV ₁ to restore the reference frames F _MB ′ and F _NB ′ of the base layer provided from the buffer 725 and the reference of the FGS layer provided from the buffer 740. The frames F _MF ′ and F _NF ′ are motion-compensated, and as a result, mc (F _MB ′), mc (F _NB ′), mc (F _MF ′), mc (F _NF ′) are provided to the frame restoration unit 755. .

When the motion vector MV _{1 in} units of 1/2 pixel is used at the time of the motion compensation, H.264 is used for the interpolation. Instead of the H.264 6-tap filter, a bilinear filter with a small amount of calculation is used, and it still does not have a great influence on the compression efficiency.

The frame restoration unit 755 includes the motion-compensated FGS layer reference frames mc (F _MF ′) and mc (F _NF ′) and the motion-compensated base layer reference frames mc (F _MB ′) and mc ( The difference from F _NB ′) is obtained. _{That, △ M (= mc (F} MF ') -mc (F MB')), and _{△ N (= mc (F NF} ') -mc (F NB')) determined. Of course, in the case of unidirectional reference, only one difference is obtained.

The frame restoration unit 755 obtains an average of the difference △ _M, △ _N, the F _△ 'and reconstructed image F _O base _layer' and adds the average of the difference. As a result, a restored image F _OF ′ of the FGS hierarchy is generated. Of course, in the case of unidirectional reference, the process of obtaining the average is not required.

Buffer 740 stores the restored image F _OF '. Of course, the image F _MF ′, F _BF ′ restored in advance can also be stored in the buffer 740.

  On the other hand, the configuration of the video decoder according to the second embodiment of the present invention can also be shown as in FIG. However, in the second embodiment, there is a difference in that a prediction frame is calculated first for each layer before obtaining a difference between layers. That is, there is a difference only in the operation of the frame restoration unit 755.

If according to the second embodiment, a frame restoration unit 755 generates a predicted frame _{F PF} of the reference frame mc _{(F MF} ') of the motion compensated FGS layer FGS layer from, mc _{(F NF'),} the motion compensation The base layer prediction frame F _BF is generated from the base layer reference frames mc (F _MB ′) and mc (F _NB ′). The process of generating a prediction frame is easily determined by averaging two motion compensated reference frames. Of course, in the case of unidirectional reference, the motion-compensated frame becomes the prediction frame as it is.

Then, the frame restoration unit 755 obtains a difference F _PF -F _PB between layers from the prediction frames F _PF and F _PB , and calculates the F _Δ ′, the restored image F _O ′ of the base layer, and the difference F _PF -F. Add _PB .

  FIG. 9 is a block diagram showing a configuration of a video decoder 900 according to the third embodiment of the present invention. In the video decoders according to the first and second embodiments, after performing motion compensation, the difference between the images is obtained. In the third embodiment, the difference between the layers of the reference image is changed by changing the order. After the calculation, motion compensation is performed. In order to avoid an overlapping description with FIG.

The subtractor 960 subtracts the base layer restored reference frames F _MB ′ and F _NB ′ provided from the buffer 925 from the FGS layer reference frames F _MF ′ and F _NF ′ provided from the buffer 940, and the result F _MF ′ −F _MB ′ and F _NF ′ −F _NB ′ are provided to the motion compensation unit 935. Of course, there is only one difference in the case of unidirectional reference.

The motion compensation unit 935 provides the difference F _MF ′ −F _MB ′, F _NF ′ − of the inter-layer reference frame provided from the differentiator 390 using the changed motion vector MV ₁ provided from the motion vector change unit 930. F _NB 'is motion compensated. When a motion vector MV _{1 in} units of 1/2 pixel is used during the motion compensation, H.264 is used for the interpolation. Instead of the H.264 6-tap filter, a bilinear filter with a small amount of calculation is used, and it still does not have a great influence on the compression efficiency.

The frame restoration unit 955 obtains the average of the motion-compensated differences mc (F _MF ′ −F _MB ′) and mc (F _NF ′ −F _NB ′), and F _Δ ′ provided from the inverse transform unit 950 and the basis The restored image F _O ′ of the hierarchy is added to the average of the differences. Of course, in the case of unidirectional reference, the process of obtaining the average is not required.

  10 to 11 are block diagrams showing configurations of video decoders 1000 and 1200 according to the fourth embodiment of the present invention.

The difference between the video decoder of the fourth embodiment and the video decoders of the first to third embodiments is that the prediction of the base layer is simply performed instead of the frame F _O ′ of the base layer restored in the addition process of the frame restoration unit. Only in that the frame _FPB is used.

10 corresponds to FIG. 8 (first embodiment), and FIG. 11 corresponds to FIG. 9 (third embodiment). Referring to FIG. 10, the frame restoration unit 1055 indicates that the base layer reference image _FPB is provided from the motion compensation unit 1020 instead of the base layer restored image F _O ′ of FIG. 8. ing. Accordingly, the frame restoration unit 1055 restores the FGS layer by adding F _Δ 'provided from the inverse transformation unit 1050, the predicted image F _PB, and the average of the differences Δ _M and Δ _N between layers. Image F _OF 'can be obtained.

Similarly, in FIG. 11, the frame recovery unit 1255 F _△ provided from the inverse transformation unit 1250 'and the prediction image _{F PB} is provided from the motion compensation unit 1220, a motion-compensated difference mc _{(F MF'} - The restored image F _OF ′ of the FGS hierarchy can be obtained by adding the average of F _MB ′) and mc (F _NF ′ −F _NB ′).

On the other hand, the fourth embodiment corresponding to the second embodiment is not shown, but has the same configuration as the configuration diagram of FIG. However, there is only a slight difference in the operation of the frame restoration unit 1255. In the fourth embodiment corresponding to the second embodiment, generates a predicted frame _{F PF} of FGS layer frame restoration unit 1255 reference frame mc of the motion compensated FGS layer _{(F MF} ') from, mc _{(F NF')} Then, the base layer predicted frame F _BF is generated from the motion compensated base layer reference frames mc (F _MB ′) and mc (F _NB ′). Then, the frame restoration unit 1255 obtains a difference F _PF −F _PB between layers from the prediction frames F _PF and F _PB, and F _Δ ′ provided from the inverse conversion unit 1250 and the prediction image provided from the motion compensation unit 1220. and F _PB, by adding the difference _F PF _{-F PB} between a predicted frame, it is possible to obtain an image _{F oF} 'restored the FGS layer.

If leaky prediction (fifth embodiment) is applied here, the frame restoration unit 1255 multiplies the difference F _PF -F _PB between the layers by a weighting factor (α), and the F _Δ ′ and the restored image F _OF ′ is obtained by adding F _O ′ and the multiplied result α × (F _PF −F _PB ).

  FIG. 12 is a configuration diagram of a system for realizing the video encoder 100, 300, 400, 600 or the video decoder 700, 900, 1000, 1200 according to an embodiment of the present invention. The system represents, for example, a TV, set-top box, desktop, laptop computer, palmtop computer, PDA, video or image storage device (eg, VCR, DVR, etc.). The system may be a combination of the devices, or the device may be included as part of another device. The system may be configured to include at least one or more video sources 1310, one or more input / output devices 1320, a processor 1340, a memory 1350, and a display device 1330.

  Video source 1310 represents a TV receiver, VCR, or other video storage device. Also, the source 1310 indicates one or more network connections for receiving video from a server using the Internet, WAN, LAN, terrestrial broadcasting system, cable network, satellite communication network, wireless network, telephone network, etc. . In addition, the source indicates a combination of the networks, or the network included as a part of another network.

  The input / output device 1320, the processor 1340, and the memory 1350 communicate via a communication medium 1360. The communication medium 1360 represents a communication bus, a communication network, or one or more internal connection circuits. Input video data received from the source 1310 may be processed by the processor 1340 by one or more software programs stored in the memory 1350 and executed by the processor 1340 to generate output video provided to the display device 1330. Can be done.

  In particular, the software program stored in the memory 1350 can include a scalable codec based on wavelet transforms that perform the method according to the invention. The encoder or the codec may be stored in the memory 1350, read by a storage medium such as a CD-ROM or a floppy (registered trademark) disk, or downloaded from a predetermined server via various networks.

  The embodiments of the present invention have been described above with reference to the accompanying drawings. However, those skilled in the art to which the present invention pertains have ordinary skill in the art without changing the technical idea or essential features. It can be understood that it can be implemented in other specific forms. Accordingly, it should be understood that the above-described embodiments are illustrative in all aspects and not limiting.

It is drawing explaining the conventional FGS technique. 1 is a diagram illustrating a conventional PFGS technique. 1 is a diagram illustrating a high-speed PFGS method according to a first embodiment of the present invention. It is a block diagram which shows the structure of the video encoder by 1st Embodiment of this invention. It is a block diagram which shows the structure of the video encoder by 3rd Embodiment of this invention. It is a block diagram which shows the structure of the video encoder by 4th Embodiment of this invention. It is a block diagram which shows the structure of the video encoder by 4th Embodiment of this invention. It is a block diagram which shows the structure of the video decoder by 1st Embodiment of this invention. It is a block diagram which shows the structure of the video decoder by 3rd Embodiment of this invention. It is a block diagram which shows the structure of the video decoder by 4th Embodiment of this invention. It is a block diagram which shows the structure of the video decoder by 4th Embodiment of this invention. 1 is a configuration diagram of a system for realizing a video encoder or a video decoder according to an embodiment of the present invention.

Explanation of symbols

100, 300, 400, 600 Video encoder 700, 900, 1000, 1200 Video decoder 105 Motion vector search unit 110, 160 Motion compensation unit 120 Conversion unit 125 Quantization unit 130 Inverse quantization unit 135 Inverse conversion unit 150 Entropy encoding unit 155 Motion vector change unit 170 Difference calculation unit 175 Conversion unit 180 Quantization unit 701 Entropy decoding unit 705, 745 Inverse quantization unit 710, 750 Inverse conversion unit 720, 735 Motion compensation unit 730 Motion vector change unit 755 Frame restoration unit

Claims (50)

Obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Generating a restored image of the current frame by quantizing a residual between the current frame and the predicted image and then inverse-quantizing;
Motion-compensating a reference frame of the FGS layer and a reference frame of the base layer using the estimated motion vector;
Obtaining a difference between the reference frame of the motion compensated FGS layer and the reference frame of the motion compensated base layer;
Subtracting the restored image and the difference from the current frame;
Encoding the subtraction result;
A video encoding method based on FGS.   The motion vector used in the step of motion compensation of the reference frame of the FGS layer and the reference frame of the base layer using the estimated motion vector has lower accuracy than the estimated motion vector. Item 4. The FGS-based video encoding method according to Item 1.   The obtained difference is an average of the first difference between the forward reference frame in the FGS layer and the forward reference frame in the base layer, and the second difference between the reverse reference frame in the FGS layer and the reverse reference frame in the base layer. The FGS-based video encoding method according to claim 1, wherein:   3. The FGS-based video according to claim 2, wherein an interpolation filter having a different form from an interpolation filter used for obtaining a prediction image for the current frame is used when interpolation is necessary for the motion compensation. Encoding method. The step of encoding the subtraction result includes:
Generating a conversion coefficient by converting the subtraction result;
Quantizing the transform coefficient to generate a quantized coefficient;
Lossless encoding the quantized coefficients;
The FGS-based video encoding method according to claim 1, further comprising: Obtaining a predicted image for the current frame;
Estimating a motion vector using the current frame and the restored frame of at least one base layer as a reference frame;
Motion compensating the reference frame with the estimated motion vector;
Determining the predicted image by averaging the motion compensated reference frames;
The FGS-based video encoding method according to claim 1, further comprising: Obtaining a predicted image for the current frame;
Estimating a motion vector using the current frame and an original frame around the current frame as a reference frame;
Motion compensating the reference frame with the estimated motion vector;
Determining the predicted image by averaging the motion compensated reference frames;
The FGS-based video encoding method according to claim 1, further comprising:   The FGS-based video encoding method according to claim 1, wherein the reference frame of the FGS layer is an original frame around the current frame, and the reference frame of the base layer is a peripheral frame restored in the base layer. .   The FGS-based video of claim 1, wherein the reference frame of the FGS layer is a peripheral frame restored in the FGS layer, and the reference frame of the base layer is a peripheral frame restored in the base layer. Encoding method.   6. The FGS base according to claim 5, wherein the magnitude of the quantization step used to quantize the transform coefficient is smaller than the magnitude of the quantization step used to quantize the residual. Video encoding method. Obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Generating a restored image of the current frame by quantizing a residual between the current frame and the predicted image and then inverse-quantizing;
Generating an FGS layer prediction frame and a base layer prediction frame by performing motion compensation on an FGS layer reference frame and a base layer reference frame using the estimated motion vector;
Obtaining a difference between the predicted frame of the FGS layer and the predicted frame of the base layer;
Subtracting the restored image and the difference from the current frame;
Encoding the subtraction result;
A video encoding method based on FGS.   The motion vector used in the step of generating the prediction frame of the FGS layer and the prediction frame of the base layer by performing motion compensation on the reference frame of the FGS layer and the reference frame of the base layer using the estimated motion vector, The method of claim 11, wherein the FGS-based video encoding method has a lower accuracy than the estimated motion vector.   The FGS layer prediction frame is an average of the motion compensated FGS layer reference frames, and the base layer prediction frame is an average of the motion compensated base layer reference frames. The FGS-based video encoding method according to claim 11.   13. The FGS-based video according to claim 12, wherein an interpolation filter having a different form from the interpolation filter used in the step of obtaining a prediction image for the current frame is used when interpolation is required for the motion compensation. Encoding method. The step of encoding the subtraction result includes:
Generating a conversion coefficient by converting the subtraction result;
Quantizing the transform coefficient to generate a quantized coefficient;
Lossless encoding the quantized coefficients;
The FGS-based video encoding method according to claim 11, further comprising:   The FGS according to claim 15, wherein the magnitude of the quantization step used to quantize the transform coefficient is smaller than the magnitude of the quantization step used to quantize the residual. Base video encoding method. Obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Generating a restored image of the current frame by quantizing a residual between the current frame and the predicted image and then inverse-quantizing;
Obtaining a difference between the reference frame of the FGS layer and the reference frame of the base layer;
Motion compensating the difference using the estimated motion vector;
Subtracting the restored image and the motion compensated result from the current frame;
Encoding the subtraction result;
A video encoding method based on FGS.   The method of claim 17, wherein a motion vector used in the motion compensation step has a lower accuracy than the estimated motion vector.   The method of claim 17, wherein the subtracted motion compensated result is an average of the motion compensated differences in the motion compensation step.   19. The FGS-based video of claim 18, wherein an interpolation filter having a different form from the interpolation filter used in the step of obtaining a prediction image for the current frame is used when interpolation is required for the motion compensation. Encoding method. The step of encoding the subtraction result includes:
Generating a conversion coefficient by converting the subtraction result;
Quantizing the transform coefficient to generate a quantized coefficient;
Lossless encoding the quantized coefficients;
The FGS-based video encoding method according to claim 17, further comprising:   The FGS of claim 21, wherein the magnitude of the quantization step used to quantize the transform coefficient is smaller than the magnitude of the quantization step used to quantize the residual. Base video encoding method. Obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Motion-compensating the reference frame of the FGS layer and the reference frame of the base layer with a motion vector having a lower accuracy than the motion vector;
Obtaining a difference between the reference frame of the motion compensated FGS layer and the reference frame of the motion compensated base layer;
Subtracting the predicted image and the difference from the current frame;
Encoding the subtraction result;
A video encoding method based on FGS. Obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Generating a FGS layer predicted frame and a base layer predicted frame by motion compensating the FGS layer reference frame and the base layer reference frame with a motion vector having a lower accuracy than the motion vector;
Obtaining a difference between the predicted frame of the FGS layer and the predicted frame of the base layer;
Subtracting the predicted image and the difference from the current frame;
Encoding the subtraction result;
A video encoding method based on FGS.   The method of claim 24, further comprising multiplying the obtained difference by a weighting factor (α). Obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Obtaining a difference between the reference frame of the FGS layer and the reference frame of the base layer;
Motion-compensating the difference with a motion vector that is less accurate than the motion vector;
Subtracting the restored image and the motion compensated result from the current frame;
Encoding the subtraction result;
A video encoding method based on FGS. Extracting base layer texture data, FGS layer texture data, and motion vectors from the input bitstream;
Restoring a base layer frame from the base layer texture data;
Using the motion vector to perform motion compensation of a reference frame of an FGS layer and a reference frame of a base layer;
Obtaining a difference between the reference frame of the motion compensated FGS layer and the reference frame of the motion compensated base layer;
Adding the base layer frame, the texture data of the FGS layer, and the difference;
An FGS-based video decoding method comprising:   The method of claim 27, wherein the motion vector used in the motion compensation step has a lower accuracy than the extracted motion vector.   The difference is an average of the first difference between the forward reference frame in the FGS layer and the forward reference frame in the base layer, and the second difference between the reverse reference frame in the FGS layer and the reverse reference frame in the base layer. 28. The FGS-based video decoding method according to claim 27.   29. The FGS-based video data according to claim 28, wherein an interpolation filter having a different form from the interpolation filter used in the step of restoring the base layer frame is used when interpolation is required for the motion compensation. Coding method.   The FGS base according to claim 27, wherein the texture data of the FGS layer to be added is a result of performing an inverse quantization process and an inverse transform process on the extracted texture data of the FGS layer. Video decoding method. Restoring the base layer frame comprises:
Dequantizing the base layer texture information;
Inverse transforming the inverse quantization result;
Generating a predicted image from a reference frame of a base layer previously restored using the motion vector;
Adding the prediction image and the inverse transformation result;
32. The FGS-based video decoding method according to claim 31, further comprising:   The size of the quantization step used in the inverse quantization applied to the texture data of the FGS layer is smaller than the size of the quantization step used in the inverse quantization of the step of restoring the base layer frame. 33. The FGS-based video decoding method according to claim 32, wherein: Extracting base layer texture data, FGS layer texture data, and motion vectors from the input bitstream;
Restoring a base layer frame from the base layer texture data;
Generating a FGS layer prediction frame and a base layer prediction frame by motion-compensating an FGS layer reference frame and a base layer reference frame using the motion vector;
Obtaining a difference between the predicted frame of the FGS layer and the predicted frame of the base layer;
Adding the texture data, the restored base layer frame and the difference;
An FGS-based video decoding method comprising:   The method of claim 34, wherein the motion vector used in the motion compensation has a lower accuracy than the extracted motion vector.   The FGS-based video data according to claim 35, wherein an interpolation filter having a different form from the interpolation filter used in the step of restoring the base layer frame is used when interpolation is required for the motion compensation. Coding method.   The FGS base according to claim 34, wherein the texture data of the FGS layer to be added is a result of performing an inverse quantization process and an inverse transform process on the extracted texture data of the FGS layer. Video decoding method. Extracting base layer texture data, FGS layer texture data, and motion vectors from the input bitstream;
Restoring a base layer frame from the base layer texture data;
Obtaining a difference between the reference frame of the FGS layer and the reference frame of the base layer;
Motion compensating the difference using the motion vector;
Adding the FGS layer texture data, the restored base layer frame, and the motion compensated result;
An FGS-based video decoding method comprising:   The method of claim 38, wherein the added motion compensated result is an average of the motion compensated differences.   The FGS-based video decoding method of claim 38, wherein a motion vector used in the motion compensation of the difference has a lower accuracy than the extracted motion vector.   The FGS-based video data according to claim 40, wherein an interpolation filter having a different form from the interpolation filter used in the step of restoring the base layer frame is used when interpolation is necessary for the motion compensation. Coding method.   The FGS base according to claim 38, wherein the texture data of the added FGS layer is a result of performing an inverse quantization process and an inverse transform process on the extracted texture data of the FGS layer. Video decoding method. Extracting base layer texture data, FGS layer texture data, and motion vectors from the input bitstream;
Restoring the predicted image of the base layer frame from the texture data of the base layer using the extracted motion vector;
Motion-compensating the reference frame of the FGS layer and the reference frame of the base layer with a motion vector having a lower accuracy than the motion vector;
Obtaining a difference between the reference frame of the motion compensated FGS layer and the reference frame of the motion compensated base layer;
Adding the FGS layer texture data, the predicted image, and the difference;
An FGS-based video decoding method comprising: Extracting base layer texture data, FGS layer texture data, and motion vectors from the input bitstream;
Restoring the predicted image of the base layer frame from the texture data of the base layer using the extracted motion vector;
Generating a FGS layer predicted frame and a base layer predicted frame by motion compensating the FGS layer reference frame and the base layer reference frame with a motion vector having a lower accuracy than the motion vector;
Obtaining a difference between the predicted frame of the FGS layer and the predicted frame of the base layer;
Adding the FGS layer texture data, the predicted image, and the difference;
An FGS-based video decoding method comprising:   45. The FGS-based video decoding method according to claim 44, further comprising a step of multiplying the obtained difference by a weighting factor ([alpha]). Extracting base layer texture data, FGS layer texture data, and motion vectors from the input bitstream;
Restoring the predicted image of the base layer frame from the texture data of the base layer using the extracted motion vector;
Obtaining a difference between the reference frame of the FGS layer and the reference frame of the base layer;
Motion-compensating the difference with a motion vector that is less accurate than the motion vector;
Adding the FGS layer texture data, the predicted image, and the difference;
An FGS-based video decoding method comprising: Means for obtaining a predicted image for the current frame using a motion vector estimated with a predetermined accuracy;
Means for generating a reconstructed image of the current frame by dequantizing the residual between the current frame and the predicted image and then dequantizing;
Means for generating a prediction frame of the FGS layer and a prediction frame of the base layer by performing motion compensation on the reference frame of the FGS layer and the reference frame of the base layer using the estimated motion vector;
Means for obtaining a difference between the prediction frame of the FGS layer and the prediction frame of the base layer;
Means for subtracting the restored image and the difference from the current frame;
Means for encoding the subtraction result;
A video encoder based on FGS, comprising:   The FGS-based video of claim 47, wherein a motion vector used in the means for generating the FGS layer prediction frame and the base layer prediction frame has lower accuracy than the estimated motion vector. Encoder. Means for extracting texture data of the base layer, texture data of the FGS layer, and a motion vector from the input bitstream;
Means for restoring a base layer frame from the base layer texture data;
Means for generating a FGS layer prediction frame and a base layer prediction frame by motion-compensating an FGS layer reference frame and a base layer reference frame using the motion vector;
Means for obtaining a difference between the prediction frame of the FGS layer and the prediction frame of the base layer;
Means for adding the texture data, the restored base layer frame and the difference;
A video decoder based on FGS.   50. The FGS-based video decoder of claim 49, wherein a motion vector used in the motion compensation has a lower accuracy than the extracted motion vector.