Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
  • H04N19/615—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding using motion compensated temporal filtering [MCTF]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]

Definitions

• Apparatuses and methods consistent with the present invention relate to a video coding technology, and more particularly to reducing mismatch between an encoder and a decoder in a motion compensated temporal filtering (MCTF) process.
• MCTF motion compensated temporal filtering
• data-compression refers to a process of removing redundant elements from data. That is, data can be compressed by removing spatial redundant elements (e.g., repetition of the same color or object in an image), temporal redundant elements (e.g., little or no variation between adjacent frames in moving picture frames or repetition of the same audio sound), and perceptual redundant elements (e.g., high frequencies beyond the noticeable range of human visual and perceptual capabilities) from the data.
• spatial redundant elements e.g., repetition of the same color or object in an image
• temporal redundant elements e.g., little or no variation between adjacent frames in moving picture frames or repetition of the same audio sound
• perceptual redundant elements e.g., high frequencies beyond the noticeable range of human visual and perceptual capabilities
• transmission mediums may have different transmission rates depending on the type of transmission medium.
• various types of transmission mediums having different transmission rates such as a high-speed communication network capable of transmitting data with a transmission rate of several tens of Mbit/sec or a mobile communication network capable of transmitting data with a transmission rate of 384 kbit/sec, are used for transmitting multimedia data.
• a scalable video coding scheme is more suitable for multimedia environments since it supports various transmission mediums having different transmission rates while allowing the multimedia data to be transmitted with a transmission rate appropriate for the transmission environment.
• the scalable video coding scheme refers to a coding scheme capable of adjusting the resolution, frame rate and SNR (signal-to-noise ratio) of a video signal by partially truncating a compressed bit stream according to the variable conditions of a transmission environment, such as a transmission bit rate, a transmission error rate or system resources.
• An MCTF technique has been widely used in the scalable video coding scheme for supporting temporal scalability, such as H.264 SE (scalable extension).
• H.264 SE scalable extension
• a 5/3 MCTF technique using both left and right adjacent frames compresses data with high efficiency and can be applied to both temporal scalability and SNR scalability, such that the 5/3 MCTF technique has been adopted in the standard draft for H.264 SE, which is being prepared by the Moving Picture Experts Group (MPEG).
• MPEG Moving Picture Experts Group
• FIG. 1 is view illustrating a 5/3 MCTF structure for sequentially performing a prediction step and an update step with respect to one GOP (group of pictures).
• the prediction step and the update step are sequentially repeated in the MCTF structure according to the order of temporal levels.
• a frame obtained through the prediction step is referred to as a high-pass frame (H) and a frame obtained through the update step is referred to as a low-pass frame (L).
• the prediction step and the update step may be repeated until one final low-pass frame (L) has been obtained.
• FIG. 2 is a view illustrating the prediction step and the update step in detail.
• subscripts (t and t+1) represent temporal levels and superscripts (2, 1, 0, -1, and -2) represent the temporal orders, respectively.
• constants (a and b) represent the weight ratio of each frame in the prediction step or the update step.
• a high-pass frame (H ) is obtained based on the difference between a current frame (L °) and a prediction frame predicted from left and right adjacent reference frames (L " and L ).
• the left and right adjacent reference frames (L ⁇ and L ') which have been used in the previous prediction step, are transformed by using the high-pass frame (H ) obtained in the prediction step.
• the update step is carried out in order to remove the high-pass element, that is, the high-pass frame (H ) from the reference frame, thus the update step is similar to the low-pass filtering process. Since transformed left and right adjacent reference frames (L " and L ) have no high-pass elements, the coding performance can be improved.
• frames of the GOP are sequentially arranged corresponding to the temporal levels thereof, one H frame (high-pass frame) is obtained by performing the prediction step per each temporal level, and two reference frames used in the prediction step are transformed by using the H frame (update step). If the above process is performed with respect to N frames located in one temporal level, N/2 H frames and N/2 L frames can be obtained. Therefore, if this process is repeated until one L frame remains, M-I H frames and one L frame may result on the assumption that the GOP has M frames. Thereafter, the remaining frames are quantized and the MCTF process ends.
• optimum blocks are obtained by performing motion estimation with respect to the left and right adjacent frames as shown in FIG. 2, and an optimum prediction block is obtained by using the optimum blocks.
• blocks included in the H frame can be obtained by calculating the difference between the optimum prediction block and the original block. Since FIG. 2 represents the bi-directional prediction, constant a is -1/2. However, if a single directional prediction is performed using the left or right reference frame, constant a may be -1.
• the update step may serve to remove the high-pass elements of the left and right reference frames by using the differential image obtained through the prediction step, that is, the H frame value.
• the left and right adjacent reference frames (L ⁇ and L ') are transformed to reference frames (L ⁇ and L ') having no high-pass elements through the update step.
• the above MCTF structure is different from the conventional data-compression scheme, such as MPEG-4 or H.264, in that the MCTF structure includes a video codec having an open-loop structure and uses the update step in order to reduce the drift error.
• the open-loop structure uses un-quantized left and right reference frames in order to obtain the differential image (high-pass frame).
• the conventional video codec mainly uses a closed-loop structure, which previously codes and quantizes the reference frames and then decodes the reference frames.
• Such an MCTF open-loop codec is better than the closed-loop codec if SNR scalability is applied, that is, when the quality of the reference frame used in a decoder side is poorer than that of an encoder side.
• the open-loop structure has an error drift problem, which results from the mismatch of reference frames between the encoder and the decoder.
• the MCTF technique removes the high-pass elements of the differential image from the L frame of the next temporal level through the update step, thereby improving the data- compression efficiency and reducing the amount of the error drift derived from the open-loop structure.
• the amount of error drift may decrease through the update step, the mismatch between the encoder and the decoder still remains in the open-loop structure, such that the performance is inevitably degraded. Disclosure of Invention
• the first is the mismatch in the prediction step.
• the left and right reference frames are used to obtain the H frame.
• the H frame derived from the left and right reference frames may not be an optimum signal in the decoder side.
• the left and right reference frames must be transformed through the update step and transformed into the H frames in the next temporal level so as to be quantized, it is difficult to previously quantize the reference frames if the MCTF structure has an open-loop structure, rather than a closed-loop structure.
• the second kind of mismatch is the mismatch in the update step.
• the high-pass frame (H ) is used for changing the left and right adjacent reference frames (L ⁇ and L ').
• the mismatch may occur between the encoder and the decoder.
• an aspect of the present invention is to provide an apparatus and method capable of improving the video compression efficiency by reducing the drift error between an encoder and a decoder in an MCTF video codec.
• Another aspect of the present invention is to provide an apparatus and method capable of effectively re-estimating a high-pass frame in an MCTF video codec.
• a video encoding method including the steps of (a) dividing input frames into one final low-pass frame and at least one high-pass frame by a motion compensated temporal filtering, (b) coding the final low-pass frame and then decoding the coded final low-pass frame, (c) re-estimating the high-pass frame by using the decoded final low-pass frame, and (d) coding the re-estimation high-pass frame.
• a video decoding method including the steps of (a) restoring a final low-pass frame and at least one high-pass frame from texture data included in an input stream, and (b) restoring low-pass frames located in a lowest time level from among the final low-pass frame and at least one high-pass frame, in which step (b) includes the substeps of (bl) inversely predicting the high-pass frame by using a first low-pass frame located in a predetermined temporal level as a reference frame, thereby restoring a second low-pass frame corresponding to the high-pass frame, and (b2) inversely updating the first low- pass frame using the restored high-pass frame.
• a video encoder including a means for dividing input frames into one final low-pass frame and at least one high-pass frame by a motion compensated temporal filtering, a means for coding the final low-pass frame and then decoding the coded final low-pass frame, a means for re-estimating the high-pass frame by using the decoded final low-pass frame, and a means for coding the re-estimation high-pass frame.
• a video decoder including a first means for restoring a final low-pass frame and at least one high-pass frame from texture data included in an input stream, and a second means for restoring low-pass frames located in a lowest time level from among the final low- pass frame and at least one high-pass frame, in which the second means includes a means for inversely predicting the high-pass frame by using a first low-pass frame located in a predetermined temporal level as a reference frame, thereby restoring a second low-pass frame corresponding to the high-pass frame, and a means for inversely updating the first low-pass frame using the restored high-pass frame.
• FlG. 1 is a view illustrating a conventional MCTF process
• FlG. 2 is a view illustrating in detail a prediction step and an update step shown in
• FIG. 1 A first figure.
• FIG. 3 is a view illustrating an MCTF process according to a first exemplary embodiment of the present invention
• FlG. 4 is a view illustrating a re-estimation process according to the first exemplary embodiment of the present invention
• FIG. 5 is a view illustrating an inverse MCTF process according to the first exemplary embodiment of the present invention
• FlG. 6 is a view illustrating a re-estimation process according to a second exemplary embodiment of the present invention.
• FIG. 7 is a view illustrating an inverse MCTF process according to the second exemplary embodiment of the present invention.
• FlG. 8 is a view illustrating an inverse MCTF process according to a third exemplary embodiment of the present invention
• FlG. 9 is a block view illustrating the structure of a video encoder according to one exemplary embodiment of the present invention.
• FlG. 10 is a block view illustrating the structure of a video decoder according to one exemplary embodiment of the present invention.
• FlG. 11 is a block view illustrating the structure of a system for realizing the operation of the video encoder shown in FlG. 9 or the video decoder shown in FlG. 10.
• the present invention provides a method of reducing the mismatch in the prediction step by re-estimating the H frame during the coding/decoding processes after the MCTF process (hereinafter, this process will be referred to as a 'frame re- estimation process').
• this process will be referred to as a 'frame re- estimation process'.
• the present invention will be described with reference to exemplary embodiments, in which each embodiment may include MCTF, re- estimation, and inverse MCTF processes.
• the MCTF and re-estimation processes are performed at the side of the video encoder and the inverse MCTF process is performed at the side of the video decoder.
• FlG. 3 is a view illustrating a 5/3 MCTF process according to a first exemplary embodiment of the present invention.
• the first exemplary embodiment of the present invention may implement the conventional MCTF scheme.
• the MCTF process is performed through a lifting scheme including a prediction step and an update step.
• L-position frames low-pass filtering
• H-position frames high-pass frames to be subjected to high-pass filtering
• the prediction step is applied to the H-position frames by using the adjacent frames, thereby obtaining the H frame.
• the update step is applied to the L-position frames by using the H frame obtained through the prediction step, thereby obtaining the L frame.
• subscripts represent temporal levels
• characters positioned in parentheses denote indexes allocated to the H frame and L frame in a specific temporal level.
• indexes allocated to the H frame and L frame in a specific temporal level For instance, referring to FlG. 3, four L frames L (1), L (2), L (3) and L (4) may exist in the temporal level 0, and two H frames H (1) and H (2) and two L frames L (1) and L (2) may exist in the next temporal level 1.
• the four L frames L (1), L (2), L (3) and L (4) correspond to the H and L frames H (1), L (1), H (2) and L (2), respectively.
• Equation 1 L () denotes an L frame obtained in the temporal level t.
• L () L ()
• Equation 2 P and U in Equation 1 can be expressed as Equation 2.
• Equation 3 P and U in Equation 1 can be represented as Equation 3.
• the prediction step and the update step may be repeated until one L frame finally remains.
• one L frame L (I) and three H frames H (I), H 1 (2) and H 2 (I) may be obtained.
• FIG. 4 is a view illustrating a re-estimation process according to the first exemplary embodiment of the present invention.
• the final L frame L (1) is coded and then decoded.
• the coding process may include a transform process and a quantization process
• the decoding process may include a de-quantization process and an inverse transform process.
• the coding and decoding processes will be referred to as a 'restoration process'.
• a finally restored L frame is represented as L '(1).
• the frame denoted with a prime mark ' ' ' refers to the frame which has undergone the restoration process.
• the frame L (I) obtained through the MCTF process is necessary. It is also possible to use the original L (2) instead of the frame L (1).
• the high-pass frame H (1) for the frame L (1) is re-estimated by using the reference frame L '(1).
• the reference frame may further include a frame of the previous GOP.
• a previously restored frame of the previous GOP can be used in the re-estimation process for the current GOP. If the index in the parenthesis of the H frame or the L frame has a negative magnitude, it refers to the frame of the previous GOP.
• the re-estimation frame is denoted with a reference character R (1).
• a calculation process for the re-estimation may be the same as the calculation process for the prediction step in the MCTF process, except that the reference frame used for the re- estimation is restored.
• the general re-estimation R (k) including the re- estimation frame R (1) can be expressed as Equation 4.
• the re-estimation frame R (1) is coded and then decoded, thereby obtaining the frame R '(1).
• the frame L '(1) is inversely updated by using the frame R '(1).
• a frame L '(2) is obtained.
• the inverse update step is performed in reverse order to the order of the update step in the MCTF process.
• the inverse update step can be expressed as Equation 5 by transforming Equation 1.
• the frame R (2) can be obtained by re-estimating the high-pass frame for the frame L (3) by using the obtained frames L '(1) and L '(2).
• the frame R (1) can be obtained by re-estimating the high-pass frame for the frame L (1) by using the frames L '(1) and L '(0), in which the L '(0) (not shown) is the frame of the previous GOP.
• FIG. 4 illustrates the GOP including four frames, if the GOP includes more than four frames, the above steps must be repeated corresponding to the number of frames.
• a video encoder quantizes the re-estimate frames R (1), R (2) and R (1) and the final low-pass frame L (I) and transmits them to a video decoder. Accordingly, the video decoder de-quantizes the re-estimate frames R (1), R (2) and R (I) and the final low-pass frame L (I), and then performs the inverse MCTF process, thereby restoring low-pass frames in the temporal level 0.
• the inverse MCTF process performed at the side of the video decoder will be described with reference to FIG. 5.
• the inverse MCTF process according to the first exemplary embodiment of the present invention is substantially identical to the conventional inverse MCTF process, except that the re-estimation frames are used instead of the high-pass frames.
• the final low-pass frame L '(1) is inversely updated by using the restored re- estimation frame R '(1) (inverse update step 1).
• the frame L '(2) is obtained.
• the re-estimation frame R '(1) is inversely predicted by using the reference frames L '(2) and L '(0), in which the reference frame L '(2) is obtained through the inverse update step and the reference frame L '(0) (not shown) is the frame of the previous GOP, thereby restoring the low-pass frame L '(1) (inverse prediction step 1).
• the frame re-estimation scheme is employed in order to apply the closed-loop structure to the MCTF technique including the prediction step and the update step.
• the open-loop type MCTF can be changed into the closed-loop type MCTF, so that the mismatch between the encoder and the decoder can be reduced.
• the re-estimation process in the encoder and the inverse MCTF process in the decoder may sequentially perform the inverse update step and the inverse prediction step.
• the mismatch between the encoder and the decoder may still exist because the update step designed for the open-loop codec is used together with the closed-loop prediction step.
• the reference frame used for inversely predicting the frame L '(1) from the re-estimation frame R '(1) is not the frame L '(1), but the frame L '(2) which is inversely updated from the frame L '(1).
• the same situation is represented in the inverse MCTF process shown in FIG. 5. That is, referring to FIG. 5, the frame L '(2), which is inversely updated from the frame L '(1), is used for inversely predicting the frame L '(1) from the re-estimation frame R '(1).
• the reference frame L '(1) is used for predicting the re-estimation frame R (I) from the low-pass frame L (I) and the reference frame L '(2) is used for restoring the low-pass frame L '(1) from the re-estimation frame R '(1).
• the MCTF scheme according to the first exemplary embodiment of the present invention can reduce the drift error because it has the closed-loop structure, since the update step is performed after the prediction step in the MCTF process and the prediction step is performed after the update step in the inverse MCTF process, the mismatch may still exist between the encoder and the decoder.
• a second exemplary embodiment of the present invention provides a method of solving the mismatch problem occurring in the first exemplary embodiment of the present invention.
• the conventional MCTF process shown in FIG. 3 is performed, thereby obtaining at least one high-pass frame H (1), H (2) or H (1) and a final low-pass frame L (1).
• the final low-pass frame L (1) is coded and then decoded.
• the high-pass frame H (1) for the low-pass frame L (1) is re-estimated by using the reference frame L '(1).
• the reference frame may further include a frame of the previous GOP.
• a previously restored frame of the previous GOP can be used in the re-estimation process for the current GOP.
• the re-estimation frame is denoted with a reference character R 2 (1). Such a re-estimation step may be performed while satisfying Equation 4.
• the re-estimation frame R (1) is coded and then decoded, thereby obtaining the frame R '(1).
• the re-estimation frame R '(1) is inversely estimated by using the reference frame L '(1), thereby obtaining the low-pass frame L is expressed in Equation 6.
• the frame L '(1) is inversely updated by using the frame R '(1), thereby obtaining the low-pass frame L '(2).
• Such an inverse update step is expressed in Equation 5.
• the step for obtaining the low-pass frame L '(1) is performed separately from the step for obtaining the low-pass frame L '(2). That is, the result of one step may not be utilized in the other step. Accordingly, it is also possible to reverse the sequence of the above two steps. To this end, the final low-pass frame L '(1) must be stored in a buffer before it is updated.
• the second exemplary embodiment of the present invention is different from the first exemplary embodiment of the present invention in that the reference frame, which is used for obtaining the low-pass frame by inversely predicting the re-estimation frame, has not been subject to the inverse update step.
• the frame R (2) can be obtained by re-estimating the high-pass frame for the frame L (3) by using the obtained frames L '(1) and L '(2).
• the frame R (1) can be obtained by re-estimating the high-pass frame for the frame L (1) by using the frames L '(1) and L '(0), in which the L '(0) (not shown) is the frame of the previous GOP. If the GOP includes many frames, the above steps must be repeated corresponding to the number of frames.
• the inverse MCTF process which corresponds to the MCTF process and the re- estimation process performed at the video encoder side, is carried out at the video decoder side.
• the inverse MCTF process according to the second exemplary embodiment of the present invention uses the reference frame, which is not inversely updated, in order to create the low-pass frame by inversely predicting the re-estimation frame.
• the re-estimation frame R '(1) is inversely predicted by using the reference frames L '(2) and L '(0), in which the reference frame L '(2) is the final low-pass frame and the reference frame L '(0) (not shown) is the frame of the previous GOP, thereby restoring the low-pass frame L '(1) (inverse predication step 1). Thereafter, the final low-pass frame L '(2) is inversely updated by using the re-esti mation frame R '(1) (inverse update step 1). As a result, the low-pass frame L '(2) is obtained.
• the step for obtaining the low-pass frame L '(1) is performed separately from the step for obtaining the low-pass frame L '(2). That is, the result of one step may not be utilized in the other step. Accordingly, it is also possible to reverse the sequence of the above two steps.
• the update step is useful in a structure supporting the temporal scalability, the number of operations may significantly increase because it requires an additional motion compensation process.
• the first and second exemplary embodiments of the present invention employ the closed-loop prediction, so that all high-pass frames and high-pass residuals can be re-estimated without causing the mismatch regardless of the update step. Accordingly, the performance may not be degraded even if the inverse update steps for the low-pass frames located in the temporal levels, in which the high-pass frames exist, are omitted.
• MCTF process performed at the encoder side may be achieved by performing the update step with respect to all low-pass frames similarly to the conventional MCTF process.
• the re-estimation process and the inverse MCTF process at the decoder side are performed while omitting the update steps for the low-pass frames located in the temporal levels, in which the high-pass frames exist, thereby significantly reducing the number of operations.
• the inverse update steps must be performed corresponding to the number of high-pass frames located in one temporal level.
• the inverse update steps must be performed with respect to all high-pass frames, that is, when the GOP has N frames, the inverse update steps may be performed N-I times.
• Equation 7 the reduced number (C) of operations for inverse update steps according to the third exemplary embodiment of the present invention.
• FlG. 8 is a view illustrating the inverse MCTF process according to the third exemplary embodiment of the present invention.
• the low-pass frames L '(1) and L '(2) located in the temporal level having no high-pass frames R '(1), R '(1) and R '(2) are inversely updated.
• the low-pass frames located in other temporal levels are not inversely updated. Accordingly, the low-pass frame L '(1) becomes the low-pass frame L '(2) at the temporal level 0 without being inversely updated.
• FlG. 9 is a block view illustrating the structure of a video encoder 100 according to one exemplary embodiment of the present invention.
• the video encoder 100 includes an MCTF unit 110, a re-estimation unit 199, a transform unit 120, a quantization unit 130, a de-quantization (inverse quantization) unit 150, an inverse transform unit 160, and an entropy coding unit 140.
• Input frames are inputted into an L frame buffer 117.
• the input frames are a part of the L frames (low-pass frames).
• the L frames stored in the L frame buffer 117 are provided to a dividing unit 111.
• the dividing unit 111 divides the L frames into Imposition frames (low-pass frames) and H-position frames (high-pass frames).
• the high-pass frames are located in odd positions (2i-l) and the low-pass frames are located in even positions (2i).
• T is an integer index representing the frame number.
• the H-position frames are transformed into H frames through the prediction step, and the L-position frames are transformed into low-pass frames adaptable for the next temporal level through the update step.
• the H-position frames are inputted into a motion estimation unit 115 and a difference unit 118.
• the motion estimation unit 115 performs motion estimation with respect to the H- position frame (hereinafter, referred to as a current frame) based on peripheral frames (frames located in different temporal position at the same temporal level), thereby obtaining a motion vector (MV).
• the peripheral frames are referred to as 'reference frames'.
• a block matching algorithm is extensively used for the motion estimation. That is, a predetermined block is moved within a specific search area of the reference frame in a pixel unit or a sub-pixel unit (e.g., 1/4 pixel) and the displacement of the block corresponding to the lowest error is estimated as a motion vector.
• a fixing vector can be used for the motion estimation
• HVSBM hierarchical variable size block matching
• the motion vector (MV) obtained by the motion estimation unit 115 is transferred to a motion compensation unit 112. Then, the motion compensation unit 112 performs the motion compensation with respect to the reference frames by using the motion vector (MV), thereby obtaining the prediction frame for the current frame.
• the prediction frame is expressed as 'P' shown in Equation 1.
• the difference unit 118 calculates the difference between the current frame and the prediction frame so as to create the high-pass frames (H frames).
• the high-pass frames are temporarily stored in the H frame buffer 117.
• the update unit 116 updates the L-position frames by using the obtained high-pass frames, thereby obtaining low-pass frames.
• a predetermined L-position frame may be updated by using two high- pass frames, which are temporally adjacent to the L-position frame.
• the update process may be performed in the single direction.
• the update process can be expressed as the second equation of Equation 1.
• the low-pass frames obtained by means of the update unit 116 are temporarily stored in the L frame buffer 118.
• the L frame buffer 118 transfers the low-pass frames to the dividing unit 111 so as to perform the prediction and update steps in the next temporal level.
• the final low-pass frame (L) is transferred to the transform unit 120.
• the transform unit 120 performs spatial transform with respect to the final low- pass frame (L ) and generates a transform coefficient.
• the spatial transform may include DCT (discrete cosine transform) or wavelet transform.
• DCT discrete cosine transform
• wavelet transform the transform coefficient is a DCT coefficient.
• the transform coefficient is a wavelet coefficient.
• a quantization unit 130 is provided to quantize the transform coefficient.
• 'quantization' means a procedure for representing the transform coefficient, which is expressed as a real number, in the form of a discrete value.
• the quantization unit 130 performs the quantization procedure by dividing the real-number transform coefficient through predetermined quantization steps and rounding off the result value to an integer value, which is called 'scalar quantization'.
• the quantization steps are provided in a predetermined quantization table.
• the quantization result obtained through the quantization procedure of the quantization unit 130, that is, the quantization coefficient of the low-pass frame L is sent to the entropy coding unit 140 and the de-quantization unit 150.
• the de-quantization unit 150 de-quantizes the quantization coefficient of the low- pass frame L .
• the term 'de-quantization' means a procedure for restoring a value matching with an index obtained through the quantization procedure by using the predetermined quantization table, which is used for the quantization procedure.
• the inverse transform unit 160 receives the de-quantized result and performs an inverse transform with respect to the de-quantized result. Such an inverse transform process may proceed inversely to the transform process through the inverse DCT transform or the inverse wavelet transform. As a result, a final low-pass frame L ' is restored and the restored final low-pass frame L ' is sent to an inverse update unit 190.
• the re- estimation unit 199 re-estimates the high-pass frames using the restored final low-pass frame L '. Examples of the re-estimation processes are shown in FIGS. 4 and 6, respectively.
• the re-estimation unit 199 includes the inverse update unit 190, a frame re- estimation unit 180 and an inverse prediction unit 170.
• the frame re-estimation unit 180 re-estimates the high-pass frames located in the temporal level identical to that of the restored final low-pass frame L ' by using the restored final low-pass frame L ' as a reference frame.
• the re-estimation step is expressed in Equation 4.
• the re-estimation high-pass frame R can be decoded through the transform unit
• the inverse prediction unit 170 inversely predicts the decoded high-pass frame by using the restored final low-pass frame L ' as a reference frame, thereby restoring a low-pass frame corresponding to the decoded high-pass frame.
• the inverse prediction step is expressed in Equation 6.
• the restored low-pass frame can be sent back to the frame re-estimation unit 180.
• the inverse prediction unit 170 may perform the inverse prediction in the next temporal level (low-class temporal level) by using a predetermined reference frame.
• the inverse update unit 190 inversely updates the restored final low-pass frame L ' by using the decoded high-pass frame.
• the inverse update step is expressed in Equation 5.
• the inversely updated low-pass frame is sent back to the frame re-estimation unit 180.
• the inverse update unit 190 may perform the inverse update process in the next temporal level (low-class temporal level) by using the decoded high-pass frame provided from the inverse transform unit 160.
• the frame re-estimation unit 180 can again perform the re-estimation process in the next temporal level by using the low-pass frames provided from the inverse prediction unit 170 and the inverse update unit 190 and the predetermined low-pass frame stored in the L frame buffer.
• MV calculated by the motion estimation unit 115 during the re-estimation step, the inverse prediction step and the inverse update step.
• the above operation of the re-estimation unit 199 may be repeated until the re- estimation has been completed with respect to all high-pass frames.
• the inverse update unit 190 further performs a step of determining whether the inputted low-pass frame is located in a position having the high-pass frames. If the inputted low-pass frame is located in the position having the high-pass frames, the inverse update step for the corresponding low-pass frame may be omitted. Otherwise, the inverse update step is performed with respect to the corresponding low-pass frame.
• the high-pass frames R which have been re-estimated in the frame re-estimation unit 180, are sequentially subject to the transform and quantization processes by means of the transform unit 120 and the quantization unit 130, respectively.
• the above processes may not be applied to the re-estimated frame, such as the frame R ', which has already been subjected to the above processes.
• the entropy coding unit 140 receives the quantization coefficient of the final low- pass frame L obtained by the quantization unit 130 and the quantization coefficient of the re-estimation high-pass frames R and codes the quantization coefficients through a lossless coding scheme, thereby obtaining bit streams.
• a lossless coding scheme includes Huffman coding, arithmetic coding, variable length coding, etc.
• FlG. 10 is a block view illustrating the structure of a video decoder 200 according to one exemplary embodiment of the present invention.
• An entropy decoding unit 210 performs a lossless decoding process, thereby extracting texture data and motion vector data for each frame from the bit streams inputted thereto.
• the extracted texture data are sent to a de-quantization unit 220 and the extracted motion vector data are sent to an inverse update unit 240 and an inverse prediction unit 250.
• the de-quantization unit 220 is provided to de-quantize the texture data outputted from the entropy decoding unit 210.
• the 'de-quantization' means a procedure for restoring a value matching with an index obtained through the quantization procedure by using the quantization table, which is used for the quantization procedure.
• An inverse transform unit 230 performs inverse transform with respect to the de- quantized result. Such an inverse transform process may proceed inversely to the transform process performed in the transform unit 120 of the video encoder 100.
• the inverse transform includes inverse DCT transform or inverse wavelet transform. As a result, the final low-pass frame and the re-estimation high-pass frame are restored.
• the restored final low-pass frame L ' is sent to an inverse prediction unit 250 and the restored re-estimation high-pass frame R' is sent to both the inverse update unit 240 and the inverse prediction unit 250.
• An inverse MCTF unit 245 may repeat the inverse prediction step and the inverse update step by using the inverse prediction unit 250 and the inverse update unit 240, thereby obtaining a finally restored low-pass frame L '. Such a repetition of the prediction and update steps may continue until the frame located in the temporal level 0, that is, the input frame of the encoder 100 can be restored.
• the inverse prediction unit 250 inversely predicts the re-estimation high-pass frame R' by using the final low-pass frame L ' as a reference frame, thereby restoring the low-pass frame corresponding to the high-pass frame R'.
• the inverse prediction unit 250 performs the motion compensation with respect to peripheral low- pass frames by using the motion vector (MV) provided from the entropy decoding unit 210, thereby obtaining the prediction frame for the current low-pass frame.
• the inverse prediction unit 250 adds the re-estimation high-pass frame R' to the prediction frame.
• Equation 6 Such an inverse prediction step is expressed in Equation 6.
• the low-pass frame obtained by the inverse prediction unit 250 is sent to the inverse update unit 240.
• the inverse update unit 240 inversely updates the low-pass frame by using the high-pass frame R' located in the temporal level identical to that of the low-pass frame.
• the motion compensation is performed with respect to the high-pass frame R' by using a motion vector, which is obtained by changing the sign of the motion vector provided from the entropy coding unit 210.
• the inverse update unit 240 may repeat the inverse update step by using the low-pass frame provided from the inverse prediction unit 250.
• the inverse update unit 240 outputs the restored low-pass frame
• the inverse update step and the inverse prediction step of the first exemplary embodiment may be performed inversely to those of the second exemplary embodiment. That is, according to the first exemplary embodiment of the present invention, the inverse prediction step is performed after the inverse update step. Therefore, the video decoding process according to the first exemplary embodiment of the present invention is substantially identical to the conventional inverse MCTF process, except that data for the inputted high-pass frame are related to the re- estimation high-pass frame.
• the inverse update unit 240 further performs a step of determining whether the inputted low-pass frame is located in a position having the high-pass frames. If the inputted low-pass frame is located in the position having the high-pass frames, the inverse update step for the corresponding low-pass frame may be omitted. Otherwise, the inverse update step is performed with respect to the corresponding low-pass frame.
• FIG. 11 is a block view illustrating the structure of a system for realizing the operation of the video encoder 100 or the video decoder 200.
• the system may include a TV, a set-top box, a desktop computer, a laptop computer, palmtop computer, a PDA (personal digital assistant), a video or an image storage device (e.g., a VCR (video cassette recorder) or a DVR (digital video recorder)).
• the system may include a combination of the above devices or a device provided as a part of other equipment.
• the system has at least one video source 910, at least one input/output unit 920, a processor 940, a memory 950, and a display unit 930.
• the video source 910 may include a TV receiver, a VCR or a video storage unit.
• the video source 910 may be at least one network connection for receiving a video signal from a server through the Internet, a WAN (wide area network), a LAN (local area network), a terrestrial broadcast system, a cable network, a satellite communication network, a wireless network or a telephone network.
• the video source 910 may include a combination of the above networks or a network provided as a part of other network.
• the input/output unit 920, the processor 940 and the memory 950 are communicated with each other through a communication medium 960.
• the communication medium 960 includes a communication bus, a communication network, or at least one internal connection circuit.
• Video data inputted from the video source 910 can be processed by means of the processor 940 according to at least one software program stored in the memory 950, or can be executed by means of the processor 940 in order to generate an output video signal transmitted to the display unit 930.
• the software program stored in the memory 950 may include a scalable video codec performing the method of exemplary embodiments of the present invention.
• the encoder or the scalable video codec may be stored in the memory 950 or a storage medium, such as a CD-ROM or a floppy disc. It is also possible to download the encoder or the scalable video codec from a predetermined server through various networks.
• the encoder or the scalable video codec can be replaced with a hardware circuit by means of the software program or can be replaced with a combination of software and hardware circuits.
• the drift error between the encoder and the decoder can be effectively reduced without deteriorating advantages of the prediction and update steps of the conventional MCTF, so that the data-compression efficiency can be significantly improved.
• the closed- loop prediction step can be applied to fast moving images for which the conventional MCTF process could not be effectively implemented due to the large residual energy caused by the fast moving images, thereby improving the performance.
• the update step can be applied to the slow moving images without causing the mismatch, thereby improving the performance.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (4)

Publications (2)

Family

ID=37809067

Family Applications (1)

Country Status (2)

Families Citing this family (3)

Family Cites Families (6)

• 2006
  • 2006-07-18 WO PCT/KR2006/002806 patent/WO2007027012A1/en active Application Filing
  • 2006-07-18 EP EP06823594A patent/EP1905238A4/de not_active Withdrawn

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events