Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
  • H04N19/36—Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
  • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Definitions

• Apparatuses and methods consistent with the present invention relate to video coding, and more particularly, to effectively coding multiple layers using interlayer information in a multi-layered video codec.
• Multimedia data requires a large capacity of storage media and a wide bandwidth for transmission since the amount of multimedia data is usually large.
• a 24-bit true color image having a resolution of 640*480 needs a capacity of 640*480*24 bits, i.e., data of about 7.37 Mbits, per frame.
• a bandwidth of 221 Mbits/sec is required.
• a 90-minute movie based on such an image is stored, a storage space of about 1200 Gbits is required.
• a compression coding method is a requisite for transmitting multimedia data including text, video, and audio.
• a basic principle of data compression is removing data redundancy.
• Data can be compressed by removing spatial redundancy in which the same color or object is repeated in an image, temporal redundancy in which there is little change between adjacent frames in a moving image or the same sound is repeated in audio, or mental visual redundancy taking into account human eyesight and limited perception of high frequencies.
• Data compression can be classified into lossy/lossless compression according to whether source data is lost, intraframe/interframe compression according to whether individual frames are compressed independently, and symmetric/ asymmetric compression according to whether the time required for compression is the same as the time required for recovery.
• Data compression is defined as real-time compression if the compression/recovery time delay does not exceed 50 ms and is defined as scalable compression when frames have different resolutions.
• lossless compression is usually used.
• lossy compression is usually used for multimedia data.
• intraframe compression is usually used to remove spatial redundancy
• interframe compression is usually used to remove temporal redundancy.
• Transmission media for transmitting multimedia information differ in performance according to the types of media transmitted.
• the transmission media currently in use have a variety of transfer rates, ranging for example, from a very-high speed communication network capable of transmitting the data at a transfer rate of tens of Mbits per second to a mobile communication network having a transfer rate of 384 Kbps.
• Previous video coding techniques such as MPEG-I, MPEG-2, H.263 or H.264 remove redundancy based on a motion compensated prediction coding technique. Specifically, temporal redundancy is removed by motion compensation, while spatial redundancy is removed by transform coding.
• Scalability indicates the ability to partially decode a single compressed bitstream, that is, the ability to perform a variety of types of video reproduction.
• Scalability includes spatial scalability indicating a video resolution, signal-to noise ratio (SNR) scalability indicating a video quality level, temporal scalability indicating a frame rate, and a combination thereof.
• SNR signal-to noise ratio
• H.264 SE' is being performed at present by a joint video team (JVT) of the MPEG (Motion Picture Experts Group) and ITU (International Telecommunication Union).
• JVT Joint video team
• An advantageous feature of H.264 SE lies in that it exploits the relevancy among layers in order to code a plurality of layers while employing an H.264 coding technique. While the plurality of layers are different from one another in view of resolution, frame rate, SNR, or the like, they basically have a substantial similarity in that they are generated from the same video source. In this regard, a variety of efficient techniques that utilize information about lower layers in coding upper layer data are proposed.
• FIG. 1 is a diagram for explaining weighted prediction proposed in conventional
• the weighted prediction allows a motion-compensated reference picture to be appropriately scaled instead of being averaged in order to improve prediction efficiency.
• a motion block 11 (a 'macroblock or 'subblock' as the basic unit for calculating a motion vector) in a current picture 10 corresponds to a predetermined image 21 in a left reference picture 20 pointed by a forward motion vector 22 while corresponding to a predetermined image 31 in a right reference image 30 pointed by a backward motion vector 32.
• An encoder reduces the number of bits required to represent the motion block 11 by subtracting a predicted image obtained from the images 21 and 31 from the motion block 11.
• a conventional encoder not using weighted prediction calculates a predicted image by simply averaging the images 21 and 31.
• the motion block 11 is not usually identical to an average of the left and right images 21 and 31, it is difficult to obtain an accurate predicted image.
• a method for determining a predicted image using a weighted sum is proposed in H.264.
• weighting factors ⁇ and ⁇ are determined for each slice and a sum of products of multiplying the weighting factors ⁇ and ⁇ by the images 21 and 31 are used as a predicted image.
• the slice may consist of a plurality of macroblocks and be identical to a picture. A plurality of slices may make up a picture.
• the proposed method can obtain a predicted image with a very small difference from the motion block 11 by adjusting the weighting factors.
• the method can also improve coding efficiency by subtracting the predicted image from the motion block 11.
• Illustrative, non-limiting embodiments of the present invention overcome the above disadvantages and other disadvantages not described above. Also, the present invention is not required to overcome the disadvantages described above, and an illustrative, non-limiting embodiment of the present invention may not overcome any of the problems described above. Apparatuses and methods consistent with present invention estimate a weighting factor that will be applied to a picture in a higher layer using information from a picture in a lower layer and perform weighted prediction on the picture in the higher layer using the estimated weighting factor.
• a method for encoding video by performing weighted prediction on a current picture in a high layer using information on a picture in a low layer including reading the information on the low layer picture, calculating weighting factors using the information on the low layer picture, calculating a weighted sum of reference pictures to the current picture using the weighting factors and generating a predicted picture for the current picture, and encoding a difference between the current picture and the predicted picture.
• a method for decoding video by performing weighted prediction on a current picture in a high layer using information on a picture in a low layer including extracting texture data and motion data from an input bitstream, reconstructing information about the low layer picture from the texture data, calculating weighting factors using the information on the low layer picture, calculating a weighted sum of reference pictures to the current picture using the weighting factors and generating a predicted picture for the current picture, and reconstructing a residual signal for the current picture from the texture data and adding the reconstructed residual signal to the predicted picture.
• a video encoder for performing weighted prediction on a current picture in a high layer using information on a picture in a low layer, the video encoder including an element for reading the information on the low layer picture, an element for calculating weighting factors using the information on the low layer picture, an element for calculating a weighted sum of reference pictures to the current picture using the weighting factors and generating a predicted picture for the current picture, and an element for encoding a difference between the current picture and the predicted picture.
• a video decoder for performing weighted prediction on a current picture in a high layer using information on a picture in a low layer, the method including an element for extracting texture data and motion data from an input bitstream, an element for reconstructing information about the low layer picture from the texture data, an element for calculating weighting factors using the information on the low layer picture, an element for generating a predicted picture for the current picture by calculating a weighted sum of reference pictures to the current picture using the weighting factors, and an element for adding texture data of the current picture among the texture data to the predicted picture.
• FlG. 1 is a diagram for explaining conventional weighted prediction proposed in
• FlG. 2 is a flowchart illustrating a multi-layered weighted prediction method according to an embodiment of the present invention
• FlG. 3 is a flowchart illustrating sub-steps for step S50 illustrated in FlG. 2;
• FlG. 4 illustrates a multi-layer video structure in which a high layer has double the resolution of a lower layer but the same frame rate as the low layer;
• FlG. 5 illustrates a multi-layer video structure in which a high layer and a low layer have a Motion-compensated Temporal Filtering (MCTF) structure;
• MCTF Motion-compensated Temporal Filtering
• FlG. 6 illustrates a multi-layer video structure in which a high layer and a low layer have a Hierarchical B structure
• FlG. 7 illustrates a multi-layer video structure in which a high layer has a MCTF structure and a low layer has a Hierarchical B structure;
• FlG. 8 illustrates a multi-layer video structure in which a high layer and a low layer have the same frame rate and pictures have multiple reference schemes
• FlG. 9 illustrates an example in which an embodiment of the present invention is applied when a current picture is an asynchronized picture
• FlG. 10 is a schematic diagram for explaining a method for using a weighting factor calculated from a low layer for a high layer
• FlG. 11 is a block diagram of a video encoder according to an embodiment of the present invention.
• FlG. 12 is a block diagram of a video decoder according to an embodiment of the present invention.
• FlG. 13 is a schematic block diagram of a system in which a video encoder and/or a video decoder according to an exemplary embodiment of the present invention operate.
• Weighted prediction can be effective for fade-in or fade-out sequences achieved by a gradual increase or decrease in brightness of a picture.
• a weighting factor to be applied to a high layer is expected to be similar to a weighting factor for a low layer. That is, information from a picture in a low layer can be used to perform weighted prediction on a picture in a high layer.
• an encoder does not need to transmit weighting factors ⁇ and ⁇ needed for weighted prediction to a decoder because the information from the picture in the low layer is available at both encoder and decoder.
• the decoder can perform weighted prediction according to the same algorithm as used in the encoder.
• FlG. 2 is a flowchart illustrating a multi-layered weighted prediction method according to an embodiment of the present invention.
• a synchronized picture means a picture having a corresponding picture in a lower layer (a 'base picture') at the same position.
• An asynchronized picture means a picture having no corresponding picture in a lower layer at the same temporal position.
• the temporally same position can be determined as a Picture of Count (POC) as defined as the Joint Scalable Video Model (JSVM).
• POC Picture of Count
• JSVM Joint Scalable Video Model
• the encoder determines the current picture has the same reference scheme and reference distance as the base picture in step S30.
• the reference scheme may be forward reference, backward reference, bi-directional reference or other multiple reference schemes.
• the reference distance refers to a temporal distance between a picture being predicted and a reference picture. In the JSVM, the temporal distance can be represented as a difference between a POC for a picture being predicted and a POC for a reference picture
• the encoder applies weighted prediction to the current picture in step S50. Conversely, if the current picture does not have different reference schemes and reference distance from the base picture (NO in step S30), the encoder does not apply weighted prediction to the current picture in step S60.
• FlG. 3 is a flowchart illustrating sub-steps for the step S50 illustrated in FlG. 2.
• step S51 the encoder reads information on the picture in the low layer.
• the information on the low layer picture contains a base picture and a reference picture motion-compensated with respect to the base picture.
• step S52 weighting factors ⁇ and ⁇ are calculated using the information on the low layer picture.
• a method for calculating the weighting factors ⁇ and ⁇ from the given pictures in step S52 (a 'weighting factor calculation algorithm') will be described later in more detail.
• step S53 the encoder uses a motion vector to perform motion compensation on a reference picture to obtain the current picture.
• a motion vector from a high layer estimated by motion estimation is used. If a plurality of reference pictures are used, motion compensation should be performed on each of the plurality of reference pictures using an appropriate motion vector.
• step S54 the encoder multiplies weighting factors ⁇ and ⁇ by the motion- compensated reference pictures and adds the products to obtain a predicted picture (or predicted slice).
• the encoder calculates a difference between the current picture (or slice) and the predicted picture (or slice) in step S56 and encodes the difference in step S56.
• the decoder can calculate a weighting factor used in the encoder by performing weighted prediction in the same manner as the encoder.
• FIGS. 4 through 8 are diagrams illustrating various multi-layer video structures to which the present invention can be applied.
• FlG. 4 illustrates a structure in which a picture in a high layer (Layer 2) has double the resolution of, but the same frame rate as, a picture in a low layer (Layer 1) and both layers have a single temporal level.
• Reference symbols I, P, and B denote an I-picture (or slice), a P-picture (or slice), or a B-picture (or slice), respectively.
• reference pictures in Layer 1 have the same positions as their counterparts in Layer 2.
• a current picture in a high layer having a base picture with the same reference scheme and reference distance can be encoded or decoded using a weighting factor applied to the base picture.
• a non-adjacent picture may be used as a reference picture.
• FlG. 5 illustrates a multi-layer video structure in which a high layer (Layer 2) has double the frame rate of a low layer (Layer 1).
• the high layer has one more temporal level than the low layer.
• the techniques described here can be applied to a structure in which the high and low layers are decomposed into hierarchical temporal levels. That is, pictures 54 through 56 that satisfy the requirements described with reference to FlG. 2 among high-pass pictures in the high layer can effectively be encoded or decoded using weighting factors applied to their corresponding base pictures 57 through 59. On the other hand, weighted prediction is not applied to high-pass pictures 50 through 53 at the highest level (level 2) in the high layer with no corresponding base pictures.
• FlG. 6 illustrates a multi-layer video structure in which a high layer (Layer 2) and a low layer (Layer 1) have a hierarchical B structure defined in H.264. Like in FlG. 5, the high layer has double the frame rate of the low layer.
• each layer is decomposed into temporal levels in a different way than for a MCTF structure. That is, a frame rate increases in the order from high to low levels.
• the frame rate of the high layer is A
• the encoder transmits only pictures at level 2.
• the encoder may transmit only pictures at levels 2 and 1.
• the encoder may transmit pictures at all levels.
• a high-pass picture 61 has the same reference scheme and reference distance as a base picture 64 and a high-pass picture 62 has the same reference scheme and reference distance as a base picture 65.
• the high-pass pictures 61 and 62 can be subjected to weighted reference using weighting factors applied to the base pictures 64 and 65.
• weighted reference is not applied to a high-pass picture 63 because it does not have a corresponding base picture.
• FlG. 7 illustrates a multi-layer video structure in which a high layer (Layer 2) has a
• MCTF structure and a low layer has a Hierarchical B structure.
• weighted prediction is not applied to high-pass pictures at level 2 in the high layer because the high pass pictures at level 2 do not have any corresponding base pictures.
• high-pass pictures at level 1 or 0 have corresponding base pictures.
• a high-pass picture 72 has a base picture 75.
• weighted prediction can be applied without any problem if the two pictures 72 and 75 have the same reference scheme and reference distance. Because the two pictures 72 and 75 actually have the same reference scheme and reference distance, weighted prediction can be applied.
• weighted prediction can be performed on a picture 73 using a weighting factor assigned to a picture 74.
• FlG. 8 illustrates a multi-layer video structure in which a high layer (Layer 2) and a low layer (Layer 1) have a single temporal level and all high-pass pictures in the high layer have corresponding base pictures. However, all the high-pass pictures in the high layer do not have the same reference scheme and reference distance as their corresponding base pictures.
• a high-pass picture 81 has the same reference scheme (bi-directional reference scheme) and the same reference distance (1) as its corresponding base picture 85.
• a high-pass picture 82 has the same reference scheme (backward reference scheme) and the same reference distance (1) as its corresponding picture 86.
• high-pass pictures 83 and 84 do not use the same reference schemes as their corresponding base pictures 87 and 88. Thus, weighted prediction is not applied to the high-pass pictures 83 and 84.
• weighted prediction is applied when the current picture is a synchronized picture and the current picture has the same reference scheme and reference distance as the base picture, the present invention is not limited to the particular embodiments shown in FIGS. 2 through 8.
• a weighting factor can be calculated using a low layer picture at the closest temporal position to the current picture and its reference picture.
• a high layer Layer 2
• a low layer Layer 1
• pictures 91 and 93 in the high layer have corresponding base pictures and pictures in the high layer and low layer have different reference distances.
• weighting factors ⁇ and ⁇ for the pictures 91 and 93 can be calculated using a picture 92 in the low layer at the closest temporal position to the pictures 91 and 93 and its reference pictures 94 and 96.
• a 'weighting factor calculation algorithm' is applied to a low layer picture 92 and the reference pictures 94 and 96 to calculate the weighting factors ⁇ and ⁇ that can be used as weighting factors for the pictures 91 and 93.
• this method exhibits high coding performance compared to the case where half the weighting factors ⁇ and ⁇ are applied.
• a low layer picture at the same temporal position as or closest temporal position to a high layer picture is hereinafter generally referred to as a 'base picture'.
• FlG. 10 is a schematic diagram for explaining a method for using a weighting factor calculated from a low layer for a high layer.
• a unit block 41 means a region having a size ranging from a block size to a picture size.
• the unit block 41 may be a motion block, macroblock, slice, or picture.
• images 42 and 43 in a reference picture corresponding to the unit block 41 have the same size as the unit block 41.
• Motion vectors 44 and 45 from the unit block 41 mean motion vectors when the unit block 41 is a motion block.
• the motion vectors 44 and 45 mean a graphical representation of a plurality of motion vectors.
• the weighting factor calculation algorithm consists of: (a) calculation of a weighted sum of reference pictures to the base picture using predetermined coefficients ⁇ and ⁇ ; and (b) determination of values of the coefficients ⁇ and ⁇ that minimizes k 1 V k r k the square of a difference between the base picture and the weighted sum.
• ⁇ and ⁇ denote forward and backward coefficients, respectively.
• a weighted prediction value P(s ) for a pixel s within the unit block 41 can be defined by Equation (1).
• Equation (2) An error E that is the difference between the actual pixel s and the prediction value P(s ) can be defined by Equation (2):
• Equation (3) [63]
• Equation (4) is obtained.
• ⁇ is set to 0 so as not to consider the impact of ⁇ .
• Equation (5) A first weighting factor ⁇ that minimizes ⁇ E can be given by Equation (5):
• Equation (6) [69]
• the weighting factors ⁇ k and ⁇ can be used as weighting factors for a block 46 in a high layer corresponding to the unit block 41.
• the blocks 46 and 41 have the same size.
• the block 46 may have a larger size than the block 41.
• the high layer uses the weighting factors ⁇ and ⁇ assigned to the low layer to calculate a weighted sum of reference images 47 and 48 instead of calculating new weighting factors, thus reducing overhead.
• the process of using the weighting factors assigned to the low layer as weighting factors for the high layer can be performed in the same way both in the encoder and the decoder.
• AA wweeiigghhttiinngg factor ⁇ that minimizes the ⁇ E in the Equation (7) can be defined bbyy EEqquuaattiioonn ((88)):: [75]
• FIG. 11 is a block diagram of a video encoder 100 according to an embodiment of the present invention.
• an input current picture F is fed into a motion estimator 105, a subtracter 115, and a downsampler 170.
• the video encoder 100 operates in the following manner.
• the downsampler 170 temporally and spatially downsamples the current picture F.
• a motion estimator 205 performs motion estimation on the downsampled picture F using a neighboring reference picture to obtain a motion vector MV .
• An original image F may be used as the reference picture (open-loop coding) or a decoded image F Or ' may be used as the reference picture (closed-loop coding). It is hereinafter assumed that both the high layer and low layer in the video encoder 100 support closed-loop coding. For motion estimation, a block matching algorithm is widely used.
• a motion compensator 210 uses the motion vector MV to motion compensate for the reference picture F ' to obtain a motion-compensated picture mc(F '). When a plurality of reference pictures F Or ' are used, motion compensation may be performed on each of the plurality of reference pictures F '.
• the motion-compensated picture mc(F predicted image calculated using the motion-compensated picture mc(F Or ') are fed into a subtracter 215.
• the motion-compensated picture mc(F ') is input.
• a predicted picture may be calculated by averaging a plurality of motion-compensated pictures mc(F Or ') before being input to the subtractor 215. Because the present embodiment of the invention is not affected by whether weighted prediction is used for the low layer, FIG. 11 shows that no weighted prediction is applied to the low layer. Of course, weighted prediction may be applied to the low layer.
• the subtractor 215 subtracts the estimated picture (or each motion-compensated block) from the downsampled picture F and generates a difference signal R , which is then supplied to a transformer 220.
• the transformer 220 performs a spatial transform on the difference signal R and generates a transform coefficient R ⁇ .
• the spatial transform method may include a Discrete Cosine Transform (DCT), or wavelet transform. Specifically, DCT coefficients may be created in a case where DCT is employed, and wavelet coefficients may be created in a case where wavelet transform is employed.
• DCT Discrete Cosine Transform
• a quantizer 225 quantizes the transform coefficient. Quantization refers to a process of expressing the transform coefficient formed in an arbitrary real value by discrete values. For example, quantization is performed such that the transform coefficient is divided by a predetermined quantization step and the result is then rounded using an integer.
• the quantization result supplied from the quantizer 225 that is, the quantization coefficient R is supplied to an entropy coding unit 150 and an inverse quantizer 230.
• the inverse quantizer 230 inversely quantizes the quantization coefficient R Q .
• the inverse quantization is performed in a reverse order to that of the quantization performed by the transformer 220 to restore values matched to indices generated during quantization according to a predetermined quantization step used in the quantization. Specifically, inverse DCT transform, or inverse wavelet transform may be used.
• An inverse transformer 235 receives the inverse quantization result and performs an inverse transformation.
• the inverse transformation is performed in a reverse order to that of the transformation performed by the transformer 220.
• inverse DCT transformation, or inverse wavelet transformation may be used.
• An adder 240 adds the inverse quantization result and the predicted picture (or motion-compensated picture) used in the motion compensation step performed by the motion compensator 210 and generates a restored picture F '.
• a buffer 245 stores an addition result supplied from the adder 240. Thus, not only the currently restored picture F ' but also the previously stored reference picture F ' are stored in the buffer 245.
• the video encoder 100 operates in the following manner.
• the video encoder 100 in the higher layer includes a motion estimator 105 calculating a motion vector MV, a motion compensator 110 that motion- compensates for a reference picture F r ' using the motion vector MV, a subtractor 115 calculating a residual signal R between the current picture F and a predicted picture P, a transformer 120 applying transformation to the residual signal R to generate a transform coefficient R , a quantizer 125 quantizing the transform coefficient R to output a quantization coefficient R Q .
• the higher layer's video encoder further includes an inverse quantizer 130, an inverse transformer 135, an adder 140, and a buffer 145. Since functions and operations of these components are the same as those in the lower layer, their repetitive description will be omitted. The following description will focus on distinguishing features of the components.
• a weighting factor calculator 180 uses a base picture F in a low layer corresponding to a current picture in a high layer and motion-compensated reference picture me (F Or ') to calculate weighting factors ⁇ and ⁇ .
• the weighting factors ⁇ and ⁇ can be calculated by the above Equations (5) and (6).
• a normalized weighting factor can be calculated by the Equation (8). Of course, when a unidirectional reference scheme is used, only one weighting factor may be used.
• a weighted sum calculator 160 uses the calculated weighting factors ⁇ and ⁇ (or the normalized weighting factor) to calculate a weighted sum of the motion- compensated pictures me (F r ') provided by the motion compensator 110.
• the weighted sum may be represented by ⁇ *F + ⁇ *F .
• the weighted sum is represented by ⁇ *F .
• weighted prediction is performed per motion block, macroblock, slice, or picture. If the weighted prediction is performed in a smaller unit than a picture, a number of a pair of weighting factors ⁇ and ⁇ corresponding to the number of units in the picture should be calculated and different weighting factors will be applied for each unit to calculate a weighted sum.
• the weighted sum i.e., the predicted picture P, obtained by the weighted sum calculator 160 is fed into the subtracter 115.
• the subtractor 115 generates the residual signal R by subtracting the predicted picture P from the current picture F and provides the same to the transformer 120.
• the transformer 120 applies spatial transform to the residual signal R to create the transform coefficient R ⁇ .
• the quantizer 125 quantizes the transform coefficient R ⁇ .
• the entropy coding unit 150 performs lossless coding on the motion-estimated motion vectors MV and MV provided from the motion estimators 105 and 205 and the quantization coefficients R and R provided from the quantizers 125 and 225, respectively.
• the lossless coding methods may include, for example, Huffman coding, arithmetic coding, and any other suitable lossless coding method known to one of ordinary skill in the art.
• FIG. 12 is a block diagram of a video decoder 300 according to an embodiment of the present invention.
• An entropy decoding unit 310 performs lossless decoding on input bitstreams and extracts motion vectors MV and MV and texture data R Q and R Q for the respective o o layers.
• the lossless decoding is performed in a reverse order to that of lossless coding performed by an entropy coder part in each layer.
• the extracted texture data R of the lower layer is provided to an inverse quantizer 420 and the extracted motion vector MV of the lower layer is provided to a motion compensator 460.
• the extracted texture data R Q of the higher layer is provided to an inverse quantizer 320 and the extracted motion vector MV of the higher layer is provided to a motion compensator 360.
• the inverse quantizer 420 performs inverse quantization on the texture data R Q .
• the inverse quantization is a process of restoring values matched to indices generated during the quantization process using the same quantization table as in the quantization process.
• An inverse transformer 430 receives the inverse quantization result R ' and performs an inverse transformation.
• the inverse transformation is performed in a reverse order to that of the transformation process performed by a transformer. Specifically, inverse DCT transformation, or inverse wavelet transformation may be used.
• the restored residual signal R ' is provided to an adder 440.
• a motion compensator 460 motion-compensates a low layer's reference picture F ', which is previously restored and stored in a buffer 450, using the extracted motion vector MV , and generates a motion-compensated picture mc(F ').
• the video decoder 300 operates in the following manner.
• the video decoder 300 in the higher layer includes an inverse quantizer 320, an inverse transformer 330, a buffer 350, and a motion compensator 360. Since functions and operations of these components are the same as those in the lower layer, their repetitive description will be omitted. The following description will focus on distinguishing features of the components.
• a weighting factor calculator 380 uses a base picture F and a motion-compensated reference picture mc(F Or ') to calculate weighting factors ⁇ and ⁇ .
• the weighting factors ⁇ and ⁇ can be calculated by the Equations (5) and (6).
• a normalized weighting factor can be calculated by the Equation (8).
• a unidirectional reference scheme when a unidirectional reference scheme is used, only one weighting factor may be used.
• a weighted sum calculator 370 uses the calculated weighting factors ⁇ and ⁇ (or the normalized weighting factor) to calculate a weighted sum of the motion- compensated pictures mc(F r ') provided by the motion compensator 360.
• a weighted sum, i.e., a predicted picture P, obtained by the weighted sum calculator 370 is fed into an adder 340.
• weighted prediction is performed per motion block, macroblock, slice, or picture. If the weighted prediction is performed in a smaller unit than a picture, a number of a pair of weighting factors ⁇ and ⁇ corresponding to the number of units in the picture should be calculated and different weighting factors will be applied for each unit to calculate a weighted sum.
• the adder 340 adds the inverse transform result R' supplied from the inverse transformer 330 and the predicted picture and restores a current picture F'.
• a buffer 350 stores the restored current picture F'.
• FIG. 13 is a schematic block diagram of a system in which the video encoder 100 and the video decoder 300 according to an exemplary embodiment of the present invention operate.
• the system may be a television (TV), a set-top box, a desktop, laptop, or palmtop computer, a personal digital assistant (PDA), or a video or image storing apparatus (e.g., a video cassette recorder (VCR) or a digital video recorder (DVR)).
• the system may be a combination of the above-mentioned apparatuses or one of the apparatuses which includes a part of another apparatus among them.
• the system includes at least one video/image source 910, at least one input/ output unit 920, a processor 940, a memory 950, and a display unit 930.
• the video/image source 910 may be a TV receiver, a VCR, or other video/image storing apparatus.
• the video/image source 910 may indicate at least one network connection for receiving a video or an image from a server using Internet, a wide area network (WAN), a local area network (LAN), a terrestrial broadcast system, a cable network, a satellite communication network, a wireless network, a telephone network, or the like.
• the video/image source 910 may be a combination of the networks or one network including a part of another network among the networks.
• the input/output unit 920, the processor 940, and the memory 950 communicate with one another through a communication medium 960.
• the communication medium 960 may be a communication bus, a communication network, or at least one internal connection circuit.
• Input video/image data received from the video/image source 910 can be processed by the processor 940 using to at least one software program stored in the memory 950 and can be executed by the processor 940 to generate an output video/ image provided to the display unit 930.
• the software program stored in the memory 950 includes a scalable wavelet-based codec performing a method according to an embodiment of the present invention.
• the codec may be stored in the memory 950, may be read from a storage medium such as a compact disc-read only memory (CD-ROM) or a floppy disc, or may be downloaded from a predetermined server through a variety of networks.
• a video encoder and video decoder enable effective weighted prediction on a high layer picture by calculating weighting factors from a base picture without transmitting additional information to the decoder, thus providing for improved compression efficiency of video data.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (3)

Publications (2)

Family

ID=37308153

Family Applications (1)

Country Status (2)

Families Citing this family (7)

Citations (2)

Family Cites Families (3)

• 2006
  • 2006-04-20 EP EP06747383A patent/EP1878252A4/de not_active Withdrawn
  • 2006-04-20 WO PCT/KR2006/001472 patent/WO2006118384A1/en active Application Filing

Patent Citations (2)

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events