TWI293833B - Method and apparatus for mpeg-4 fgs performance enhancement - Google Patents

Description

1293833 玖, invention description: [Technical field of the invention] The present invention relates to a codec with fine granularity scalable (FGS), in particular to a codec with fine scalability The architecture, prediction mode, and bit allocation. [Prior Art] The application of multimedia has become more and more widespread in the world today, for example, listening to a CD player or accessing a webpage through the Internet. Through the Internet ► A common problem with multimedia applications is that uncompressed video data is too large to be easily stored and transmitted. Therefore, international organizations such as the ITU-T and ISO MPEG committees have developed several coding standards for data compression. With the establishment of these standards, the storage and transmission of video materials has become much simpler. Since the technology of the Internet has made great progress in the past few years, people can now read web pages, play games, or download files through the Internet. Among them, streaming video is an important network application. Through this application, people can access pre-encoded video clips (vide〇 clip) from the video server (vide〇 server). The biggest advantage of streaming video is that people can receive video from any location via the Internet connection. By streaming video, people can access video from asymmetric networks such as ADSL, Cable Modem, etc. For the provider of streaming 1293833 video, because the bandwidth of people receiving video is different, the bitstream of the video must be transmitted at various bitrates. There are several traditional methods for adjusting the bit stream bit rate. One method is to encode the video-programmed program into a bit stream of multiple rates during encoding, which can solve the problem of different bandwidths at the receiving end. However, in a group of multicast environments, hundreds or thousands of receivers will simultaneously access the same video program. As a result, the bit rate required by this method at the video stream provider side will be much higher. The sum of the bit rates of the bit stream. Another way is to encode the bit stream at the highest possible connection rate and then transcode it to a different rate. First, the transcoder first decodes the bit stream and then transcodes it into a suitable bit rate for each receiver. In this way, the contention video provider can use the transcoder to provide bit streams of various rates according to the respective receiving ends. ^ %:. MPEG-4 Draft Amendment 4 proposes and standardizes a new concept called FGS. The FGS compressed video includes a base layer and an enhancement layer. The base layer is generated using an MPEG-4 encoder at the lowest bit rate of all possible connections. The FGS uses the original and reconstructed discrete cosine transform (DCT) coefficients to generate a reinforced layer bit stream in bit-plane coding. The method produced is such that by subtracting the reconstructed coefficients from the original coefficients of the DCT, the error values introduced by the quantization process can be obtained. After 1293833, the FGS codec encodes these residual errors with bit-level coding, and the resulting bit layers are output from the most significant bit (MSB) to the least significant bit (LSB). The enhancement layer can therefore be truncated at any number of bits. If the receiving end has a bandwidth after receiving the base layer bit stream, the receiving end can continue to receive the enhanced layer bit stream, and the more the enhanced layer bit layer is received, the reconstructed video quality will be better. Since FGS can provide a wide range of rates from the base layer bit rate to the receiver end bandwidth limit, FGS is well suited for multi-streaming streaming video. As shown in Figure 1, all receivers (receivers 1, 2, 3) receive the FGS base layer at the lowest visual quality. Since the bandwidth of the receiving end 1 is insufficient, the receiving end 1 cannot receive the FGS enhancement layer, and the receiving ends 2 and 3 can receive the FGS enhancement layer bit layer as much as possible. Because FGS can provide a wide range of bit rates to respond to the bandwidth variation of the receiving end, it is more flexible in video streaming applications than other encoding technologies, and thus more and more widely used in streaming video applications. However, although the FGS encoder provides such a high flexibility of response bandwidth, its coding efficiency is worse at the same bit rate than other non-scalable encoders. There are two main factors in the poor efficiency of FGS coding. First, the motion-compensated predictive coding of the Fgs base layer uses only coarse prediction, but does not use coded residuals from the enhancement layer reconstruction (details of the image). Second, the FGS enhancement layer encoder has no motion compensated prediction loop. That is to say, each FGS enhancement layer frame 1293833 is intra-layer coded. Since the fGS base layer is encoded at the lowest bit rate with the lowest human visual quality, the FGS base layer's time-dependent prediction 'its coding gain is usually not as scalable as the non-scalable encoder. Figure 2 illustrates the encoding process for generating the FGS base layer and the enhancement layer bit stream. The base layer is encoded with an MPEG-4 non-scalable encoder at bit rate. The enhancement layer uses the original and reconstructed de-quantized coefficients as input, and the bit layer coding produces a reinforced layer beer stream. The coding procedure for the enhanced layer bit stream is: First, the original DC D number is subtracted from the dequantized DCT coefficients to obtain a quantization error. Next, after generating all DCT quantization errors for a picture, the enhancement layer encoder finds the maximum absolute value of these DCT quantization errors to determine the maximum number of bit levels for this picture. After determining the maximum number of bit layers in a picture, the FGS enhancement layer code: the code from the most significant bit layer (MSBplane) - one bit layer of one bit layer of the output enhancement layer data, up to the lowest effective element layer ( LSBplane) So far. Each bit of the meta layer is first converted to a symbol and then encoded in a variable length to produce a stream of output bits. The following example illustrates the above encoding process, taking the absolute quantization error of a DCT block as an example. 5,0,4, 1,2,0, ··· 〇, 最大值 The maximum quantization error of this DCT block is 5 ' and the number of bits required to represent 5 ( 101) is ^ 3. The absolute quantization error 1293383 is written as a binary system, and the 3 bit layers are as follows (MSB) (MSB-1) (LSB) 2, 〇, 1, 〇, 0, 0··· 〇 , 〇0,0,0,0,1,0 ··· 〇,〇Uo, 1,0,0 ··· 〇,〇 Figure 3 illustrates the FGS decoding process for enhanced layer picture reconstruction. The decoding process of the FGS base layer is the same as the decoding process of the -MPEG_4 non-scalable bit stream. Built into the KFGS bit stream, the decoder receives and converts the bit layer from the MSB bit layer to the LSB bit layer in a variable length manner. Because the decoder does not necessarily receive all blocks of a particular bit-specific layer, the decoder fills in the unreceived blocks of these bit layers and performs inverse discrete cosmetans formation (IDCT). To convert the received DCT coefficients into pixel values. These pixel values are then added to the image that has been decoded by the base layer to obtain the final improved video image.

Although FGS can support a wide range of bit rates to simplify adjustments to bandwidth variations, FGS has some drawbacks. Referring to FIG. 2, the input signal fed to the enhancement layer decoder is a quantization error of the prediction error of the input video reconstructed relative to its base layer. The base layer is at the lowest visual quality. Encoding is performed at the lowest bit rate. Therefore, it is usually impossible to accurately approximate the input video, so the quantization error will be quite large, resulting in low coding efficiency. The performance of single layer coding is better than FGS 1293833 at the same transmission bit rate because single layer coding is predicted with full quality video. This reduction in utility can range from 1.5 to 2.5 dB as described in the prior art. To overcome this problem, several related studies have been proposed to improve the visual quality of FGS coding, and these methods are briefly described below. A method called adaptive motion compensated PGS ' AMC-FGS, whose codec features two simplified scalable codecs: single prediction loop. MC-FGS and dual prediction The loop MC_FGS, both of which have different degrees of horoscope efficiency and error resilience. Dual Predictive Loop MC-FGS uses an additional MCP loop for the enhancement layer encoder only for B-frames. Since other pictures will not be referenced to the B_ picture for prediction during codec, if there is missing team picture data, there will be no errors, and the difference will spread. If a B-picture has drift error (drifling eiT〇r) This drift error will not spread to subsequent pictures. The single prediction loop MC-FGS uses fine quality prediction (fme prediction) for both P-pictures and pictures, so it has a relatively high coding efficiency compared with the double prediction loop MC_FGS. However, if the predicted enhancement layer data used in the base layer of the p_picture layer is lost due to insufficient bandwidth or transmission channel error and cannot be received by the decoder, its error toughness (err〇r rObustness) is significantly reduced. Reduced ground. AMC-FGS uses an adaptive prediction mode decision algorithm that can switch between two prediction techniques to achieve a better compromise between coding efficiency and error resistance (^ found 12 1293833 tradeoff) A new fgS architecture called progressive FGS (PFGS), whose enhancement layer can refer not only to the base layer, but also to the previous enhancement layer data. However, when the bandwidth is reduced, the same drift error can also disturb the round-out quality if the referenced bit layer is not guaranteed to be sent to the decoder. The other is called the method of error robust FGS (robustFGS, RFGS)/. This method adds a set of motion compensated prediction loops to the enhancement layer and adds leaky prediction to obtain a better compromise between coding performance and error tolerance. This additional motion-compensated prediction loop can improve the coding performance by referring to the high quality frame memory, and use leakage prediction to suppress the accompanying drift. The enhanced layer motion compensation prediction loop obtains a reconstructed picture using a leakage factor a (0 da □ 1) which is based on the predicted drift error. In addition, another parameter, the number of bit layers referenced, is also utilized in partial prediction. - By adjusting these two parameters, RFGS can provide flexibility for various coding methods. If the leakage factor (4) is set to 〇, then the RFGS is almost the same as the original FGS. If each reference picture is set to 1, the prediction mode of the RFGS is the same as the MC-FGS. SUMMARY OF THE INVENTION The present invention enhances the performance of the FGS codec, and its main purpose is to provide a new architecture of an FGS codec of 13 1293833. The new architecture has three prediction modes that can be appropriately selected. Another object of the present invention is to provide a method for appropriately selecting a prediction mode for each macroblock (macr〇bl〇ck) of an input signal. It is still another object of the present invention to provide a reinforced layer bit layer dicing method for an FGS codec. According to the present invention, both the encoder and the decoder of the FGS codec have a base layer and a reinforcement layer, the base layer including a coarse quality prediction loop and a base layer mode selector, and this adds money. The _ layer contains a fine quality prediction loop and an enhancement-layer mode selector. This base layer mode selector can be controlled to select whether the output of the base layer is a coarse quality prediction or a fine quality prediction. Similarly, the enhancement layer mode controller can also be controlled to select whether the output of the enhancement layer is a rough image quality prediction or a fine image quality prediction. ^ V . The FGS codec of the present invention provides three prediction modes: an all-fine prediction mode means that the base layer mode 'selector and the enhancement layer mode selector select a fine image quality prediction output; The all-coarse prediction mode means that the base layer mode selector and the enhancement layer mode selector both select a coarse image quality prediction output; the hybrid prediction mode refers to the base layer mode selector selection. The coarse image quality predicts the output, while the enhancement layer mode selector selects the fine image quality predictive output. 1293833 The encoder of the present invention can appropriately select the prediction mode for each macroblock of the input video signal. The present invention uses a two-pass encoding procedure. The first stage of the code collects the coding parameters of all macroblocks, including the prediction error values of the rough image quality prediction and the fine image quality prediction, and the best case and the worst case predicted error. The error mismatch error is caused by the decoder not receiving the enhancement layer data applied to the fine image quality prediction during the fine image quality prediction. Then, a coding gain is derived from the prediction error of the coarse image quality prediction and the fine image quality prediction. An estimated error matching error value is estimated from the error and error values of the best case and the worst case. Based on this, the coding benefit of each macroblock can be calculated. This coding benefit is defined as the coding gain divided by the estimated error matching error value. The mean (mean) and standard deviation (standard, .deviatiotf) of the coding benefits of all macroblocks in a picture are also calculated. Then, according to the coding benefit of each macroblock, these macroblocks are divided into three groups. The macroblocks of each group are encoded in the same prediction mode. If the coding efficiency of a macroblock is less than the difference between the coding efficiency average and the pre_determined multiple of the coding benefit standard deviation, the macroblock is coded in a full coarse quality prediction mode. To code. If the coding efficiency of a macroblock is greater than the sum of the coding efficiency average and a predetermined multiple of the coding efficiency standard deviation, the macroblock is encoded in a full fine quality prediction mode. Otherwise, 15 1293833 This macroblock is encoded in a mixed prediction mode. The present invention provides a new scale mother algorithm, which cuts the bit layer of the enhancement layer according to the three available bandwidths, namely: low bit rate, median rate, and high bit. rate. In the case of low bit rate, the enhancement layer layer of the Ι/P-picture (l/p_frame) will be cropped as long as the 'bit allocation' is only given to the i/p-昼, and the reinforcement layer data of the B-frame is completely cut. In the case of the median rate, when the bit allocation of the Ι/P-picture can guarantee the Ι/P-picture bit layer for fine 预测 quality prediction, after the complete send, the excess bit The meta is assigned to the B_ screen. In the case of high bit rates, the number of bits allocated is controlled by the size of the bit layer and varies with a particular bit rate. If there are no extra bits allocated to the j/ρ-picture, in order to avoid too much variation between the two adjacent pictures, the distribution of the bit allocation between the pictures must be balanced. The above and other objects and advantages of the present invention will be described in detail with reference to the accompanying drawings. [Embodiment] Figs. 4 and 5 illustrate block diagrams of a novel 3-mode FGS codec of the present invention. As shown in Fig. 4, the encoder architecture of the present invention includes a reinforcement layer and a base layer. The enhancement layer has a DCT unit 401, a bit-plane shift unit 402, a maximum value finder 403383, a one-bit variable length coder 404, and a fine 昼. Quality prediction loop. The fine image quality prediction loop includes a bit-plane divider 405, an IDCT unit 406, a fine picture memory 407, and a motion compensation provided with the switch SW1. Unit 408, this switch SW1 is used to select the prediction mode in the enhancement layer. The base layer has a DCT unit 411, a quantization unit 412, a variable length coder 413, and a coarse picture quality prediction loop. The coarse quality prediction loop includes an inverse quantization unit 414, an IDCT unit 415, a coarse frame memory 416, a motion estimation unit 417, and a A motion compensation unit 418 of the switch SW2 is provided, and the switch SW2 is used to select a prediction mode in the base layer. The decoder architecture of the present invention as shown in Figure 5 also includes a reinforcement layer and a base layer. The enhancement layer has a one-bit variable length decoder 501, a first IDCT unit 502, and a fine quality prediction loop. The fine image quality prediction loop includes a bit layer splitter 503, a second IDCT unit 504, a fine enamel picture memory 505, and a motion compensation unit 506 provided with a switch SW3 for selecting Strengthen the prediction mode in the layer. The base layer has a variable length masher 510, a dequantization unit 511, a second IDCT unit 512, and a coarse quality prediction loop. The coarse picture quality prediction loop includes a coarse picture memory 513, and a motion compensation unit 514 provided with a switch SW4, which uses 17 1293833 to select a prediction mode in the base layer. The principle and operation of the FGS codec used in the present invention are well known and described in the prior art, and the new FGS codec architecture provided by the present invention uses the switches SW1, SW2, SW3 and SW4 to be suitable. Select the prediction mode to improve coding efficiency and performance. The principles of these prediction modes and how they operate are described below. As shown in Fig. 4, the encoder includes two switchers swi. ~ SW2' for selecting the prediction modes of the two motion compensated prediction loops in the enhancement layer and the base layer encoder, respectively. The upper SW1 is used to compensate the motion compensation loop of the enhancement layer encoder to select the prediction from the fine picture memory or the coarse picture memory. SW2 is used to select the prediction mode of the base layer (SW=1: fine image quality prediction, SW=〇: coarse image quality prediction). As extracted from the first table, the present invention provides three kinds of macros in the encoder. Block level coding mode: full fine picture quality prediction (MP: SWl = l and SW2 = 1), full coarse picture quality prediction (ACp: swl = 〇 and 〃 SW2 = 0), and mixed prediction (Mp · · and SW2 = 0). According to the present invention, the code "|| _ 妓 is appropriately selected according to the characteristics of each macro block of the input video signal via the mode switch SW1 and SW2. As shown in Fig. 4 , SW1 and SW2 are controlled by a mismatch estimation and mode decision unit 419. This error matching estimate and mode 1293833 determines unit 419 to calculate the best case and the worst case error matching error. Estimate the amount to make a prediction mode decision. Therefore, in addition to the best case coarse quality prediction output by the motion compensation unit 418, the worst case base-layer decoder 420 also outputs the worst case rough. Image quality reconstructs the picture value. About the dynamic The method of selecting the prediction mode will be described in detail later. One or two of each macroblock is sent to the decoder via variable length coding (VLC) bits to inform the prediction mode used. The mode has different characteristics in coding efficiency and error resistance. If the AFP mode is selected, the base layer and the enhancement layer will use the prediction of the fine picture memory to achieve the highest coding efficiency. However, this mode also has The high risk of drift error is caused by the fact that the receiving end may not be able to completely receive the enhanced layer bit layer for fine-grained and quality prediction because of insufficient bandwidth or packet loss. Overall, how this mode works It is very similar to single prediction loop motion compensation FGS (MC-FGS). Conversely, both the base layer and the reinforcement layer of the 'ACP mode use coarse quality prediction, and if the base layer bit stream can be completely received, this mode is Guaranteed that there will be no drift error, but its coding efficiency is the lowest of the three modes. MP mode is a compromise between coding efficiency and error tolerance, it is The strong layer uses fine image quality prediction, and the base layer uses coarse quality prediction. When the enhancement layer bit layer used for prediction is partially lost, this mode will have drift error in the enhancement layer; If the decoder can receive the complete base layer data, there will be no drift error in the base layer. 19 1293833 In addition to this new 3-mode codec, another special case is the MP and ACP encoding modes. A simplified FGS codec that sacrifices some of the coding gain provided by the AFP coding mode, but reduces the drift error on the other hand. Without this AFP encoding mode, the codec leaves the encoder and decoder architecture as shown in Figures 6 and 7, respectively. This dual mode codec is referred to as the "low-drift" mode, while the 3_mode codec is referred to as the "hign-gain" mode. In this new codec, the cost burden of sending its coding mode is reduced to one bit per macroblock. Table 1 is an excerpt from the prediction mode used by the codec of the present invention. Table 1 Three prediction modes used in the FGS coding method of the present invention

• Both the base layer and the reinforcement layer use fine image quality predictions. Same as the original FGS. • Strong fault tolerance, but coding efficiency is low. Full coarse quality prediction (SW1 = 〇 and SW2 = 0) Low drift error: 1 High gain: 10 Full fine image quality prediction (SW1 = 1 and SW2: Low drift error: NA 1) High gain: 10 Base layer and enhancement The layers are all predicted using fine image quality. Same as single-loop MC-FGS. Has the highest coding efficiency, but is susceptible to drift

The enhancement layer uses fine image quality prediction, mixed prediction (SW1 = 1 and SW2 = 0). Low drift error: 〇 High gain: 0 The base layer uses coarse quality prediction. Same as PFGS. The drift error in the base layer is limited, and the high bit rate is higher than the original FGS

According to the present invention, the motion from the base layer encoder is obtained for the county to perform motion re-estimation (the re-estimation of the line (10) 20 1293833 re-estimation) and to avoid the need to send an additional motion vector (motion vect〇r) for each macroblock. The motion compensation for the enhancement layer encoder is repeated inward s. However, the motion vector of the base layer may not be optimal for encoding this enhancement layer bitstream. As described above, encoding with coarse quality prediction (that is, Acp mode) is less efficient than encoding with fine image quality (that is, APP*^ mode), and if the image quality prediction is made, However, some of the enhancement layer bit layers used for prediction are not received by the decoder, and thus drift errors may occur. The green color of the exposed seed is estimated to be the best choice for estimating the prediction mode when the user bit rate is unknown before encoding. As shown in Figure 8, the present invention uses a two-stage encoding procedure. When the first-loop coding is performed, the coding parameters of all the macroblocks are collected, and the prediction parameters of the parameters are analyzed, and the prediction error values of the fine image quality prediction and the fine image quality prediction are used for prediction. The estimated mismatch error derived from the enhancement of the layer data that cannot be received by the decoder. Among these parameters, the difference between the predicted errors of the two predictions reflects the difference in their coding gains, and the error matching error will spread to subsequent pictures. For example, the coding gain of fine picture quality is significantly higher than the rough picture quality prepayment, and this difference can be estimated by the difference between the coarse picture quality and the fine picture quality prediction error value as follows: (1) 21 1293833 'Especially represents the first/incoming macroblock, and the group ^ and Hu Yi represent ^: the rough image quality prediction and fine image quality prediction. Note that the two absolute values of equation (i) represent the energy values (ie, their size) of the coarse image prediction mode and the fine image prediction mode prediction error value, respectively. A large G value represents that the fine enamel prediction of this macroblock is much more accurate than the coarse quality prediction. However, the coding gain may cause a risk of drift error, because the fine picture prediction uses part of the enhancement layer data, which may not be completely received by the decoder due to insufficient simplification or packet loss, in order to effectively To estimate such drift error, we use the following two estimates: ^ = ^ ^ \\PX^ ^ ^ PXEL n)\\ (2) ^ Wpx^u ^ ^ ~px^ ^ ni\ (3) A and ^ represent the best and worst estimates of the error matching error of the error concealment using the zero motion vector, respectively.乂12 is predictive coding using the previous one (using only the data of the base layer without using any enhancement layer data for reconstruction), so its coding efficiency is relatively low. Therefore, the optimal value β means that it is assumed that the face before the coded picture has received the complete enhancement layer data, and then estimates the error matching error amount of the coarse picture quality prediction code. Conversely, the worst value ^ means that the picture before the encoded picture only receives the data of the base layer, and the data of all the enhancement layers 22 1293833 (10), the error matching error of the transferred code quality code the amount. In this case, the problem that causes the error error continues to spread, and the profit of the drift error is serious. Generally speaking, in the pre-stored video streaming application, since the size and condition of the user's network bandwidth are known in advance, the method accurately estimates the error matching error value. However, since it is known that the actual error matching error falls between these two estimates, it is also nB < Γ) ^ nw ,,,. The present invention uses the weighted average of these two estimates to predict the actual error matching error: > PDt^kDDf +(l-A:D)Z^w (4) Its center Ε[0,1]. The choice is based on the available bandwidth of the decoder. In order to determine the coding mode of each macroblock to achieve good coding performance while taking into account sufficient error resistance, use a new CODE (coding gains over drifting error): - CODE. ^GJPD, (5) where 0, and />/) are obtained by equations (1) and (4), respectively. The CODE index value of equation (5) can effectively represent the relationship between the coding benefit and the drift error of each macroblock. For example, the larger the CODE value, the coding benefit obtained by predictive coding using the system image quality reference picture. The higher and the amount of drift error preceded by it is relatively small. 23 1293833 After extracting the features of all the macroblocks of a picture, the average value of the CODE value and the standard deviation 〇coDE can be calculated as follows··

WCODE

CODE, (6) (7) ac〇〇H=^|(C〇DE/^CODE)2 where is the number of macroblocks in a picture. These macroblocks are then divided into three groups, each of which is coded in a different prediction mode (ie ACP, AFP, MP mode). The macroblock is the average of the CODE values. The standard deviation is classified as follows: fACP //CODE, < Wc〇DE^ac〇D£ MODE, =. AFP if CODE, > mC0DE + kaC0DE (8) MP otherwise Figure 9 illustrates the multiple macroblocks, The relationship between the error matching error and the coding gain and an example distribution map drawn. The X-axis and the γ-axis represent the error matching error and coding gain calculated by Equation (4) and Equation (1), respectively. The higher the Y value of a macroblock, the more layer of the enhancement layer is used in the memory of the fine image reconstruction. Therefore, the finer the prediction of the fineness of the macroblock is, the higher the yield is. The coding gain. However, with these extra bits, the increase in coding gain is accompanied by an increase in drift error. Each point in Figure 9 represents the (G/) pair of one of the macroblocks in each category. The upper and lower solid lines in the figure represent the CODE value equal to 24 1293833 (in this moxibustion = 1) All (G/>) pairs, and the dashed line represents the CODE value equal to gas. All (sand) pairs of DE. (Sand) The macroblock above the upper solid line is encoded in AFP mode, as this can be expected to achieve higher coding performance, while the decoder does not receive some enhancement layer data for prediction. Packets, introduced drift errors will not be so serious. Conversely, (<^D) macroblocks located below the lower solid line are encoded in ACP mode because they are more susceptible to drift errors. Other macroblocks encode the pool in MP mode to achieve a better compromise between coding gain and drift error. 1 ¥ Since the P-picture is used as a reference for subsequent B/P-picture coding, the method of the present invention for determining the prediction mode is applied to the P-picture. Furthermore, since the team picture is not used as a prediction for other pictures, this drift error will not propagate to other pictures. Therefore, in the present invention, the B-pictures all use full-precision image quality prediction, and the coding achieves the highest coding efficiency. When providing the video stream, the video stream server will cut each enhancement layer's surface to an appropriate size. To match the bandwidth of the client terminal device. If fine-quality prediction is used to encode the base layer and the enhancement layer, the bit-allocation scheme used to crop the FGS enhancement layer has a large impact on performance. For example, if more bits can be reasonably assigned to the Ι/P-昼 plane instead of the picture, the decoder may receive more bit layers of the j/p_ picture, resulting in lower drift errors and higher Video quality. In addition, the B-picture can also be referenced to the better quality image when the encoder side is predicted, or when the numerator end 25 1293833 is reconstructed, if there are more enhancement layer layers for these reference images for prediction. Received words. The invention further proposes a new rate adjustment enhancement layer bit layer layer cutting algorithm, which cuts the enhancement layer at the video server end with three different available bandwidths of low bit rate, median rate and high bit rate. Meta layer. In the case of low bit rates, during the encoding process, the available bandwidth is insufficient to deliver all of the enhancement layer bits of the ι/p picture for the two-layer fine picture prediction. Therefore, the tank-shift error is unavoidable when part of the reinforcement layer data used for prediction is discarded during the cutting process. On the other hand, "If the available bandwidth is large enough to send out all enhancement layer bits for fine image quality prediction, but the bandwidth is still smaller than the #ΒΡ of all B-pictures of a group of pictures (GOP) When the number of bits in the MSB bit layer is enhanced, all the extra bits can be allocated in the B-picture to balance the image between the I/P_ picture and the B_ picture, 'quality. If the bandwidth In the better case, the extra bits will also be allocated to the Ι/P-昼 face, and the associated bits will be preserved to avoid drift errors. Such reinforced layer level layer cropping bit rate adjustment actions It can be done at the server or router. The cutting methods for different situations will be detailed as follows. Table 2 below shows the various parameters used in the servo bit layer cutting algorithm of the present invention. Table 2 is used for server rate Description of the parameter parameters of the adjustment _ MjOP GOP screen number N \ I-picture and p-picture number in the GOP Nb A B-picture in a GOP (A / b = β Μ & ι >) 26 1293833 Encoder pre- Pre-encoding The bit layer used for fine image quality prediction in A^bp encoding PBel - G 0P Total number of enhancement layer bits for fine image quality prediction BL - GOP for all images of fine image quality prediction and ρ· picture enhancement layer number PBbxl - JV of all B-pictures in a GOP The number of bits in the enhancement layer MSB bit layer is the number one for fine image quality prediction in the I&PJEL-GOP! /^Written enhancement layer bit number PB^ One GOP in the first b B-picture #BP enhancement layer MSB bit layer bit number Server end bit layer clipping parameter tbel a GOP reinforcement layer The number of bits allocated -- ( ^ ^l & PJEL - GOP in the enhancement layer "the number of bits of the i / p - picture assignment TOm a GOP reinforcement layer; w B - screen assignment of the number of bits Case 1: low available bandwidth In this case, the available bandwidth of the channel is estimated to be less than the number of enhancement layer bits of the picture and the p_ picture used for fine picture prediction at the time of encoding. Since the available bandwidth is not enough to send all the bits for fine image quality prediction, the present invention cuts the enhancement layer of the I-picture and the picture as much as possible. The cutting method of each Ι/p-picture is based on The number of bits used for prediction is adjusted as follows...

27 (9) 1293833 In this case, the number of bits allocated by the B-picture is set to zero, that is, 〇, m=l, 2, ···, and outside. Equation (9) is used when the number of available bits is smaller, and the bits are only assigned to the μ picture and the ρ·book face, and all the enhancement layer data of the frame are discarded in the crop. This method achieves better performance at low bit rates. Case 2 · Medium available bandwidth If the available bandwidth is larger than all the enhancement layer bits sent out for the fine picture prediction and the P-picture, but the bandwidth is still less than P5bel, The device will first assign the bit to the Ι/P-picture to ensure that the Ι/P-picture bit layer for fine 昼 quality prediction can be completely sent to the receiving end, and then the server will assign the extra bits to B-picture. Case 3: high available bandwidth If the available bandwidth is greater than the bandwidth required to send the number of enhancement layer bits for fine quality prediction, then the number of bits used for dispatching It is controlled by the size of the bit layer and varies with the specific bit rate. However, when the bit cell rate is rapidly increased, if there are no more bits assigned to the face, then two adjacent There is a big difference between the pictures. Therefore, the bits need to be balancedly assigned to each picture to avoid huge quality differences. The enhanced layer bit allocation algorithm of the present invention can be summarized by a pseudo program. as follows:

Begin: 28 1293833 if (73⁄4 ^ PBi&p^l ) perform low-rate bit truncation */ TB^l-PBel n = 1,2, ···,M&P; tK^l = 0» m = l ,2,.",iVB; else if (TB^ ^ PBEL) /* perform medium-rate bit truncation */ ^®I&PJEL ~ w = 1,2, ···,M&p; τΒζ^- ΡΒ^x^b^l,mm m-0 else /* perform high-rate bit truncation */

Endif, End The validity of the codec of the present invention can be presented by the experimental results of the simulation. In this experiment, two sets of test screens for C〇astgUard and Mobile were used. These pictures are encoded in a (30, 2) GOP architecture, and the base layer is encoded at a rate of 384 kbits/sec using the TM5 rate control method and its face rate. The size of the face is aF 352 heart 8. Fine picture quality prediction (ie, in AFT and MP modes) uses two enhancement layer bit layers. Fig. 10 and Fig. 10 illustrate the performance of the method of the present invention and the other three methods 29 1293833 in the two sets of test pictures, respectively. The three methods are: baseline FGS, full-precision quality prediction, and single-layer MPEG] codec. The results of the simulation show that the method of the present invention does outperform the other three methods over a wide range of bit rates. The full-fine mode and the baseline FGS method represent two important different quality boundaries at the highest bit rate and the lowest bit rate, respectively. The object of the present invention is to find a good compromise between the two methods over a wide range of bit rates. And this purpose is achieved by introducing a predetermined number of bit layers in the motion compensation prediction of the base layer, but at a low range of low bit rate (384-512 kbps), it can be observed due to drift error. Minor quality is reduced. The method of the present invention is more error tolerant than full fine quality prediction. Full-precision image quality prediction is used in all B-pictures, which can significantly improve coding efficiency without causing error propagation. Motion vectors are derived from high quality predictions. The inter-layer selection method is implemented on the P-picture to improve the coding efficiency of the base layer, and the motion compensation reference writing can be different for the two layers with the same motion information. The base layer and the enhancement layer do not require two sets of motion vectors because more operations and additional bit rates are needed to estimate and transmit an extra set of motion vectors. Therefore, the motion vector estimated by the base layer is repeatedly used for motion compensation in the enhancement layer. When the bit rate is low, the full fine enamel prediction suffers from a loss of approximately 1 dB. With the present invention, the quality degradation due to the drift error at a low bit rate can be significantly reduced, and at a high bit rate of 30 1293833, the achieved coding gain is about 丨~丨5 dB higher than the original generation. Figure 12 and Figure π illustrate the method of the present invention and the other three methods in the two sets of test picture series, the base layer bit rate of 384 kbps, and the two enhancement layer bit rates·· Okbps, 256 kbps, and 768 kbps. , PSNR performance of continuous pictures. The method of the present invention can reduce the drift error more efficiently than the full-fine image quality prediction when the available bandwidth is low, and can maintain the coding efficiency close to the full-fine image quality prediction when the available bandwidth is high. The method of the present invention achieves significantly higher PSNI^ quality improvements than the original FGS. Figure 14 illustrates two images decoded using the present invention and the original FGS to provide an objective performance comparison. However, the above description is only for the preferred embodiment of the present invention, and the scope of the present invention cannot be limited thereto. That is to say, the average change and modification made by Li Fanyuan should be within the scope of this creation patent. 31 1293833 [Simple description of the diagram] Figure 1 shows that the FGS bit stream is transmitted to the receiving end of different bandwidths. Figure 2 illustrates the encoding process for generating the FGS base layer and the enhancement layer bit stream. Figure 3 illustrates the decoding process of the FGS base layer and enhancement layer picture reconstruction. Figure 4 illustrates the coded Is architecture of the new fgs codec for intra-layer prediction in accordance with the present invention. Figure 5 illustrates the decoder architecture of the new fgs codec for intra-layer prediction in accordance with the present invention. Figure 6 illustrates a new fgs codec-encoder architecture for intra-layer prediction in accordance with the present invention, wherein the base layer has only coarse quality predictions. > Figure 7 illustrates a decoder architecture of a new fgs codec for intra-layer prediction in accordance with the present invention, wherein the base layer has only coarse quality prediction. Figure 8 illustrates the two-stage encoding procedure of the present invention. Figure 9 illustrates the relationship between the error matching error and the coding gain for a plurality of macroblocks and an example distribution map. Figure 10 illustrates the performance comparison of the method of the present invention with three other conventional methods using the Mobile Test Sequence. Figure 11 illustrates the performance comparison of the method of the present invention with the other three conventional methods using the Coastguard test series. Figure 12 illustrates the performance comparison between the method of the present invention and three other conventional methods using the Coaster test sequence at a 384 kbps base layer bit rate and three enhancement layer bit rates, (a) 0 kbps (b) 256 kbps (c) 768 kbps. . Figure 13 illustrates the method of the present invention and three other conventional methods in the M〇bne 32 1293833 test sequence, with a base layer bit rate of 512 kbps and three enhancement layer bit rates, (8) Okbps (b) 256 kbps (c) 768 kbps, Performance comparison. Figure 14 illustrates the fourth decoded picture of the 512 kbps base layer and the 512 kbps enhancement layer with (8) the original FGS encoder (27.5 dB) and (b) the Hybrid MB-MSFGS method of the present invention (32.4 dB). image. Figure No. 401 DCT unit 403 maximum value finder 405 bit layer layer splitter 407 fine picture quality picture memory 411 DCT unit 413 variable length coder 415 IDCT unit 4.17 motion estimation unit 402 bit layer shift 404 bit layer variable length coding 406 IDCT unit 408 motion compensation unit 412 quantization unit 414 dequantization unit 416 coarse enamel picture memory 418 motion compensation unit

419 error matching estimation and mode decision unit 420 base layer decoder SW1, SW2 switch 501 bit layer variable length decoder 503 bit layer splitter 505 fine picture quality memory 510 variable length decoder 512 third IDCT unit 502 first IDCT unit 504 second IDCT unit 506 motion compensation unit 511 dequantization unit 513 coarse picture memory 33 1393833 514 motion compensation unit SW3, SW4 switch

Claims (1)

I. Application ¥^Scope·· 1. An encoder with detailed scalability, comprising: a base layer coding block, including a rough image quality prediction loop and a base layer mode selector, the rough image quality prediction The loop has a coarse quality prediction output; J-enhanced layer coding block includes a fine image quality prediction loop and a reinforcement layer mode selector, the fine image quality prediction loop has a fine quality prediction output; and a mismatch An estimation and mode decision unit for appropriately controlling the enhancement layer and the base layer mode selector, and estimating an error matching error between the coarse image quality prediction output and the fine quality prediction output; wherein, when the base layer When the mode selector and the enhancement layer mode selector both switch to select the fine quality prediction output, the encoder operates in a full fine quality prediction mode, and when the base layer mode selector switches to select the rough quality Predicting the output, and when the enhancement layer mode switch switches to select the fine picture quality compensation output, the encoder operates in a hybrid prediction Type, and when the mode selector base layer and the reinforcing layer are the mode selector switch to select the pay coarse quality output, the encoder system to operate in a full coarse picture prediction mode. 2. The encoder with fine scalability as described in item i of the patent application, the encoder further includes a worst case base layer decoder for providing a worst case rough quality output to the The error match estimate and mode determine the XJX3 rate. 3·—A decoder with meticulous scalability, including: 35 1293833 9W day correction i: - base layer decoding block, including - rough 昼 quality compensation loop, the coarse 昼 quality surface loop has - first-switch To select a prediction mode in the base layer, the coarse quality prediction loop is provided with a coarse quality prediction output; a reinforcement layer decoding block includes a fine quality prediction loop, and the fine quality prediction loop has a second switching The switch selects a prediction mode in the enhancement layer, and the fine quality compensation loop is provided with a fine image quality prediction output; wherein 'when the first switching switch and the second switching switch are both switched to select the fine image quality prediction output The decoder operates in a full fine quality prediction mode, and when the first switching switch switches to select the coarse quality prediction rounding, and the second switching switch switches to select the fine and fine quality predicted output The decoder operates in a hybrid prediction mode, and when the first switching switch and the second switching switch are both switched to select the coarse 昼 面 曰 output, 忒 decoding A whole day to make a rough qualitative prediction mode. 4. An encoding method having at least two encoding modes, the encoding method comprising: (a) - a base layer of coarse image quality prediction and a reinforcing layer of a fine image quality prediction, and from a plurality of macro regions of the input signal Each macroblock of the block collects coding parameters including at least a fine picture quality prediction error value, a coarse picture quality prediction error value, and an optimum condition and a worst case condition under fine picture quality prediction. Error matching error; (b) analyzing the encoding parameters to determine the encoding mode of each macroblock; and (c) encoding each macroblock according to the encoding mode determined in the step (b). 5. The encoding method having at least two encoding modes as described in claim 4, wherein the plurality of macroblocks are classified as 36 1293833 and 11⁄2 i % in the step (b). Repair, original: At least two encoding groups, each macroblock in the same group is assigned the same encoding mode. 6. The encoding method having at least two encoding modes as described in claim 4, wherein the encoding method has a full coarse quality prediction mode, a full fine image quality prediction mode, and a hybrid prediction mode. And the plurality of macroblocks are classified into a full coarse quality prediction group, a full fine image quality prediction group, and a mixed prediction group in the step (b), the full coarse prediction group Each macroblock in the group is specified to adopt the full coarse quality prediction formula, and each macroblock in the full fine quality prediction group is designated to adopt the full fine image quality prediction mode, the hybrid Each macroblock in the prediction group is assigned the hybrid prediction mode. 7. The encoding method having at least two encoding modes as described in claim 4, wherein the plurality of tiles are classified into at least two encoding groups according to a coding gain and an estimated error matching error. And the coding gain is derived from the fine picture quality prediction error value of each macro block and the coarse picture quality prediction error value, and the estimated error matching error is obtained from each macro block block. The best case is derived from the worst case error matching error. 8. The encoding method having at least two encoding modes as described in claim 7, wherein the encoding method has a full coarse prediction mode, a full fine negative prediction mode, and a hybrid prediction a pattern, and the plurality of macroblocks are classified into a full coarse quality prediction group, a full fine image quality prediction group, and a mixed prediction group in the ahai step (b), the full rough drawing Each macro-block in the quality prediction group specifies the general prediction mode of the lining, and each macroblock in the full-precision image prediction group is designated 37 1293833. The iL does not adopt the full-precision quality prediction mode, and each macroblock in the hybrid prediction group is designated to adopt the hybrid prediction mode. 9. The encoding method having at least two encoding modes as described in claim 7, wherein the encoding gain of the macroblock divided by the error matching error of the macroblock is defined as the macro The encoding efficiency of the block, and then the macroblock is specified according to its encoding efficiency, using the full coarse image quality prediction mode, the full fine image quality prediction mode, and the hybrid prediction mode. An encoding method having at least two encoding modes as described in claim 9 wherein, from the encoding efficiency of the plurality of macroblocks, a standard deviation between the encoding efficiency average and the _ encoding efficiency is calculated, and then by ratio: The coding efficiency of a given macroblock, and the value determined by the coding efficiency standard mean value of the coding efficiency standard deviation, the macroblock is designated to adopt the full coarse quality prediction mode, the full fine quality prediction One of the mode, and the hybrid prediction mode. The encoding method having at least two encoding modes as described in the item _1G, wherein the encoding efficiency of the macroblock is smaller than a difference between the encoding average and a predetermined multiple of the encoding standard deviation. , the macro block «定采职录录动动_, the code efficiency of the no block is greater than the slanting _ coding county difference - the financial multiples of the tooth giant block is designated to use the full fine 昼Quality prediction mode, otherwise, the macro block is assigned to adopt the hybrid prediction mode. 12. - Cut - The image of the group image - The bit face in the enhancement layer is assigned to the bit channel. The bit is sent to the client channel. The method consists of the following steps: 38 1293833 (8) If the total number of bits allocated by the enhancement layer is less than or equal to the total number of enhancement layer bits of all i/p_pictures used for fine image quality prediction of the group of images, then low rate bit clipping is implemented; (b) If the total number of bits that can be used for the enhancement layer is less than or equal to the total number of enhancement layer bits used by the group of images for fine enamel prediction, but greater than the total number of enhancement layer bits for the group of images. 'The implementation of the rate bit truncation; and (c) if the total number of bits that the 3 enhancement layer can use for dispatching is greater than the total number of enhancement layer bits used by the set of images for fine quality prediction, then a high rate bit truncation is implemented. cut. 13. A method of assigning a bit in a reinforcing layer of a set of images to assign a bit as described in claim 12, wherein the low rate bit is truncated by one bit. Give each of the enhancement layers! /p_昼, the number of bits is proportional to a ratio, the ratio is: each: ^: ^ The number of bits used for prediction and all Ι/P-昼 of the set of images are used for fine 昼 prediction The ratio of the total number of bits, and the low rate bit truncates the unallocated bits to any B-picture of the enhancement layer. 14. A method of assigning a bit in a reinforcing layer of a set of images to assign a bit as described in claim 13 wherein the medium rate bit is truncated by one bit. For each I/p_ picture of the enhancement layer, the number of bits is the number of bits used for fine quality prediction for each Ι/P-picture, and the medium rate bit is truncated by one-bit number For each B_昼 face of the enhancement layer, the number of turns is proportional to a ratio, which is the most significant number of enhancement layers for each B_昼 surface used for fine image quality prediction and the set of images. B_昼面 is used for the ratio of the most significant number of significant layers of the reinforcement layer predicted by Jing 39 1293833 ¢ 5573 and Tian Danbei. Declaring the method described in item 4 of the general plan - the method of assigning bits in a reinforcement layer of the image-group image, wherein the high-rate bit is truncated by the number of bits 70, plus In addition, the bit field and all the pictures of the set of images are used for the total number of most significant bits of the enhancement layer of the fine picture quality, and are assigned to each - I/p_ face of the force layer, the number of bits is The work A picture is used to finely draw the number of bits predicted by f, which is proportional to the - ratio, which is the number of bits used for fine quality prediction per Ι/P-昼 face and the set of pictures. The ratio of the total number of bits used for fine image quality predictions for all ι/p-pictures, and the high-rate bits are truncated with the number of bits, and all the frames of the set of images are used for the enhancement of fine image quality prediction. The total number of most significant bits in the layer, assigned to each B-picture of the enhancement layer, the number of bits being proportional to a ratio, which is the most significant number of significant layers of the enhancement layer used for fine image quality prediction in each b_ All B_pictures of the set of images are used to compare the ratio of the most significant bits of the enhancement layer of the fine picture quality. 16. The method of assigning a bit in a reinforcing layer in a reinforcing layer of the thief cut_group image as described in claim 15 of the patent application, wherein the medium rate bit is truncated by one-digit number For each J/P_ plane of the enhancement layer, the number of bits is the number of bits used for fine 昼 quality prediction for each Ι/P-picture, and the medium rate bits are truncated by one-digit number Assigned to each B_昼 surface of the enhancement layer, the number of bits is proportional to a ratio, which is the most significant number of significant layers of the enhancement layer used for each fine-tune prediction and the set of images. The ratio of the most significant number of bits in the enhancement layer used for fine image quality prediction. 17. A method of cutting a set of images as described in claim 12, 1293833 The method of assigning bits in the bit layer of the strong layer, the high rate bit is truncated by one-digit number, plus another-bit number, and all B-pictures of the set of images are used for fine enamel The most significant bit reading of the predicted enhancement layer is assigned to each [P-picture of the enhancement layer, the number of bits is used for the fine enamel prediction number, and the number of bits is proportional to the The ratio is the ratio of the number of bits used for each fine I/p_picture to the fineness prediction and the total number of bits used for the fine quality prediction of the set of images, and the high-speed bits are clipped to - bit reading, and the total number of linings of the set of images for the fine/beat reinforcement layer, assigned to the enhancement layer. The picture 'the number of bits is proportional to the ratio. The ratio is the ratio. A household account is used to fine-tune the ratio of the most significant number of bits in the enhancement layer to the total number of significant bits in the enhancement layer of the set of images. 41