KR20050095237A - Method and apparatus for coding video data based on switching between single description and multiple description - Google Patents

Abstract

The present invention relates to a data compression technology, and provides a video data compression method and apparatus using a hybrid compression transmission technology using a single expression / multiple expression conversion scheme. The data compression method or apparatus of the present invention provides a method or apparatus for converting input data into a discrete signal, a step or means for quantizing the transformed discrete signal, and whether to compress the quantized signal into a single expression or multiple expressions. A decision step or means for determining a signal, and if said decision step or means determines multi-expression compression, dividing said quantized signal into multiple representations and compressing each representation, said single representation / multiple The representation decision is determined so that the restoration error is minimal at the same amount of redundancy. In the present invention, by using a single-expression / multi-expression switching technique using a bit rate-distortion optimization, the redundant amount that is unavoidable in the conventional multi-expression compression (MDC) is eliminated, and in order to increase the compression efficiency in a channel environment with low packet loss rate, Representation compression is selected, and multi-expression compression is selected to improve error resilience in a channel environment with high packet loss rate. In addition, the single-expression / multi-expression conversion of the present invention can be optimized by considering the compression capacity and the image quality of the video or by using the human visual characteristics.

Description

Method and apparatus for compressing video data using single / multiple expression switching {Method and Apparatus for Coding Video Data Based on Switching between Single Description and Multiple Description}

The present invention relates to a data compression technology, and more particularly, to a video data compression method and apparatus applying a hybrid compression transmission technology using a single expression / multiple expression conversion scheme.

The increase in digital multimedia data has created a need for the storage and transmission of such data. In particular, as the Internet, mobile phones, personal mobile terminals, and the like are widely used, interest in the transmission of video data is greatly increased. In the conventional transmission using a dedicated line or a telephone line, the probability of error is very low, but in the new transmission medium, the Internet or wireless environment, the probability of error is very high and such errors occur continuously. In addition, since a moving picture has a very large capacity, a moving picture is compressed, stored and transmitted using a method such as MPEG. This video compression has the advantage of reduced capacity, while the compressed signal is very sensitive to errors. In addition, when an error occurs due to the characteristics of the compression method, the error is propagated not only to the part but also to the periphery, and thus, the image quality is greatly degraded.

In order to solve this problem, an error-resistant compression transmission technique has been developed. Compression transmission techniques that are robust against errors include a method of adding information that can be restored when an error occurs as additional information, and a method of hiding an error that occurs when an error occurs without adding additional information. However, the method of adding additional information has a disadvantage of increasing the capacity of the compressed oriental image, and the method of hiding an error has a disadvantage of increasing the amount of calculation. Also, the biggest problem with this method is that it is not effective if there is a high probability of error or if a continuous error occurs. That is, it is not suitable for the Internet and wireless environment.

In contrast, the multiple description compression technique can perform effective data transmission even when a high probability of error or a continuous error occurs. Multiple expression techniques for original signals have been widely studied in the field of information theory and signal processing (e.g., VA Vaishampayan, "Design of multiple description scalar quantizer", IEEE Trans. Inform. Theory No. 39, pp. 821-834 , May 1993 or V. Goyal, "Multiple description coding: Compression meets the network", IEEE Signal Processing Mag., Vol. 18, pp. 74-93, Sep. 2001). This technique is a method of transmitting data through different paths by compressing the video into multiple representations to minimize redundancy. When data is transmitted using different paths, the probability of the same error occurring in all paths is very small, so that even if an error occurs in some paths, it can be effectively transmitted.

However, in the multi-expression compression technique, even if only a part of the multiple representations is received, the bit rate is increased because overlapping information is inserted between each representation to obtain satisfactory image quality. Increasing the robustness according to the increase of the bit rate is effective in an error-prone environment, but the compression efficiency is greatly reduced in an error-prone environment. Therefore, an optimal control of the bit rate versus robustness is a task of multi-expression video compression.

An object of the present invention is to provide a multi-expression compression transmission technology with improved performance compared to the existing multi-expression compression technology used when transmitting video in the Internet and wireless environment.

Another object of the present invention is to provide an apparatus and a method for compressing video data, in which bit rate and error robustness are optimally adjusted.

According to the first aspect of the present invention, a video may be selected and transmitted in a single expression or multiple expressions according to a channel environment. That is, the data compression method or apparatus of the present invention comprises the steps or means for converting the input data into a discrete signal, the step or means for quantizing the transformed discrete signal, and compressing the quantized signal into a single expression or multi-expression. A decision step or means for deciding whether or not to make a decision, and if said decision step or means determines multi-expression compression, dividing said quantized signal into multiple representations and compressing each representation or means; Multiple expression decisions are determined so that the reconstruction error is minimal at the same amount of redundancy.

The present invention eliminates excessive redundancy in the conventional multiple description coding (MDC) using a single expression / multiple expression switching technique using a rate distortion (RD) optimization. In the conventional MDC, duplication is inevitable due to correlation between representations. However, in the case of a low packet loss rate, the duplication reduces compression efficiency. In order to solve the problem of the conventional MDC, the present invention adopts a so-called hybrid MDC that controls single-expression / multi-expression switching according to the channel environment. For example, in a channel environment with a low packet loss rate, single expression compression is selected to improve compression efficiency, and in a channel environment with a high packet loss rate, multi-expression compression is selected. In order to optimally control the single-expression / multi-expression switching, the present invention uses a bit rate distortion optimization technique. For example, the Lagrangian optimization technique is used to convert the single expression / multiple expression so that the bit rate distortion cost function D + λ R (where R is actually the compressed bit rate and D is the calculated reconstruction error) is minimized.

In the present invention, the determination of the single expression / multiple expression is selected in consideration of the compression capacity and the image quality of the video. In addition, the determination of monoexpression / multiexpression of the present invention can be optimized using human visual characteristics.

In the present invention, a multi-expression video compressor including a central motion predictor and a lateral motion predictor can be used to compress the multi-expression, and a new optimization algorithm using redundancy, alternating and limiting quantization according to frequency position and a given objective function is used. do.

Embodiment

Embodiments of the present invention will be described below with reference to the drawings. In the following description, 'compression' corresponds to 'coding' or 'coding', and 'distortion' and 'error' are used in the same meaning (distortion).

A bit stream of a video to which the present invention can be applied may be configured as shown in FIG. 1.

As shown in FIG. 1, the basic block of the video is an 8 × 8 pixel block, and one macro block includes four luminance signal blocks Y1, Y2, Y3, and Y4 and two color difference signal blocks Cb, Cr). A DCT operation is performed on this macro block. Temporal information compression (interframe prediction) is performed by motion compensation on the luminance signal block, and spatial information compression by DCT can be performed on each block. GOB (Group of Blocks) is composed of eleven macroblocks, for example, and constitutes an image frame with nine GOBs. The image frame becomes one screen having a common attribute, and the GOB becomes a small screen obtained by dividing one screen into 12 screens.

The image frame shown in FIG. 1 is merely exemplary and the present invention can be applied to an image frame having a different configuration (for example, 288 lines x 352 pixels, etc.). It will be easy to understand. The hybrid compression transmission method using the single-expression / multi-expression switching of the present invention for the video bit stream illustrated in FIG. 1 is performed in units of frames.

2 is a block diagram showing the progress of a hybrid compression method according to the present invention.

Referring to FIG. 2, an input frame is discrete cosine transformed using a discrete cosine transform (DCT) (10), and quantized using a quantum and coefficient (QP) as a standard video coder (20). In a next step, it is determined whether to compress the quantized frame into a single description or multiple description (30). If it is determined as a single-expression compression, the quantized frame is compressed in the same manner as the general video compression method and transmitted through the channel 40. On the other hand, if the decision step or the decision means 30 determines with multi-expression compression, the frame is divided into multiple representations through an optimization process and then the respective representations are compressed in the same way as the existing video compression method, and channels of different paths are used. 40 is sent to the user. The channel 40 also transmits the above-determined result, that is, SD / MD (single-expression / multi-expression) switching information, so that the receiving side can use it for decoding.

Determination of the single expression / multiple expression made in the decision step or the determining means 30 uses a Lagrange multiplier method to minimize the reconstruction distortion at the same redundancy. It is preferable to follow. In order to calculate the amount of redundancy, a bit rate to be generated is calculated using entropy coding such as Huffman coding and arithmetic coding. The reconstruction error used in the Lagrangian multiplier method is obtained using an error model considering the probability of an error occurring in a channel and an error concealment method performed by the decoder when an error occurs. The reconstruction error d _n ⁱ for each pixel obtained by the error model can be obtained by the following equation.

In Equation 1 above Is the original pixel expressed as the random variable and i-th pixel of n-th frame Is a decompression pixel of the decoder after error concealment. About the encoder Is a random variable. Equation 1 above uses a minimum squared error to calculate the reconstruction error, which is simple to calculate and the value obtained is closely related to the image quality that a person feels. Therefore, in the present invention, it is possible to optimize the conversion or determination of single expression / multiple expression using human visual characteristics.

Restoration error In Equation 1, the condition of the decoder (that is, the decoder restoration error at the compression moment) may be expressed as Equation 2 below using a packet loss rate (PLR).

Where D is the distortion and C ₁ , C ₂ are the DCT coefficients for the representations S1, S2, respectively.

Meanwhile, in Equation 1, the expectation varies depending on whether the expression is single or multiple. It also depends on whether it is compressed in intra mode or inter mode. Therefore, the following four cases should be considered when finding the restoration error by the given error model. In the following equation, i is an index of a current pixel, j is an index of a motion compensated pixel, and k is an index of a pixel used for error concealment.

(1) In the case of the intra form in the single expression method

In Equation 3 and Equation 4 above, the capital letter 'I' denotes an intra form, p denotes a packet loss rate, and (1-p) denotes an error × probability, Denotes the original quantized pixel. Also, Means a pixel which can be restored using the surrounding pixels even if an error occurs. Means a pixel that cannot be restored.

(2) In case of inter format in single expression method

In equations (5) and (6), P stands for Prediction and represents an inter form, Is the remaining value after motion compensation.

(3) In the case of the intra form in the multiple expression method

(4) In case of inter format in multiple expression method

3 is a block diagram illustrating an SD / MD determination process according to the present invention. Assuming that there are N frames in the video sequence and M blocks in one frame, each of N frames (quantized frames) of M blocks is input in step 100 of FIG. 3. The optimal selection of SD compression or MD compression for such an input frame may be made by selecting a mode that minimizes the cost function, as expressed by Equation 11 below.

Here, E _{i, mode} is a cost function that can be defined by Equation 12 below.

Where μ is a constant associated with the average of the Lagrange's multipliers used for optimal division of the DCT coefficients, and D _{i, mode} and μR _{i, mode} are the reconstruction error and the overall bit rate of the i th frame, respectively. Equations 13 and 14 may be defined.

Here, d _{i, j, mode} and r _{i, j, mode} are the reconstruction error and the bit rate of the j th block, respectively, which can be obtained through the optimal division of the DCT coefficients.

In the Lagrange multiplier described above, the SD / MD decision is chosen to minimize the Lagrange cost.

In Equation 15, [lambda] is a Lagrange multiplier predetermined in consideration of video quality and bit rate, D _total is a sum of reconstruction errors obtained by the error model described above, and ρ is an overlapping amount. The overlapping amount ρ becomes '0' in the case of a single expression, and in the case of multiple expressions, the bit amount required to compress the single expression is subtracted from the bit amount necessary to compress all the expressions. On the other hand, in Equation 15, D _total is a variable that allows the decision in consideration of the image quality in the single / multi-expression decision, ρ is a variable considering the compression capacity. In general, smaller D means better image quality, and smaller ρ means smaller compression capacity. Therefore, the single expression / multi-expression conversion or determination of the present invention can be selected in consideration of the compression capacity and the image quality of the video.

First, J _{single expression} and J _{multiple expression} are obtained (steps 130 and 160 of FIG. 3). In order to obtain J _{single expression} and J _{multiple expression} , error prediction (step 110 and 140) and overlapping amount calculation (step 130 and 150) for single expression and multiple expressions are performed through Equations 1 to 10 above. Here, the amount of redundancy can be calculated by subtracting, for example, 'Rate _{multi-expression} -rate _{single-expression} ', that is, 'bit when single-expression compression' is subtracted from 'bit when multi-expression compression'. Next, J _{single expression} is compared with J _{multiple expression} (step 170), and the selected value is selected to be smaller and transmitted to a separate decoder or included in a compressed bit string (step 180, 190, 200).

Multiexpression Compression

In the multi-expression compression of the present invention, the multi-expression video compressor of FIG. 4 may be used.

There are three types of multi-expression video compression, including methods using a predesigned multi-expression scalar quantizer (MDSQ) (e.g. VA Vaishampayan & S. John ' Balanced interframe multiple description video compression ' Prod. Int. Conf. Image Processing, Oct. 1999, pp. 812-816), methods using pairwise correlating transform (PCT) (e.g., A. Reibman, H. Jafarkhani, Y. Wang, M. Orchard & R. Puri, ' Multiple description coding for video using motion compensated prediction '' Proc. Int. Conf. Image Processing, Oct. 1999, pp. 837-841), methods using bitrate-distortion optimization (eg, A. Reibman, H. Jafarkhani, Y. Wang & M. Orchard, `` Multiple description video using rate-distortion splitting '' Proc.Int.Conf.Image Processing, May 2000, pp. 978-981). The bit rate-distortion splitting method has a merit that it can be coded as a standard, simple to implement, and has higher performance than the method using multiple description scalar quantization (MDSQ) and PCT at a high bit rate. In the present invention, a method of using and overlapping or alternately storing frequency characteristics as well as the magnitude of DCT coefficients, and using a new algorithm considering limited quantization. This can find more candidate points on the Redundancy-Rate Distortion (RDR) curve and consequently increases performance. We also define and use a new distortion measurement method that considers not only the lateral recovery error but also the central recovery error.

The multi-expression video compressor 300 of FIG. 4 assumes that the input stream is divided into two representations. k th frame F _k and (k-1) th reconstructed frames Given this, the encoder 350 can predict the motion vector (MV). Is a reconstruction frame using both representations, and Are reconstructed frames using only one representation each. The encoder 350 may use all of the reconstructed frames to predict the motion vector (MV). MV is applied to each motion predictor 320, 330, 340 to predict the signal And predictive error signals To produce The prediction signal P _{0, k} is called central prediction, and P _{1, k} and P _{2, k} are called side prediction. Central prediction error Has two representations Wow Divided into two independent channels and transmitted to the decoder. Side prediction error Wow Is a side loop distortion Used to calculate. This side reconstruction distortion is an important cause of the mismatch between the prediction loops in the encoder and decoder in a multi-express video compressor. If the channel is broken, a reference frame at the encoder And reconstruction frames at decoders Is There is a difference. As a result, there is a mismatch between the encoder and the decoder, resulting in poor performance. Distortion under excess bit rate constraints to eliminate inconsistency between encoder and decoder in advance Should be minimized. The MD encoder of FIG. 4 minimizes this mismatch according to the optimization algorithm using the bit rate-distortion operator of the present invention.

The optimization algorithm using the bit rate-distortion operator of the present invention applies three methods of duplication, alternation and constrained quantization according to the frequency position and the given objective function. Redundancy stores coefficients identically in two representations, alternating stores coefficients in only one of the two representations, and constrained quantization quantizes into two new coefficients such that the sum of the quantized coefficients equals the original coefficients. The divided coefficients are stored in each representation. Redundancy and alternating operations are similar to the rate-distortion division method. Theoretically, an input block with 64 coefficients requires 3 ⁶⁴ operations to optimize with three operators. However, this method is impossible to implement, so we use the greedy approach. In other words, optimization is performed in order from coefficients with small indexes to coefficients with large indexes. In this way, the maximum number of operations is 64x3.

CODEC implementation

An MDC codec embodying the present invention based on the ITU-T H.263 video codec will be described. Unlike conventional SD and MDC video coders, the MDC codec of the present invention should inform the decoder of the SD / MD conversion. In order to preserve the H.263 bit stream syntax, extra insertion information (PEI) and supplemental enhancement information (PSUPP) bits are used in a picture header area. PSUPP data consists of a 4-bit function type indication FTYPE followed by a 4-bit variable data size specification DSIZE. Decoders that receive unsupported functional notation may discard the variable data size for this function. Since an extended function type is used to exchange SD / MD switching information, a video decoder that does not support such a function type may discard the SD / MD switching information.

The TMN8 rate control scheme and Lagrange multiplier selection are used to meet the bit rate constraints. According to the inventors' experiment, the optimal Lagrange multiplier λ is determined by the following equation (16).

Where the constant c is 0.85 in the H.263 video encoder. In SD-ROPE, 0.85 can be used as the value of c. Since MD video compression is different from SD video compression, a new Lagrangian multiplier selection method as shown in Equation 17 below is used.

In MDC, Lagrange multipliers control not only compression efficiency but also error resilience. If the PLR (Packet Loss Rate) is high, more redundancy is needed to prepare for packet loss errors. It is therefore desirable to include the PLR in the Lagrange multiplier selection function. As the inventors repeated the simulation, in the case of MD-ROPE and hybrid MDC, the constant c may be selected as 0.15 in Equation 17 above.

Performance evaluation result

The inventors compared the performance of the hybrid MDC of the present invention with SDC-ROPE and MDC-ROPE using TMN8 bit rate control and the new Lagrangian multiplier selection method described above. 5 shows the concept of three algorithms applied to this evaluation. In FIG. 5, the upper row shows a case where the packet loss rate PLR is high, and the lower row shows a case where the packet loss rate PLR is low. Gray blocks mean intra blocks. Hybrid MDC of the present invention uses SD / MD switching according to channel environment, while SDC-ROPE and MDC-ROPE use intra / inter mode switching. The inventor's evaluation uses baseline H.263 and includes GOB headers. Each frame is divided into two representations, in the case of SD, one representation is assigned to the odd GOBs and the other representation is assigned to the even GOBs. The update rate of the intra frame was set to every 10 frames.

Thirty different packet loss patterns were injected for each video sequence and the results averaged. 6 and 7 show packet loss performance for the Mother & Daughter sequence and Foreman sequence, respectively. In case of mother & daughter sequence, the performance of SDC-ROPE is slightly better than that of hybrid MDC of the present invention at low packet loss rate (PLR), because of the low motion characteristics and SD / MD conversion information of the sequence. However, in the case of Formenam sequences, the hybrid MDC of the present invention is shown to be much better than the conventional SD-ROPE and MD-ROPE for both low packet loss rate (PLR) and high packet loss rate (PLR).

The embodiments of the present invention have been described above with reference to the drawings. However, this is only a description for the implementation of the invention and is not intended to limit the scope of the invention. For example, a method of obtaining a restoration error has been described by presenting Equations 1 to 9, but the present invention is not necessarily able to obtain a restoration error only through such a method. For example, in the above description, the restoration error is obtained in units of pixels. It is also possible to obtain the restoration error in macroblock units within the scope of the present invention.

As described above, according to the hybrid compression transmission method using the single expression / multiple expression conversion of the present invention has the following effects. Conventional multi-expression compression methods have been effective in a high error environment, but have low compression efficiency in a low error environment. However, in the method of the present invention, even in a changing channel environment, the robustness of the error can be increased and the error is less generated by using multiple expressions in an environment where a large number of errors are generated by switching a single expression / multiple expression adaptively to the channel environment. In one environment, single-expression compression can be used to increase compression efficiency.

In particular, in an environment where more error occurs than conventional multi-expression compression schemes, the compression performance is better in an environment where the method of the present invention is more robust to errors and less error-prone.

1 is a block diagram showing the configuration of a video bit stream to which the present invention can be applied.

2 is a block diagram showing the overall flow and configuration of a video data compression method or apparatus using a single expression / multiple expression conversion according to the present invention.

3 is a complete flow diagram of a hybrid compression technique using the single / multiple expression conversion of the present invention.

Figure 4 is a block diagram showing the structure of a multi-expression video compressor that can be used in the present invention.

5 is a view for explaining the concept of three algorithms for evaluating the performance of the video data compression apparatus according to the present invention.

6 is a graph for explaining a result of evaluating the performance of the video data compression apparatus according to the present invention.

7 is a graph for explaining a result of evaluating the performance of the video data compression apparatus according to the present invention.

<Description of Major Symbols in Drawing>

10: DCT (Discrete Cosine Transform)

20: Quantization 30: SD / MD (Single Expression / Multiple Expression) Determination

40: channel 320: center motion predictor

330, 340: side motion predictor

Claims (13)

As a data compression device, Means for converting input data into discrete signals, Means for quantizing the transformed discrete signal, Determining means for determining whether to compress the quantized signal into a single representation or multiple representations; Means for dividing the quantized signal into multiple representations and compressing each representation when the decision means determines with multi-expression compression, And said determining means is a step of determining such that reconstruction distortion is minimized at the same redundancy. In claim 1, And the reconstruction error is obtained using an error model considering a probability of an error occurring in a channel and an error concealment method performed by a decoder when an error occurs. In claim 1, And said determining means uses a Lagrange multiplier method. In claim 1, And calculating the bit rate to be generated by using entropy encoding. In claim 1, It said determining means D _{i, mode} + μR _i, select the mode in which the cost function is expressed as a _mode to a minimum, where and μ is a constant, D _{i, mode} and μR _{i, mode} is restored error of the i-th frame, respectively And the total bit rate And And d _{i, j, mode} and r _{i, j, mode} are the reconstruction error and the bit rate of the j th block, respectively. In claim 3, The determining means is Lagrange Coast Where λ is the Lagrange multiplier, D _total is the sum of the restoration errors obtained by the error model, and ρ is the restoration amount. In claim 6, The Lagrange multiplier λ is or Wherein c is a constant, QP is a quantization coefficient, and PLR is a packet loss rate. In claim 1, Means for calculating an error prediction value and an overlap amount of a single expression for a signal quantized by the quantization means; Means for calculating an error prediction value and an overlap amount of the multi-expression of the signal quantized by the quantization means; A single expression cost and a multi-expression cost are obtained based on the values calculated by the calculating means, and the values are compared with each other, and when the value of the single-expression cost is smaller than the value of the multi-expression cost, the quantized signal is single. Means for causing representation compression and for causing the quantized signal to be multi-express compressed if the value of the multi-expression cost is less than the value of the single-expression cost. In claim 1, And when the decision means determines the multi-expression compression, compressing the quantized signal using a multi-expression video compressor including a central motion predictor and a lateral motion predictor. As a data compression device, Means for converting input data into discrete signals, Means for quantizing the transformed discrete signal, Determining means for determining whether to compress the quantized signal into a single representation or multiple representations; Means for dividing the quantized signal into multiple representations and compressing each representation when the decision means determines with multi-expression compression, And said determining means determines single-expression compression and multi-expression compression such that the bit rate distortion cost function D + lambda R is minimum, where D is the calculated reconstruction error and R is the actual compressed bit rate. As a data compression method, Converting the input data into a discrete signal, Quantizing the transformed discrete signal, A decision step of determining whether to compress the quantized signal into a single expression or multiple expressions, If the decision step is determined by multi-expression compression, dividing the quantized signal into multiple representations and compressing each representation; The determining step determines the single-expression compression and the multi-expression compression such that the bit rate distortion cost function D + λ R is minimum, where D is the calculated reconstruction error and R is the actual compressed bit rate. In claim 11, The determining step is Lagrange Coast Where λ is the Lagrange multiplier, D _total is the sum of the restoration errors obtained by the error model, and ρ is the restoration amount. In claim 11, Calculating an error prediction value and an overlap amount of a single expression for the quantized signal; Calculating an error prediction value and an overlapping amount of the multi-expression of the quantized signal; The single-expression cost and the multi-expression cost are obtained based on the values calculated by the calculation steps, and the values are compared with each other. Causing the quantized signal to be multi-express compressed if the value of the multi-expression cost is less than the value of the single-expression cost.