KR100877127B1 - Method, apparatus, and system for enhancing robustness of predictive video codecs using a side-channel based on distributed source coding techniques - Google Patents

Abstract

A method, apparatus, and system for providing distributed source coding techniques that improve data coding performance, such as video data coding, when channel errors or losses occur. Errors in the reconstruction of the data are eliminated or reduced by sending extra information. The correlation between the predicted sequence and the original sequence can be used to design the codebook and find the corset required to represent the original image. This information may be transmitted on another channel, namely the second channel.

Description

METHOD, APPARATUS, AND SYSTEM FOR ENHANCING ROBUSTNESS OF PREDICTIVE VIDEO CODECS USING A SIDE-CHANNEL BASED ON DISTRIBUTED SOURCE CODING TECHNIQUES}

35 Claims of priority claims under U.S.C §119

This application is entitled “ Method, Apparatus, and System for Improving the Robustness of Predictive Video Codec Using Side-Channel Based on Distributed Source Coding Techniques”, filed June 1, 2004, Claims priority to US provisional patent application 60 / 576,193, assigned to the applicant and incorporated herein in its entirety.

Reference to Joint Application for this Application

This application is entitled "Method, Apparatus and System for Encoding and Decoding for Side Information for Multimedia Transmission", filed March 24, 2003, assigned to the applicant of this specification, US Patent Application No. 10 / 396,982, which is incorporated by reference.

Background

1. Technology Field

The present application is generally related to multimedia by a network, which may be in error, and more particularly to video coding and decoding techniques used for multimedia by a wireless network.

2. Background Technology

Current video compression techniques, such as ITU-T H.263, H.263 +, H263 ++, ISO MPEG-4 and JVT H.264, are very efficient in the compressed video domain. The ability to achieve good visual quality at relatively low bit rates has led to an increase in popularity for video applications in emerging multimedia applications for bandwidth limited channels, such as wireless channels. However, the prediction characteristics of the compression techniques used in current video codecs make the compressed video bit streams very vulnerable and easily degraded due to packet loss in the channel.

Channel errors or packet loss can result in a loss of synchronization between the video encoder and the decoder. More specifically, the locally decoded copy of the reconstructed video frame at the encoder may not match the corresponding reconstructed video frame at the decoder. Because there is a drift between the receiver's decoder and the sender's decoder, this loss of synchronization between the video encoder and the decoder is also sometimes called "drift". Drift can cause large losses in the decoded video quality.

The reduction in decoded quality due to drift is a direct result of the prediction based coding underlying current video codecs. In other words, to decode the current video frame now, all previous video frames need to be correctly reconstructed. This problem is exacerbated in wireless channels, where packet losses are much more frequent than wire-line networks and tend to burst.

Re-transmission schemes or Forward Error Correction (FEC) schemes, such as Automatic Repeat Request (ARQ), handle drift problems between the encoder and decoder and thereby ensure It was used to mitigate the loss. However, ARQ or FEC schemes (or a combination of both) have not proven effective because the latency constraints of video applications prevent the use of such schemes. In addition, FEC basic schemes cannot guarantee that data will be received, and if not, the drift will continue until the next intra-frame ¹ is received. (Intra-frame ¹ is used by video encoders to break the prediction loop to stop drift.)

Wyner-Ziv theory of AD Wyner and J.Ziv "rate distortion function for source coding with side information at the decoder (The rate distortion function for source coding with side information at the decoder)" IEEE Trans . Inf . Theory , vol. 22, pp. 1-10, January 1976 issue. This theory solves the problem of source coordination with side information. Consider two correlated sources X and Y. Only when the decoder has access to source Y the encoder needs to compress the source X. The mean squared error is the distortion magnitude and X = Y + N, and the ratio-distortion performance for X coding is the same whether or not the encoder has access to Y.

Recently, based on the principle of source coding with side-information between the Wyner-Ziv frameworks, junction source-channel coding techniques have been proposed to solve the problem of drift. For example, Allerton of R. Puri and K. Ramchandran Conference on Communication , Control and "PRISM: A New Robust Video Coding Architecture based on Distributed Compression Principles" in Computing , 2002; IEEE of A. Sehgal, A. Jagmohan, and N. Ahuja Trans . on Multimedia , vol. 6, pp. 249-258, 2004, "Wyner-ziv Coding of Video: An Error-Resilient Compression Framework"; Proc . In A. Aaron, S. Rane, R. Zhang, and B. Girod . IEEE Data Compression Conf . , "Wyner-Ziv Coding of Video: Applications to compression and error resilience" in 2003; And Proc . By A. Aaron, S. Rane, D. Rebollo-Monedero, and B. Girod . IEEE Int . Conf . Image Proc . See, "Systematic Lossy Forward Error Protection for Video Waveforms," in 2003.

Allerton by R. Puri and K. Ramchandran Conference on Communication , Control and "PRISM: A New Robust Video Coding Architecture based on Distributed Compression Principle" in Computing , 2002; And Proc . In A. Aaron, S. Rane, R. Zhang, and B. Girod . IEEE DATA Compression Conf . The codecs described in "Wyner-Ziv Coding of Video: Applications to compression and error resilience" in 2003, are full-lengths that avoid the predictive coding framework. fledged) video codec.

Meanwhile, IEEE of A. Sehgal, A. Jagmohan, and N. Ahuja Trans . on Multimedia , vol. 6, pp. 249-258, 2004, "Wyner-ziv Coding of Video: An Error-Resilient Compression Framework"; And Proc . By A. Aaron, S. Rane, D. Rebollo-Monedero, and B. Girod . IEEE Int . Conf . Image Proc . The codecs described in "Systematic Lossy Forward Error Protection for Video Waveforms of Video Waveforms" in 2003 maintain a predictive coding framework but transmit extra information to mitigate the effects of drift.

Despite these developments, in addition to enabling robust video delivery over wireless channels by helping to mitigate the effects of packet loss on the compressed video bit stream transmitted for such wireless channels, You need skills that satisfy your needs.

Can work with existing compression techniques.

Low processing latency.

• Quickly remove / reduce drift between video encoder and decoder.

Require a lower bit rate compared to that used by the video codec.

Summary of Embodiments

Embodiments disclosed herein satisfy this need by providing distributed source coding techniques that improve video coding performance when channel errors or packet losses occur.

The following terms are used.

Main stream: video data compressed using predictive codecs

● Main channel: Logical channel through which the main stream is sent

● WyZER stream: Wy ner Z iv E rror R esilience stream composed using the distributed coding principles

● Secondary channel: Logical channel through which the WyZER stream is sent.

● Main Channel Encoder: the encoder used to encode the main stream

● main channel decoder: decoder used to decode the main stream

● WyZER encoder: Encoder used to encode the WyZER stream

● WyZER Decoder: Decoder used to decode WyZER streams

Side information: Decoded main stream data used by the WyZER decoder to reconstruct a frame at the receiver.

DSC: distributed source coding

The correlation between the reconstructed video frame at the decoder with the channel error and the video frame at the encoder is evaluated. In one embodiment, the correlation is calculated in the discrete cosine transform (DCT) domain. The interrelationship is filed on March 24, 2003, entitled "Method, Apparatus and System for Encoding and Decoding Side Information for Multimedia Transmission". It can be used to find the coset described in US Patent Application No. 10 / 396,982.

Embodiments of the present invention may help to mitigate the drift problem in conventional video codecs by the WyZER stream as an error recovery stream. Embodiments can help eliminate or reduce drift by sending extra information. For example, in one embodiment, an auxiliary channel technique can be used in which the WyZER stream can be used to improve video quality and stop drift. The interrelationship between the predicted sequence and the original sequence can be used to find the corsets required to design the codebook and represent the original image. This information may be transmitted for another channel, namely the second channel. In another embodiment, the number of required corsets may be communicated to the WyZER decoder for the second channel in the form of header information.

In one embodiment, using DSC, another technique of the video sequence is at a lower rate relative to the second channel that can be used to correct errors caused by channel losses in the reconstructed video frame at the receiver. Can be sent. The bit stream transmitted for the second channel can be decoded with the help of the video frame reconstructed by the prediction decoder. As a result, the influence of error propagation can be mitigated.

In one embodiment, if the decoder can only decode main stream data, the decoder will discard the additional information transmitted on the auxiliary channel. If the decoder decodes all of the streams, the decoder first decodes the main stream data and then uses this information to decode the WyZER stream data and give the user the final reconstructed image.

Brief description of the drawings

1 is a block diagram of a wireless communication system.

FIG. 2 (a) is a diagram illustrating a peak signal with noise ratio (PSNR) to frame number for a video sequence encoded with an H.263 + encoder and DSC techniques.

2 (b) is a diagram of PSNR vs. frame number for a video sequence encoded with H.263 + encoder, FEC, and DSC techniques.

FIG. 2 (c) is a diagram of PSNR versus frame number for another video sequence encoded with an H.263 + encoder, FEC, and DSC techniques.

FIG. 2 (d) is a diagram of PSNR versus error probability for a video sequence named “Stefan” encoded with H.263 + encoder, FEC and DSC techniques.

2 (e) is a diagram of PSNR vs. error probability for a video sequence named “Football” encoded with H.263 + encoder, FEC and DSC techniques.

2 (f) is a diagram of the average burst length to PSNR ratio for a “Football” video sequence encoded with FEC and DSC techniques.

3 is a block diagram illustrating a portion of a wireless communication system that can implement DSC techniques.

4 is a flow diagram illustrating an embodiment for determining intra-frame bit allocation for a side-channel encoder.

5 is a flow diagram illustrating an embodiment for determining the maximum number of divisions of a source codebook that can be used by the side-channel encoder.

6 is a block diagram of a WyZER encoder.

7 is a block diagram of a WyZER decoder.

details

Current video coding standards use a prediction framework to achieve good compression performance for video signals. Examples of such compression schemes include MPEG-4, H.263 + and H.264. However, when such compressed bit streams are transmitted for channels that may be in error, video quality is severely compromised, as demonstrated by using an objective PSNR metric in FIGS. 2 (a)-(c). do. Examples of channels that may be in error include cdma2000®, WCDMA, GSM and other emerging wireless networks. In the present invention, video data compressed using prediction codecs is called a "main stream" and a logical channel through which the main stream is transmitted is called a "main channel".

Examples of mitigating the negative effects of channel errors include retransmission of transmission units with errors (RLP retransmission, hybrid ARQ, etc.), error recovery packetization (data where the more significant bits in the transmission units are fed with greater error protection. Distribution, data interleaving, etc.), and bit stream syntax that limits the range of lost source data (resynchronization markers, reversible variable length coding, etc.).

In the present invention, individual bit streams are created for use with the main stream to improve error recovery. This stream is constructed using Wyner-Ziv coding of video data based on distributed source coding principles. We call this stream Wy ner Z ev er R e silience stream (WyZER stream). The WyZER stream is transmitted on a logical channel, " second channel, " different from the primary channel used for video codecs such as MPEG-4, H.263 + and H.264. The video frame reconstructed by the predictive video coder serves as side-information to the WyZER decoder for the purpose of decoding the WyZER stream.

According to another aspect of the invention, the techniques are described in R. Zhang, SL Regunathan, and K. Rose Proc . 33 rd Ann . Asilomar Conf . on Sig . Syst . Comp . Discrete Cosine Transformation (DCT) using some of the concepts disclosed in 1999, "Optimal intra / inter mode switching for robust video communication over Internet." It is provided to evaluate the interrelationship between the frame itself and the misconstructed frame in the domain.

According to another aspect of the invention, the WyZER encoder operates in the DCT domain to take advantage of the energy density characteristics of the DCT. Depending on the rate available for side-channel and correlation evaluation, as many DCT coefficients as possible, starting from DC and low-frequency coefficients, are encoded.

If the customer receives in both the main stream and the WyZER stream but can only decode the main stream (i.e. has only a predictive decoder, such as an MPEG-4 or H.263 + or H.264 decoder), then the customer only Will decode and ignore the WyZER stream. A customer with both a predictive decoder and a WyZER can decode all of the bit streams and thus receive a higher quality decoded video than a customer with only a predictive decoder.

IEEE of A. Sehgal, A. Jagmohan, and N. Ahuja Trans . on Multimedia , vol. 6, pp. 249-258, 2004, "Wyner-ziv Coding of Video: An Error-Resilient Compression Framework"; And Proc . By A. Aaron, S. Rane, D. Rebollo-Monedero, and B. Girod . IEEE Int . Conf . Image Proc . Algorithm in “Systematic Lossy Forward Error Protection for Video Waveforms of Video Waveforms,” in 2003, also uses wrong frames reconstructed by the predictive coder as side information in the auxiliary channel decoder, There are many important differences between the disclosed techniques and the present invention. For example, A. Sehgal, A. Jagmohan, and N. Ahuja in IEEE Trans. on Multimedia , vol. 6, pp. 249-258, 2004, in "Wyner-ziv Coding of Video: An Error-Resilient Compression Framework," authors write "peg""Display some frames as frames. These techniques allow error propagation to occur from one peg frame to the next. In each peg frame, some extra information is sent from the encoder to the decoder, which allows the decoder to correct errors in the peg frame.

In contrast, aspects of the present invention stop or reduce drift, ie errors, as soon as possible. In this way, it becomes possible to maintain a rather stable quality, which is important in visual terms.

Meanwhile, A. Aaron, S. Rane, D. Rebollo-Monedero, and B. Girod's Proc . IEEE Int. Conf . Image Proc . In 2003, "Systematic Lossy Forward Error Protection for Video Waveforms," an independent description of the video sequence is transmitted in each frame, but with coarser quantization. do. This coarser technique can be coded using the Wyner-Ziv framework and decoded with the help of error frames in a decoder that acts as side-information. Thus, the method of the document does not attempt to stop the drift because the method may at best recover poor technology. The techniques described in this document also work in the pixel domain and do not work in the DCT domain, thus giving up taking advantage of spatial interrelationships.

For source coding with side information, it is necessary for the decoder to be able to decode an encoded codeword susceptible to correlation noise N between the source and side-information, while the encoder encodes the source under distortion constraints. It is necessary to do Although the results demonstrated by Wyner and Ziv are inherently asymptotic and non-constructive, many interpretative methods to solve this problem have since been proposed, where the source codebook is interrelated. It is distributed to the corsets of the channel code that match the noise N. The number of distributions or corsets depends on the statistics of N.

1 is a system block diagram illustrating aspects of encoded video for a wireless channel according to one embodiment. The system includes a predictive encoder 10, comprising a frame buffer, which communicates a bit stream to the predictive decoder 20 over a wireless primary channel 25, and an auxiliary channel (WyZER) decoder over a wireless auxiliary channel 45. And an auxiliary channel (WyZER) encoder 30 that communicates the bit stream to 40.

In FIG. 1, predictive decoder 20 receives a bit stream from predictive encoder 10 over a wireless channel 25 and is a representation of a video signal X input to encoder 10. Reconstruct Predictive decoder 20 reconstructed bit stream, May be affected by errors in the transmission of the bit stream over the wireless channel. Reconstructed signal Is the final reconstruction It serves as side information for the WyZER decoder 40 that outputs.

The input video signal X is in one embodiment the current video signal encoded by the predictive encoder 10, which predicts the input video signal and compresses the input video signal and measures the motion estimation (ME) and the displaced frame difference (ME). A compressed bit stream may be transmitted based on techniques, such as conversion of Displaced Frame Difference (DFD). WyZER encoder 30 sends the independent description of X to WyZER decoder 40 using the Wyner-Ziv framework.

WyZER decoder 40 then decodes the bit stream transmitted on the second channel, and reconstructs the signal. Using Final reconstruction of video signal X Outputs Final reconstructed signal As described above, since the WyZER decoder 40 can correct some errors that occur during transmission of the bit stream over the wireless channel, the reconstructed signal More typically, there will be a better reconstruction of the original video signal X. Final reconstruction signal May be written back to the frame buffer of prediction decoder 20.

In this embodiment, for the purpose of designing the WyZER encoder 30, the ratio assigned to the WyZER encoder is fixed and the correlation structure is X = + Z, where X represents the original input signal, Denotes the reconstruction of the original signal X by the predictive decoder, and Z is assumed to represent the correlation noise vector.

Also, in this embodiment, since the operation is performed in the discrete cosine transform (DCT) domain, the components of the correlation noise vector Z are assumed to be independent. The components can be modeled as distributed Gaussian, with Gaussian having maximum entropy and thereby providing worst case analysis, with variance variance with equal differences. WyZER encoder 30 needs to find the number of distributions of the source codebook, i.e., the number of corsets, for each component in X, so that WyZER encoder 30 recognizes the variation of each component of Z. It is necessary to do To find these variations, correlation evaluation techniques can be used as described below.

For example, in one embodiment, the correlation evaluation algorithm is described in R. Zhang, SL Regunathan, and K. Rose's Proc . 33 rd Ann . Asilomar Conf . on Sig . Syst . Comp ., 1999, employs some aspects of the algorithm presented in "Optimal intra / inter mode switching for robust video communication over Internet." However, unlike the above documents, embodiments of the present invention are implemented in the DCT domain. For both primary and secondary channels, packet extinction on both channels is independent and it is assumed that packets can be decoded independently. The probability of packet loss is assumed to be equal to the probability of loss of information for the block. When a block is lost, the prediction decoder replaces the block with a block in frame memory at the same location in the previous frame. The probability of packet loss on the primary and secondary channels is denoted by p and q, respectively.

As described in co-pending US Application No. 10 / 396,982, referred to as "Method, Apparatus and System for Encoding and Decoding Side Information for Multimedia Transmission", "Method, Apparatus and System for Encoding and Decoding Side Information for Multimedia Transmission". Likewise, the WyZER encoder can communicate a corset index to the WyZER decoder. The WyZER decoder then decodes the index to identify a corset containing a codeword that together represent side information. In one embodiment, SS Pradhan and K. Ramchandran, Proc . IEEE DATA Compression The concepts disclosed in Conf ., 1999, "Distributed source coding using syndromes (DISCUS): Design and construction" can be used. Typically, the source codebook may be distributed into corsets of any channel code.

A detailed block diagram of the WyZER encoder is shown in FIG. The WyZER encoder consists of three blocks, a distortion statistics calculator, a corset encoder and a frame statistics calculator. The WyZER encoder uses the predicted reconstruction of the video frame and the current video frame data at the decoder to calculate statistics of correlation noise. These statistics are used by the corset encoder and generate a WyZER bit stream. Frame statistics are updated after WyZER encoding so frame statistics can be used in the next frame.

7 shows a block diagram of a WyZER decoder. WyZER bits consisting of corset bits and corset index information are used with side information (from the main decoder) to generate a WyZER decoded output.

R. Zhang, SL Regunathan, and K. Rose Proc. 33rd Ann. Asilomar Conf. on Sig. Syst. In accordance with the notation of "Optimal intra / inter mode switching for robust video communication over Internet" in Comp., 1999, Is the original value of the i ^th DCT coefficient in the zig-zag scan order in the k ^th block of the n ^th frame. Denotes the encoder reconstruction value, i.e. Is Is a quantized representation of. By predictive decoder This allows these coefficients to be reconstructed. Reconstruction Is any variable for the encoder.

Consider the situation where there is no second channel. In both cases, there are cases where a block is coded intra (I block) or inter (prediction, ie P block). I blocks are self-contained, that is, they contain all the information needed to be or represent one complete block of data. The P block is not self-inclusive and will typically include differential information for the previous frame, such as motion vectors and differential texture information. p is considered the possibility that an information unit is lost due to channel errors. It is also assumed that this area corresponding to the lost information unit is filled by the previously reconstructed, co-located video. Then, if the block is intra-coded,

Formula (1)

If the block is inter-coded,

Formula (2)

Where, Is And the prediction, that is, the best prediction for the k ^th block in the n ^th frame (n-1) is the j ^th block in the ^th frame. When a block is lost, the block is replaced with the block at the same position in the previous frame. Even if the prediction bit stream for a particular block is received without error, Is Is not the same, i.e., the block may nevertheless exist because previous errors exist. The expected distortion for this case is as follows.

Formula (3)

Expected distortion, Can be calculated for all DCT coefficients in the block and the WyZER encoder should encode for this correlation. The goal is to stop drift by improving misreconstructed DCT coefficients into error-free quantized representations. However, it may not be possible to encode all coefficients given the ratio constraint of the second channel. The correct approach here is to use a possible ratio that refines the subset of DCT coefficients that minimizes the expected distortion. However, performing this optimization for every block is very complicated. Based on the off-line test, the best strategy is to improve the lower frequency coefficients before improving the higher frequency coefficients most of the time. This is also meaningful from a visual point of view. Thus, for the on-line algorithm, only the m lowest frequency DCT coefficients that were acceptable under the given rate for the second channel were transmitted. For these m DCT coefficients, the WyZER decoder will recover the encoder reconstruction of the coefficients at the decoder when both the main stream and the WyZER stream arrive. For these m DCT coefficients, the reconstruction at the decoder would be as follows.

Formula (4)

For remaining DCT coefficients that are not encoded by the WyZER encoder, Is given by the above formula (2).

Expected Distortion Using Equation (3) To calculate, And Is already known to the encoder) And It is necessary to calculate. Formulas (1), (2) and (4) are And To And Can be used to relate to This WyZER encoder is described in (R. Zhang, SL Regunathan, and K. Rose, Proc. 33rd Ann. Asilomar Conf. On Sig. Syst. Comp., 1999, "Optimal Intra / Inter for Robust Video Communication over the Internet. Using a simple induction algorithm (similar to the algorithm used by "Optimal intra / inter mode switching for robust video communication over Internet") And Calculate and use equation (3) Can be calculated. For the purpose of encoding, the prediction encoder is careful to divide the frame into a grid of non-overlapping spatial blocks. If the prediction block (j ^th block in (n-1) ^th frame in the analysis) is not aligned with this grid of non-overlapping blocks, then WyZER (because j is not an explicit block in (n-1) ^th frame) While the encoder encodes (n-1) ^th frames And It will not store the values of. In this case, the block with the maximum overlap for the j block in the (n-1) ^th frame, which is a valid block, is used in the calculation. Instead, a weighted average of the corresponding blocks in the (n-1) ^th frame overlapping j ^th blocks can also be used.

Correlation noise estimate ( ) Determines the number of corsets or the number of distributions of the source codebook. This information, the number of corsets, can communicate to the WyZER decoder via the second channel in the form of header information. In order to limit the size of the header, the possible values that the number of corsets can take may be limited for each coefficient. The limitation on possible values may be based on extensive testing.

2 (a)-(f) are graphs of simulation results for various performance comparisons demonstrating the effectiveness of the WyZER codec described above. For the simulation, standard video sequences, known to those skilled in the art, were encoded with an H.263 + coder used as the predictive video codec. The second channel is allocated at a rate of about 30% on the primary channel.

FIG. 2 (a) is a diagram of a peak signal of noise ratio (PSNR) to frame number for a video sequence encoded with the DSC techniques 204 and H.263 + encoder 202 described above. In Figure 2 (a), the third frame of the standard "Flower Garden" video sequence (352x240, 15 fps, one intra frame followed by 29 intra frames) is dropped. For a fair comparison, H.263 + is supplemented with an extra proportion of the same size. Figure 2 (a) shows that with DSC techniques, the PSNR increases steadily following the dropped frame because the drift is corrected, about 7-8 dB better than H.263 + until the end of the 30th frame sequence. .

The aforementioned DSC techniques have also been tested using a wireless channel simulator that adds packet errors to multimedia data streams transmitted over a wireless network conforming to the CDMA 2000 1X Standard 5. For this test, DSC technology has been modified such that the proportion of the portion on the auxiliary channel is assigned to the forward modification code (FEC) and the remaining portion is assigned to the WyZER stream. The reason for this is that, unlike the case where FEC is used, the WyZER stream cannot be recovered immediately from loss (see Figure 2 (a)) and the number of extinctions is less than the correction performance of the FEC. This is important from both visual and PSNR perspectives. However, as mentioned above, since it does not guarantee error recovery, the overall ratio for the side-channel is not assigned to FECs. This is especially true for wireless channels that often have large error bursts.

FIG. 2 (b) is a diagram of PSNR versus frame number for video sequences encoded with the H.263 + encoder 212, FEC 214, and DSC technique 216 described above. The standard video sequence used when generating FIG. 2 (b) is a “Stefan” video sequence (352 × 240, 15 fps, 1 GOP). In FIG. 2 (b), the three different encoding techniques compared are as follows.

1) H.263 + (curve 212) encoded in equal proportions in total assigned to primary and secondary channels

2) the full auxiliary channel (curve 214) assigned to the FEC (using Reed-Solomon code), and

3) The proposed algorithm (curve 216), which is modified by assigning a portion of the ratio to the FEC (RS codes are used again) on the auxiliary channel.

For both FEC and DSC techniques, for (2) and (3) above, the latency constraint is one frame. Tests were performed in the case of (3) to find the proper ratio allocation between the WyZER stream and the FEC. In most cases, it was best to allocate approximately equal proportions to the FEC and WyZER streams, which is the setup for the results presented here. The main and WyZER streams were transmitted on the same (simulated) radio channel.

In the case of the Stefan sequence shown in FIG. 2 (b), the WyZER 216 method presented is 2.9 dB higher than 214 for FEC only and achieves an average PSNR of 5.2 dB higher than baseline H.263 +, Better than others (curve 212).

FIG. 2 (c) is a diagram of PSNR vs. frame number for another video sequence encoded with H.263 + encoder 222, FEC 224, and WyZER technology 226 described above. The standard video sequence used when making Figure 2 (c) is the "Football" video sequence (352x240, 15 fps, 1 GOP). 2 (c) shows the typical performance of the Stefan and Football sequences (352x240, 15 fps, 1 GOP) for a packet loss rate of 8.5%.

In Figure 2 (c), the PSNR of the proposed DSC technique 226 has two large drops, in frames 23 and 26, but recovers quickly in both cases. The average PSNR for the WyZER technique 226 for this test is 3.1 dB higher than 224 for FEC only and 5.6 dB higher than baseline H.263 + (curve 222).

FIG. 2 (d) is a plot of PSNR versus error probability for video sequence “Stefan” encoded with H.263 + encoder 232, FEC 234, and DSC technology 236. 2 (e) is a plot of PSNR vs. error probability for video sequence “Football” encoded with H.263 + encoder 242, FEC 244, and DSC technology 246. As shown in Figures 2 (d) and 2 (e), in the conventional performance of the three methods of encoding, the methods of the present invention are consistently superior to the other two methods. Results are given only for the SIF (352x240) sequence, but a similar aspect is observed for other spatial size video sequences, such as QCIF, CIF.

Experiments have been performed to study the effect of burst length on performance. FIG. 2 (f) is a plot of PSNR versus average burst length for a “Football” video sequence (176 × 144, 15 fps, 30 frames) encoded with FEC 252 and DSC 254. As shown in FIG. 2 (f), if the average burst length is greater than 0.6, the techniques 254 according to the present invention are superior to the case of FEC 252 alone, but if the burst length is less than 0.6, only FEC Worse than the case.

Therefore, simulation results using the above-described techniques show significant progress in performance through conventional error protection schemes such as forward error correction codecs under fair latency constraints.

The DSC techniques described may operate with many wireless devices and wireless standards. Examples of wireless devices that can implement the described DSC techniques include cellular telephones, wireless communicable PCs, personal digital assistants (PDAs), and other wireless devices. Examples of wireless standards in which the described DSC technologies can operate include Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), and Enhanced Data GSM Environment (EDGE). Standards include Enhanced Data GSM Environment (TIA / EIA-95-B (IS-95), TIA / EIA-98-C (IS-98), CDMA2000, Wideband CDMA (WCDMA), and the like.

3 is a block diagram illustrating a portion of a wireless communication system that can implement DSC techniques. As shown in FIG. 1, the wireless communication system includes a first wireless communication device (WCD) 302, also called a mobile station (MS). The wireless communication system includes a second wireless device 304, which may be part of a wireless structure, and a landline device connected to other wireless devices, ie, the wireless structure.

The first wireless device 302 includes a WyZER encoder 306, a WyZER decoder 308, a transmitter 310 and a receiver 312. The second wireless device 304 includes a WyZER encoder 320, a WyZER decoder 322, a transmitter 324 and a receiver 324. The second wireless device 304 may receive an input signal X ₁ , for example a video signal. The input signal X ₁ is connected to the DSC encoder 320. DSC encoder 320 includes prediction encoder 330 and WyZER encoder 332. The input signal is encoded according to the DSC techniques described above, and the main stream and the WyZER stream are output from the predictive encoder 320 and the WyZER encoder 332 to the transmitter 324.

The primary and WyZER streams are sent to the receiver 312 of the first wireless device 302 via a wireless communication channel (multiple logical channels can be used to send two streams). The main stream is output from the receiver 312 to the prediction decoder 334 in the DSC decoder 308 at the first wireless device 302. The WyZER stream is output from the receiver 312 to the WyZER decoder 336. Output of prediction decoder 334 Is also the input of the WyZER decoder 336. According to the DSC techniques described above, the prediction decoder and the WyZER decoder cooperate with each other to reconstruct the signal of the input signal X ₁ . Create and print

In a manner similar to the method described above, the first wireless device receives an input signal X ₂ encoded by the prediction encoder 340 and the WyZER encoder 342 included in the DSC encoder 306 of the first wireless device 302. You may. The main and WyZER streams are output from the DSC encoder 306 to the transmitter 310. The receiver 326 of the second wireless device 304 receives the bit streams and outputs the predicted bit stream from the DSC decoder 322 of the second wireless device 304 to the prediction decoder 344 and outputs the WyZER stream to the DSC decoder ( And output to the WyZER decoder 346 in 322. Output of predictive decoder 344 Is also the input to the WyZER decoder 346. According to the DSC techniques described above, the prediction and WyZER decoders cooperate with each other to reconstruct the input signal X ₂ . Create and print

In another embodiment, the wireless device may include only a DSC encoder or only a DSC decoder. For example, if the device is configured to broadcast, i.e. transmit, but not receive encoded data, the device may have a DSC encoder that encodes the broadcast data but may not have a DSC decoder because the device does not receive the encoded data. have. Similarly, an apparatus configured to receive and decode a signal but not to transmit encoded data may include a decoder but not an encoder. An example of this type of system would be a broadcast system, such as a video broadcast system, where a broadcaster transmits encoded data to one or more users, typically many users, but the users do not transmit the encoded data back to the broadcaster. will be. In this example, the broadcast station wireless device may include a DSC encoder for encoding broadcast data, and the user wireless device may include a DSC decoder to decode the received encoded broadcast.

4 is a flowchart illustrating an embodiment of determining intra frame bit allocation for a WyZER encoder. The flowchart begins at block 402 where the entire data frame is examined, distributed to the data blocks. The flow chart continues at block 404, which determines which data blocks are to be protected. For example, data blocks are " ranked " based on the detrimental effects that loss of an individual data block can bring during the reconstruction of a data frame of a receiver if a particular block is lost or corrupted. The loss of a block is more harmful for reconstruction, the block has a higher rank.

The flowchart continues at block 406 where the amount of data allocated to the WyZER encoder is determined. Typically, it is desirable to allocate the minimum amount of data allocated to the second channel since the amount of data allocated must be transmitted on the second channel and consumes additional system resources. Thus, more data will be sent and allocated on the second channel, depending on how desirable it is to protect a particular block of data. For example, if a data block is " ranked " high, then more data for that block will be sent and allocated to the second channel than the lower " ranked " block.

5 is a flowchart illustrating an embodiment for determining the maximum number of distributions of source codebooks used by the WyZER encoder. The flowchart begins at block 502 where a correlation noise estimate is determined. For example, correlation noise of the transform coefficients may be determined for a case where a predictive bit stream representing a data block is successfully transmitted from a predictive encoder to a predictive decoder. Although the block is successfully transmitted, the decoded data may have errors due to errors in the previous blocks. The flowchart continues to block 504 where the number of distributions of the source codebook is used, which is used to represent the data block. The distribution number of the source codebook is also called a corset.

In one embodiment, the number of corsets may be sent to the WyZER decoder via wireless auxiliary channel communication in the header of the WyZER stream.

Those skilled in the art will appreciate that signals and information may be represented using a variety of other technologies and technologies. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the specification may appear as precursors, currents, electromagnetic waves, magnetic or magnetic particles, photons or photons, or combinations thereof. Can be.

In addition, those skilled in the art may appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or a combination thereof. . To clearly illustrate this alternative possibility of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above primarily in terms of their functionality. Whether such functionality is implemented in hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present invention.

The various illustrative logical blocks, modules, circuits described in connection with the embodiments disclosed herein may be general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate array signals (FPGAs), or Other programmable logic devices, separate gate or transistor logic, separate hardware components, or any combination thereof, designed to perform the functions described herein, may be implemented or performed. A general purpose processor may be a microprocessor, but in other ways, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, software module, or a combination of the two executed by a processor. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, which can read information from and write information to the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside within an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Those skilled in the art will clearly appreciate various modifications to these embodiments, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (63)

A prediction encoder configured to receive an input signal, encode the signal according to a prediction encoding standard, and output a prediction encoded bit stream, and An auxiliary channel encoder configured to receive the input signal, encode the signal according to an auxiliary channel encoding standard, and output an auxiliary channel encoded bit stream; And the auxiliary channel encoder includes a distortion statistics calculator that calculates a correlation noise estimate between current video frame related data and data related to expected reconstruction of the current video frame. The method of claim 1, And the predictive encoded bit stream is transmitted over a primary communication channel. The method of claim 1, The auxiliary channel encoded bit stream is transmitted over an auxiliary channel communication channel. The method of claim 1, And the predictive encoding standard is a standard compatible with the MPEG-4 standard. The method of claim 1, And the predictive encoding standard is a standard compatible with the H.26x standard. The method of claim 1, The auxiliary channel encoding standard is a standard compatible with Wyner-Ziv technologies. The method of claim 1, And the input signal is a video signal. The method of claim 1, The input signal is an intra-coded data block and an inter-coded data block. The method of claim 8, Wyner-Ziv techniques are applied to the inter-coded data block. The method of claim 1, And the auxiliary channel encoder encodes the input signal using a predetermined number of cosets. The method of claim 10, And the predetermined number of corsets is included in a header of the auxiliary channel encoded bit stream. A predictive decoder configured to receive an original signal of an inter-prediction encoded bit stream indication, decode the inter-prediction encoded bit stream according to a prediction decoding standard, and output a reconstructed signal; and Receive the original signal and the reconstructed signal of an auxiliary channel encoded bit stream indication, decode the auxiliary channel encoded bit stream according to an auxiliary channel decoding standard, and generate and output a final reconstruction of the original signal. An auxiliary channel decoder configured to combine a channel encoding bit stream and the reconstructed signal, The auxiliary channel decoder is further configured to receive header information comprising a number of corsets used to decode the auxiliary channel encoding bit stream, And the number of corsets is based on a correlation estimate computed while encoding the auxiliary channel encoding bit stream. The method of claim 12, And the inter-prediction encoded bit stream is received over a primary communication channel. The method of claim 12, And the auxiliary channel encoding bit stream is received over an auxiliary channel communication channel. The method of claim 12, And the predictive decoding standard is a standard compatible with the MPEG-4 standard. The method of claim 12, And the predictive decoding standard is a standard compatible with the H.26x standard. The method of claim 12, The auxiliary channel decoding standard is a standard compatible with Wyner-Ziv technologies. The method of claim 12, The original signal is a video signal. The method of claim 12, The original signal comprises an intra-coded data block and an inter-coded data block. The method of claim 19, Wyner-Ziv techniques are applied to the inter-coded data blocks. The method of claim 12, And the auxiliary channel decoder decodes the auxiliary channel encoding bit stream using a predetermined number of corsets. The method of claim 21, The predetermined number of corsets is included in a header of the auxiliary channel encoding bit stream. As an encoder, A predictive encoder configured to receive an input signal, encode the signal according to a predictive encoding standard, and output a predictive encoded bit stream; and An auxiliary channel encoder configured to receive the input signal, encode the signal according to an auxiliary channel encoding standard, and output an auxiliary channel encoding bit stream; The auxiliary channel encoder includes a distortion statistics calculator that calculates a correlation noise estimate between current video frame related data and data related to expected reconstruction of the current video frame; And As a decoder, A prediction decoder configured to receive a prediction encoding bit stream, decode the prediction encoding bit stream according to the prediction decoding standard, and output a reconstructed signal, and Receive the auxiliary channel encoded bit stream and the reconstructed signal, decode the auxiliary channel encoded bit stream according to an auxiliary channel decoding standard, generate and output a final reconstruction of the original signal, and And a secondary channel decoder configured to combine the reconstructed signal. The method of claim 23, And the predictive decoding standard is a standard compatible with the MPEG-4 standard. The method of claim 23, And the predictive decoding standard is a standard compatible with the H.26x standard. The method of claim 23, The auxiliary channel decoding standard is a standard compatible with Wyner-Ziv technologies. The method of claim 23, And the input signal is a video signal. The method of claim 23, And the wireless communication device is a mobile phone. The method of claim 23, And the wireless communication device is part of a wireless infrastructure. As a first wireless device, A prediction encoder configured to receive an input signal, encode the signal according to a prediction encoding standard, and output a prediction encoding bit stream representation of the input signal; An auxiliary channel encoder configured to receive the input signal, encode the signal according to an auxiliary channel encoding standard, and output an auxiliary channel encoding bit stream representation of the input signal, The auxiliary channel encoder comprises a distortion statistics calculator for calculating a correlation noise estimate between current video frame related data and data related to expected reconstruction of the current video frame; And As a second wireless device, A prediction decoder configured to receive the prediction encoding bit stream output by the prediction encoder, decode the prediction encoding bit stream according to the prediction decoding standard, and output a reconstruction of the input signal; Receive the auxiliary channel encoded bit stream output by the auxiliary channel encoder, decode the auxiliary channel encoded bit stream according to an auxiliary channel decoding standard, and generate and output a final reconstruction of the input signal. And a secondary channel decoder configured to combine a channel encoded bit stream and the reconstructed signal. The method of claim 30, And the predictive decoding standard is a standard compatible with the MPEG-4 standard. The method of claim 30, And the predictive decoding standard is a standard compatible with the H.26x standard. The method of claim 30, The auxiliary channel decoding standard is a standard compatible with Wyner-Ziv technologies. The method of claim 30, And the first wireless device is a mobile phone. The method of claim 30, And the first wireless device is part of a wireless communication infrastructure device. The method of claim 30, And the second wireless device is a mobile phone. The method of claim 30, And the second wireless device is part of a wireless communication infrastructure device. Receiving an input signal, Encoding the signal according to a prediction encoding standard, outputting a prediction encoding bit stream, and Encoding the signal according to an auxiliary channel encoding standard and outputting an auxiliary channel encoding bit stream; Encoding the signal according to the auxiliary channel encoding standard includes calculating a correlation noise estimate between current video frame related data and data related to expected reconstruction of the current video frame. The method of claim 38, And the predictive encoding standard is a standard compatible with the MPEG-4 standard. The method of claim 38, And the predictive encoding standard is a standard compatible with the H.26x standard. The method of claim 38, The auxiliary channel encoding standard is a standard compatible with Wyner-Ziv technologies. The method of claim 38, And the input signal is a video signal. The method of claim 38, Wherein the input signal comprises an intra-coded data block and an inter-coded data block. The method of claim 43, Wyner-Ziv techniques are applied to the inter-coded data blocks. The method of claim 38, The auxiliary channel encoder encodes the input signal using a predetermined number of corsets. The method of claim 45, And the predetermined number of corsets is included in a header of the auxiliary channel encoding bit stream. Receiving an original signal of an inter-prediction encoded bit stream representation; Decoding the inter-prediction encoded bit stream according to a prediction decoding standard and outputting a reconstructed signal; Receiving header information including an original signal of an auxiliary channel encoded bit stream indication, the reconstructed signal and the number of corsets used to decode the auxiliary channel encoded bit stream, wherein the number of the corsets is equal to the auxiliary channel. Receiving based on a correlation estimate calculated during encoding of the encoding bit stream; And Decoding said auxiliary channel encoding bit stream in accordance with an auxiliary channel encoding standard and combining said decoded auxiliary channel encoding bit stream and said reconstructed signal to generate and output a final reconstruction of said original signal. Decoding method. The method of claim 47, And the predictive decoding standard is a standard compatible with the MPEG-4 standard. The method of claim 47, And the predictive decoding standard is a standard compatible with H.26x techniques. The method of claim 47, The auxiliary channel decoding standard is a standard compatible with Wyner-Ziv techniques. The method of claim 47, And the original signal is a video signal. The method of claim 47, The original signal is an intra-coded data block and an inter-coded data block. The method of claim 52, wherein Wyner-Ziv techniques are applied to the inter-coded data block. The method of claim 47, And the auxiliary channel decoder decodes the auxiliary channel encoding bit stream using a predetermined number of corsets. The method of claim 54, wherein And the predetermined number of corsets is included in a header of the auxiliary channel encoding bit stream. delete delete A computer readable medium implementing the data encoding method, The method, Receiving an input signal, Encoding the signal according to a prediction encoding standard, outputting a prediction encoding bit stream, and Encoding said signal according to an auxiliary channel encoding standard, and outputting an auxiliary channel encoding bit stream; Encoding the signal according to the supplemental channel encoding standard includes determining a correlation noise estimate between current video frame related data and data related to expected reconstruction of the current video frame. A computer readable medium implementing the data decoding method, The method, Receiving an original signal of a first bit stream representation, wherein the first bit stream of the original signal is an inter-prediction encoded bit stream; Decoding the first bit stream according to a prediction decoding standard and outputting a reconstructed signal; Receiving header information including an original signal of a second bit stream indication, the reconstructed signal and the number of corsets used to decode the auxiliary channel encoding bit stream, wherein the number of corsets is the second bit; Receiving based on a correlation estimate calculated during encoding of the stream; And Decoding the second bit stream in accordance with an auxiliary channel decoding standard and combining the decoded second bit stream and the reconstructed signal to generate and output a final reconstruction of the original signal. media. The method of claim 59, And the first bit stream is encoded according to a prediction encoding standard. The method of claim 59, And the second bit stream is encoded according to an auxiliary channel encoding standard. Means for receiving an input signal; Means for encoding the signal according to a prediction encoding standard and outputting a prediction encoding bit stream; And Means for encoding said signal according to an auxiliary channel encoding standard and outputting an auxiliary channel encoding bit stream; Encoding the signal according to the auxiliary channel encoding standard includes determining a correlation noise estimate between current video frame related data and data related to expected reconstruction of the current video frame. Means for receiving an original signal of a first bit stream representation, the first bit stream being an inter-prediction encoded bit stream; Means for decoding the first bit stream and output a reconstructed signal in accordance with a predictive decoding standard; Means for receiving header information including an original signal of a second bit stream indication, the reconstructed signal and the number of corsets used to decode the second bit stream, wherein the number of corsets is the second bit stream. Receiving means based on a correlation estimate calculated during encoding; And Means for decoding the second bit stream according to an auxiliary channel encoding standard and combining the decoded second bit stream and the reconstructed signal to generate and output a final reconstruction of the original signal. .