JP5004877B2 - Image encoder, image decoder, image encoding method, image decoding method, and program - Google Patents

Description

  The present invention relates to an image encoder, an image decoder, an image encoding method, an image decoding method, and a program, and in particular, lightweight encoding and multi-view image encoding, that is, encoding each other independently. The present invention relates to an image encoder, an image decoder, an image encoding method, an image decoding method, and a program of a necessary system and a network communication system in which loss such as wireless or Internet occurs.

  In recent years, there has been a shift from analog signal systems to digital signal systems, and the demand for digital images has increased. However, since a digital image has a huge amount of data as it is, an encoding technique for efficiently compressing the image is important, and an international standard algorithm is widely used. As an example, MPEG-2 is used for digital TV, and H.264 / AVC is used for the Blu-ray Disc standard. The specific compression algorithm of these image compression algorithms is such that motion compensation (MC) is first performed on the encoder side, temporal signal redundancy is removed, and then frequency conversion such as discrete cosine transform (DCT) is performed. Thus, entropy coding is performed after the redundancy of the spatial signal is removed. The decoder side performs decoding by performing the reverse process.

  On the other hand, in recent years, a new paradigm video encoding method called Distributed Video Coding (DVC) has attracted attention. DVC is an application of the principle of Distributed Source Coding (DSC) to video coding. As shown in FIG. 8, the DSC is a system in which a plurality of correlated information sources are distributed and encoded without observing each other, and each data is decoded on the receiving side. DSC was originally developed in the 1970s by the Slepian-Wolf theorem established by Slepian and Wolf (for example, see Non-Patent Document 1) and the Wyner-Ziv theorem established by Wyner and Ziv (for example, Non-Patent Document 2). Is based. The Slepian-Wolf theorem gives an allowable compression rate region that can be decoded without distortion when two information sources are distributedly encoded. The Wyner-Ziv theorem distorts one information source in two information sources. A rate distortion region is given in the case of occurrence of From these theorems, it is proved that the compression limit when encoding in a distributed manner is within the range of the conditions of the Slepian-Wolf theorem and the Wyner-Ziv theorem, but is the same as the case without the dispersion. . In recent years, research has been conducted on how close to the limits given by the Slepian-Wolf theorem and Wyner-Ziv theorem using Turbo codes (see, for example, Non-Patent Document 3 and Non-Patent Document 4).

  DVC, which applies DSC theory to video coding, is expected to be applied to lightweight encoders, robust coding, and multi-view video coding because of its dispersibility. For example, a compression algorithm for moving images that has been standardized in recent years has a high compression efficiency, but the amount of computation has also become enormous. H.264 requires a large amount of arithmetic processing, and it is known that the arithmetic processing amount increases by about 5 to 10 times compared to MPEG-2. This large amount of arithmetic processing is necessary for complicated inter-frame prediction, variable macro book size optimization, and the like, and is necessary on the encoder side. On the other hand, even if the arithmetic processing on the decoder side increases, there is a demand for applications such as sensor camera and mobile phone video processing that it is preferable that the processing on the encoder side be less. DVC is an encoding method in which heavy load calculation processing on the encoder side is shifted to the decoder side, and a lightweight encoder can be realized.

  Conventionally, in an encoding method typified by MPEG, a prediction error signal is compressed / decompressed using an information source encoder. In DVC, a prediction signal of a Wyner-Ziv frame is created by creating supplementary information from a Key frame. , The prediction error signal is regarded as a certain type of error, and the channel encoder corrects the error to perform compression / decompression. However, in the creation of auxiliary information from the Key frame, it is necessary to retransmit parity bits and the like when supplementary information is lost, and the coding efficiency also decreases. Therefore, in order to improve coding efficiency in DVC, it is important to create auxiliary information with high prediction accuracy.

  Although there are various methods for configuring DVC, as shown in FIG. 9, a method using a turbo code for encoding and decoding a Wyner-Ziv frame has been proposed (see, for example, Non-Patent Document 5). Furthermore, in order to improve encoding efficiency, a transform area DVC in which the pixel area DVC in the technique of Non-Patent Document 5 described above is extended has been proposed (see Non-Patent Document 6, for example). In this transform area DVC, the spatial redundancy of the signal is removed by using DCT, and the coding efficiency is improved as compared with the pixel area DVC.

A wavelet domain DVC using wavelet transform instead of DCT has also been proposed (see, for example, Non-Patent Document 7). The wavelet region DVC has been reported to be superior in encoding efficiency compared to the pixel region DVC, and has excellent features such as resolution scalability and SNR scalability that the DCT region DVC does not have.
D. Slepian and JK Wolf, "Noiseless coding of correlated information sources," IEEE Trans. Inform. Theory, vol. 19, no.4, pp. 471-480, 1973) A. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder," IEEE Trans. Inform. Theory, vol. 22, no. 1, pp1-19, 1976) J. Bajcsy and P. Mitran, "Coding for the Slepian-Wolf problem with turbo codes," in Proc.IEEE Global Communications Conf., Vol.2, 2001, pp.1400-1404. A. Aaron and B. Girod, "Compression with side information using turbo codes," in Proc. IEEE Data Compression Conf., 2002, pp. 252-2561 B. Girod, A. Aaron, S. Rane and D. Rebollo-Menedero, "Distributed video coding," Proceedings of the IEEE, Special Issue on Video Coding and Delivery, vol. 93, no.1, pp.71-83 , January 2005. A. Aaron, S. Rane, E. Setton and B. Girod, "Transform-domain Wyner-Ziv codec for video," VCIP-2004, San Jose, CA, Jan. 2004. Y. Tonomura, T. Nakachi and T. Fujii, "Distributed video coding using JPEG 2000 coding scheme," IEICE Trans. On Fundamentals, E90-A, pp. 581-589, March 2007.

  As described above, the DVC is expected to be effective as a coding for a best-effort communication system such as multi-view video coding or IP network. However, DVC has a problem that encoding efficiency is still lower than a method such as MPEG which removes redundancy on the encoder side.

  The present invention has been made in view of the above points, and an object thereof is to provide an image encoder, an image decoder, an image encoding method, an image decoding method, and a program capable of realizing high encoding efficiency. To do.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (claim 1) is an image encoder for encoding a moving image frame using a wavelet domain Distributed Video Coding (DVC),
Encoding means 35 for encoding the input key frame;
A DWT 110 that converts the input Wyner-Ziv frame into a wavelet domain;
Quantizing means 120 for quantizing a wavelet domain signal and dividing subbands of the lowest frequency band into bit planes;
An entropy encoding unit 150 that performs entropy encoding of the output from the quantization unit 120 based on the mode switching determination result from the decoder 200;
A Slepian-Wolf encoding unit 140 that performs Wyner-Ziv encoding on the bit plane acquired by the quantization unit 120 based on the mode switching determination result from the decoder or 200;
A demultiplexer 130 that distributes the output from the quantization means 120 to either the entropy encoding means 150 or the Slepian-Wolf encoding means 140 based on the mode determination result obtained from the decoder 200,
Slepian-Wolf encoding means 140
Low frequency band divided into small rectangular blocks when the decoder has auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frames As a result of calculating SAD (Sum of Absolute Differences), which is an index representing the correlation between each decoded subband LL _s and each auxiliary information, the auxiliary information for obtaining the smallest SAD among those SADs is decoded. Means for obtaining and encoding from a container,
The demultiplexer 130 is
When MAD (Mean Absolute Difference) and SAD of the decoded subband LL _s obtained from the decoder 200 are MAD <μSAD (where μ is a constant), the entropy encoding means 150 is instructed to encode, In the case of MAD ≧ μSAD, there is included means for instructing encoding to the Slepian-Wolf encoding means 140 that performs Wyner-Ziv encoding.

The present invention (Claim 2 ) is an image decoder for decoding a moving image frame using a wavelet domain DVC,
Decoding means 44 for decoding the input Key frame;
Auxiliary information generating means 210 for generating auxiliary information which is a subband LL _L of the lowest frequency of the s-th stage by performing s times DWT conversion on the estimated value of the forward prediction and the backward prediction for the Wyner-Ziv frame. ,
Reconstructing means 260 for reconstructing the subband of the lowest frequency band of the Wyner-Ziv frame using auxiliary information;
The subbands LH _s , HL _s , and HH _s in the middle and high frequency bands in the wavelet stage s of the Wyner-Ziv frame are divided into small rectangular blocks, and entropy coding or Wyner-Ziv coding is performed for each small block. Mode determination means 270 for determining whether to perform the determination and sending the mode determination result to the encoder;
Wyner-Ziv subbands high and high frequency bands in the wavelet first s stage frame LH _s, HL _s, the entropy decoding unit 290 for entropy decoding the intra-coded blocks of the subband HH _s in the high frequency band ,
And Slepian-Wolf decoding means 240 for Slepian-Wolf decoding the Wyner-Ziv encoded block,
Slepian-Wolf decoding means 240
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frame, the decoded sub-band of the low frequency band divided into rectangular small blocks As a result of calculating SAD, which is an index indicating the correlation between the band LL _s and each auxiliary information, including means for performing decoding using auxiliary information from which SAD having the smallest value among those SADs is obtained,
The mode determination means 270
If the MAD and SAD of the decoded subband LL _s are MAD <μSAD (where μ is a constant), entropy coding is performed. If MAD ≧ μSAD, Wyner-Ziv coding is determined. Means to do.

The present invention (claim 3) is an image encoder for encoding a moving image frame using a wavelet domain DVC,
Encoding means for encoding the input Key frame;
DWT (Discrete Wavelet Transform) that transforms the input Wyner-Ziv frame into the wavelet domain,
Quantizing means for quantizing the wavelet domain signal and dividing the subband of the lowest frequency band into bit planes;
Entropy encoding means for performing entropy encoding of the output from the quantization means based on the mode switching determination result from the decoder;
Based on the mode switching determination result from the decoder, the Slepian-Wolf encoding means for performing Wyner-Ziv encoding on the bit plane acquired by the quantization means,
A demultiplexer that distributes the output from the quantization means to either the entropy encoding means or the Slepian-Wolf encoding means based on the mode determination result obtained from the decoder,
Slepian-Wolf encoding means
Low frequency band divided into small rectangular blocks when the decoder has auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frames As a result of calculating MSEs of the decoded subbands LL _s and the respective auxiliary information, auxiliary information from which the smallest MSE (Mean Square Error) among those MSEs is obtained is obtained from the decoder and encoded. Including means,
The demultiplexer
If the variance σ and MSE of the decoded subband LL _{s in} the low frequency band obtained from the decoder are σ <μMSE (where μ is a constant), the entropy coding means is instructed to encode, and σ ≧ In the case of μMSE, it includes means for instructing encoding to the Slepian-Wolf encoding means for performing Wyner-Ziv encoding.

The present invention (Claim 4 ) is an image decoder for decoding a moving image frame using a wavelet domain DVC,
Decryption means for decrypting the input Key frame;
For the Wyner-Ziv frame, auxiliary information generating means for generating auxiliary information that is the subband LL _L of the lowest frequency of the s-stage by performing s times DWT conversion on the estimated value by the forward prediction and the backward prediction in both directions ,
Reconstruction means for reconstructing the subband of the lowest frequency band of the Wyner-Ziv frame using auxiliary information;
The middle and high frequency band subbands LH _s and HL _s and the high frequency band subband HH _s in the wavelet stage s of the Wyner-Ziv frame are each divided into rectangular small blocks, and entropy coding is performed for each small block, or Mode determination means for determining whether to perform Wyner-Ziv encoding and sending a mode determination result to the encoder;
Wyner-Ziv subbands high and high frequency bands in the wavelet first s stage frame LH _s, and HL _s, entropy decoding unit entropy decodes the coded blocks in the frame of the sub-band HH _s in the high frequency band,
And Slepian-Wolf decoding means for Slepian-Wolf decoding the Wyner-Ziv encoded block,
Slepian-Wolf decoding means
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frame, the decoded sub-band of the low frequency band divided into rectangular small blocks As a result of calculating the MSE of the band LL _s and each auxiliary information, including means for decoding using the auxiliary information that can obtain the smallest MSE among those MSEs,
The mode judgment means is
Entropy coding is performed when the variance σ and MSE of the decoded subband LL _{s in} the low frequency band is σ <μMSE (where μ is a constant), and Wyner-Ziv coding when σ ≧ μMSE. Means for determining that the operation is to be performed.

  FIG. 2 is a diagram for explaining the principle of the present invention.

The present invention (Claim 5 ) is an image encoding method for encoding a moving image frame using a wavelet domain DVC,
An encoding step (step 1) in which the encoding means encodes the input key frame;
A wavelet transform step (step 2) in which the DWT transforms the input Wyner-Ziv frame into a wavelet domain;
A quantization step (step 3) in which the quantization means quantizes the wavelet domain signal and divides the subband of the lowest frequency band into bit planes;
A distribution step (steps 4 and 5) in which the demultiplexer distributes the output from the quantization unit to either the entropy encoding unit or the Slepian-Wolf encoding unit based on the mode determination result from the decoder;
An entropy encoding means for performing entropy encoding of the output from the quantization means (step 6);
The Slepian-Wolf encoding means performs a Slepian-Wolf encoding step (Step 7) for performing Wyner-Ziv encoding on the bit plane obtained by the quantization means based on the mode switching determination result from the decoder. ,
In the sorting step (step 5),
Low frequency band divided into small rectangular blocks when the decoder has auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frames get the decoded subband LL _s of SAD is an index representing a correlation of the respective auxiliary information, MAD and the SAD of the decoded sub-band LL is, MAD <μSAD (where, mu is a constant) in the case of Instructing encoding to the entropy encoding means, and if MAD ≧ μSAD, instructing encoding to the Slepian-Wolf encoding means for performing Wyner-Ziv encoding ,
In the Slepian-Wolf encoding step (Step 7),
Auxiliary information for obtaining the SAD having the smallest value among the SADs is obtained from the decoder and encoded .

The present invention (Claim 6) is an image decoding method for decoding a moving image frame using a wavelet domain DVC,
A decoding step in which the decoding means decodes the input Key frame;
Auxiliary information generation means generates auxiliary information that is the subband LL _L of the lowest frequency of the s-th stage by performing DWT transformation on the estimated value of the forward prediction and the backward prediction for the Wyner-Ziv frame s times. An auxiliary information generation step;
A reconstruction means for reconstructing the subband of the lowest frequency band of the Wyner-Ziv frame using auxiliary information;
The mode judging means divides the sub-bands LH _s and HL _s in the medium and high frequency band and the sub-band HH _s in the high frequency band in the wavelet stage s of the Wyner-Ziv frame into small rectangular blocks. A mode determination step of determining whether to perform entropy encoding or Wyner-Ziv encoding and sending a mode determination result to the encoder;
Entropy decoding means, to high and high frequency band of the subband LH _s, HL _s, entropy decoding the intra-coded blocks of the subband HH _s in the high frequency band in wavelet first s stage of Wyner-Ziv frames An entropy decoding step;
Slepian-Wolf decoding means performs a Slepian-Wolf decoding step of Slepian-Wolf decoding a Wyner-Ziv encoded block;
In the mode determination step,
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frame, the decoded sub-band of the low frequency band divided into rectangular small blocks get the band LL _s and SAD is an index representing a correlation of the respective auxiliary information, MAD and the SAD of the decoded subband LL _s is the case of MAD <μSAD (where, mu is a constant), entropy coding If MAD ≧ μSAD, it is determined that Wyner-Ziv encoding is performed ,
In the Slepian-Wolf decoding step,
Decoding is performed using auxiliary information that can obtain the SAD having the smallest value among the SADs .

The present invention (Claim 7 ) is an image encoding method for encoding a moving image frame using a wavelet domain DVC,
An encoding step in which the encoding means encodes the input Key frame;
A wavelet transform step in which DWT transforms the input Wyner-Ziv frame into a wavelet domain;
A quantization means for quantizing a signal in the wavelet domain and dividing a subband of the lowest frequency band into bit planes;
An entropy encoding means for performing entropy encoding of the output from the quantization means based on the mode switching determination result from the decoder;
Slepian-Wolf decoding means, based on the mode switching determination result from the decoder, Slepian-Wolf encoding step for performing Wyner-Ziv encoding on the bit plane obtained by the quantization means;
A demultiplexer performs a distribution step of distributing the output from the quantization means to either the entropy encoding means or the Slepian-Wolf encoding means based on the mode determination result obtained from the decoder;
In the sorting step,
When the decoder has auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frames, it is divided into rectangular small blocks from the decoder. low frequency band of the decoded subband LL _s and each MSE of auxiliary information to get the result of the calculation, the variance sigma and the MSE of the decoded subband LL _s is, σ <μMSE (where, mu is a constant and ) Instruct the encoding to the entropy encoding means, and in the case of σ ≧ μMSE, instruct the encoding to the Slepian-Wolf encoding means to perform Wyner-Ziv encoding,
In the Slepian-Wolf encoding step,
Auxiliary information that provides the smallest MSE (Mean Square Error) among the MSEs is obtained from the decoder and encoded .

The present invention (Claim 8 ) is an image decoding method for decoding a moving image frame using a wavelet domain DVC,
A decoding step in which the decoding means decodes the input Key frame;
Auxiliary information generation means generates auxiliary information that is the subband LL _L of the lowest frequency of the s-th stage by performing DWT transformation on the estimated value of the forward prediction and the backward prediction for the Wyner-Ziv frame s times. An auxiliary information generation step;
A reconstruction means for reconstructing the subband of the lowest frequency band of the Wyner-Ziv frame using auxiliary information;
The mode judging means divides the sub-bands LH _s and HL _s in the medium and high frequency band and the sub-band HH _s in the high frequency band in the wavelet stage s of the Wyner-Ziv frame into small rectangular blocks. A mode determination step of determining whether to perform entropy encoding or Wyner-Ziv encoding and sending a mode determination result to the encoder;
Entropy decoding means, to high and high frequency band of the subband LH _s, HL _s, entropy decoding the intra-coded blocks of the subband HH _s in the high frequency band in wavelet first s stage of Wyner-Ziv frames An entropy decoding step;
Slepian-Wolf decoding means performs a Slepian-Wolf decoding step of Slepian-Wolf decoding a Wyner-Ziv encoded block ;
In the mode determination step,
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction, and bidirectional prediction of Wyner-Ziv frame, the decoded sub-band of the low frequency band divided into rectangular small blocks Gets the results of calculating the MSE band LL _s and each of the auxiliary information, the variance σ and the MSE of the decoded subband LL _s is the case σ of <μMSE (where, mu is a constant), entropy coding When σ ≧ μMSE, it is determined that Wyner-Ziv encoding is performed ,
In the Slepian-Wolf decoding step,
Decodes using auxiliary information that gives the smallest MSE among MSEs,

The present invention (Claim 9 ) is an image encoding program for causing a computer to function as each means constituting the encoder according to Claim 1 or 3 .

The present invention (Claim 10) is an image decoding program for causing a computer to function as means constituting the decoder according to Claim 2 or 4 .

  As described above, according to the present invention, in Distributed Video Coding (DVC) in which each information source is distributedly encoded with respect to a plurality of correlated information sources (such as multi-viewpoint images) and collectively decoded on the receiving side, Of the Key frame and Wyner-Ziv frame that composes the video frame, when encoding the Wyner-Ziv frame, the sub-band of the middle and high frequency band is divided into rectangular blocks, and each of them has two indices Mean Absolute Difference ( MAD), Sun of Absolute Difference (SAD) is used to determine whether entropy encoding or Slepian-Wolf encoding is to be performed, and the accuracy of the prediction value of the Wyner-Ziv frame is improved by sending it to the transmission side At the same time, it is possible to realize high encoding efficiency of Distributed Video Coding (DVC).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, the operation principle of the pixel area DVC and the conventional wavelet area DVC will be described.

[Pixel area DVC]
In general, in DVC, a video frame is divided into a Key frame and a Wyner-Ziv frame. Here, for simplicity, assign the odd frame X _{2n ± 1} for Key frames, it allocates the even frame X _2n are the Wyner-Ziv frames.

Hereinafter, description will be made using the configuration of the pixel region DVC of FIG. 9 described above. This figure is an example of the pixel region DVC proposed in Non-Patent Document 5 described above. The Key frame is encoded by a conventional intraframe encoder 11 such as JPEG. The Wyner-Ziv frame X _2n is uniformly quantized by the quantizer 12, and the quantized signal is divided into bit planes and then sent to the Splepian-Wolf encoder 13. Each bit plane is sent from the most significant bit (MSB) to the Slepian-Wolf encoder 13, and sufficient syndrome bits are generated for each bit plane. Each bit plane is stored in the buffer 132 in order, and syndrome bits are transmitted in order from the upper bit plane in response to a request from the decoder side.

On the other hand, on the decoder side, block unit auxiliary information (Side Information) Y _2n is created from the Key frame X _{2n ± 1} using motion compensation by the following equation (1).

但し、式中MVはKeyフレームから作成された動きベクトルである。

In the expression, MV is a motion vector created from the Key frame.

Next, the bit likelihood is estimated, and the bit likelihood is given to the Slepian-Wolf decoder 133. The bit likelihood is generally estimated using the following equation since the difference signal between the wavelet regions of the Wyner-Ziv frame X _2n and the auxiliary information Y _2n , that is, the prediction error signal is approximated by a Laplace distribution.

但し、αはラプラス分布のパラメータである。Slepian-Wolf復号器１３３では、ビット尤度とエンコーダ１３１から受信するシンドロームビットを用いて量子化されたビットプレーンq'_2nを復号する。

Where α is a parameter of the Laplace distribution. The Slepian-Wolf decoder 133 decodes the bit plane q ′ _2n quantized using the bit likelihood and the syndrome bits received from the encoder 131.

Finally, the decoded bit plane is sent to the reconstructor 24 and reconstructed by the following equation using the auxiliary information Y _2n .

この画素領域DVCは、以後、数多く発明されるDVCの原型となった。

This pixel region DVC became the prototype of many DVCs that have been invented thereafter.

[Wavelet domain DVC]
Next, the wavelet area DVC will be described.

  In order to improve the encoding efficiency, Girod et al. Have proposed a transform area DVC that is an extension of the pixel area DVC of the technique shown in Non-Patent Document 5 (for example, Non-Patent Document 6). In this transform area DVC, the spatial redundancy of the signal is removed by using DCT, and the coding efficiency is improved as compared with the pixel area DVC. A wavelet domain DVC using wavelet transform instead of DCT has also been proposed (for example, Non-Patent Document 7). Here, the wavelet area DVC will be described. FIG. 3 shows the principle configuration of the wavelet domain DVC. The difference from the pixel area DVC is that error correction by the channel encoder performed in the pixel area is performed in the wavelet area.

In the wavelet domain DVC, the Wyner-Ziv frame X _2n is converted into the wavelet domain by the discrete wavelet transform DWT 31, and then the signal of each subband is 2 ^M ≦ {0,4,8,16,32 by the quantizer 32. , 64, 128, 256, ...} are uniformly quantized. The quantized signal is divided into bit planes by the sub bit plane dividing unit 33 and then sent to the Slepian-Wolf encoder 34 for each sub band.

On the other hand, on the decoder side, a predicted value Y _2n is created for each block from the Key frame X _{2n ± 1} using motion compensation. Here, the motion compensation process uses the same technique as that of the pixel region DVC.

The predicted value Y _2n is used as auxiliary information after being converted into a wavelet region by the discrete wavelet transform DWT 46. Next, the bit likelihood is estimated and the bit likelihood is given to the Slepian-Wolf decoder 41. The bit likelihood is generally approximated by a difference signal between the wavelet regions of the Wyner-Ziv frame X _2n and the predicted value Y _2n , that is, a Laplace distribution, and is estimated by a method similar to that for the pixel region DVC. The decoded bit plane is sent to the reconstructor 42 and reconstructed using the auxiliary information. The reconstructed signal is finally subjected to inverse discrete wavelet transform IDWT in the IDWT 43 and output.

  The wavelet region DVC of the present invention will be described below.

  Regardless of whether the pixel area DVC or the wavelet area DVC, the encoding efficiency of the DVC depends largely on the prediction system of the supplementary information generated by the decoder. In DVC, supplementary information created from a Key frame is used as a Wyner-Ziv frame prediction signal, and the auxiliary information and the Wyner-Ziv frame prediction error signal are regarded as a certain type of error, and the channel encoder corrects the error. Compress / decode with. If the supplementary information is lost, it is necessary to retransmit the parity bit and the like, and the coding efficiency also decreases. Therefore, it is necessary to provide auxiliary information with high prediction accuracy. In the generation of the auxiliary information, in the conventional conversion area DVC such as the pixel area DVC, the wavelet area DVC, and the DCT area DVC (for example, Non-Patent Document 6), the auxiliary information is generated only by information of the Key frame. . This is due to the fact that there is no Wyner-Ziv frame information in the decoder.

  In the present invention, in order to provide auxiliary information with higher estimation accuracy, the decoded subband of the low frequency band is used in the process of decoding the signal in the wavelet domain from the subband of the low frequency band toward the high frequency band. A method for providing auxiliary information with higher estimation accuracy will be described. If it is determined that the auxiliary information estimation system is poor, entropy coding such as Huffman coding or arithmetic coding is used.

  FIG. 4 shows the configuration of the wavelet region DVC in one embodiment of the present invention.

  The encoder 100 shown in the figure includes a DWT 110, a quantizer 120, a demultiplexer 130, a Slepian-Wolf encoder 140, and an entropy encoder 150. The decoder 200 includes an auxiliary information generator 210, a DWT 220, a Slepian-Wolf decoder 240, a multiplexer 250, a reconstructor 260, a mode determiner 270, an IDWT 280, and an entropy decoder 290.

  The difference between the DVC shown in the figure and the conventional DVC wavelet domain DVC is the addition of a function that determines whether entropy encoding or Slepian-Wolf encoding is used when encoding a Wyner-Ziv frame signal. And a mode determination unit 270 having a function of selecting auxiliary information determined to have high prediction accuracy from among a plurality of auxiliary information even when it is determined as Slepian-Wolf. It is. The specific operation procedure is shown below.

  FIG. 6 is a sequence chart of encoding / decoding according to an embodiment of the present invention.

Step 101) On the encoder 200 side, the input Key frame X _{2n ± 1} is encoded using a conventional intraframe encoder such as JPEG2000.

Step 102) On the encoder 200 side, the input Wyner-Ziv frame X _2n is converted into a wavelet domain by the discrete wavelet transform DWT110. FIG. 5 shows an example of a two-level wavelet transform. Thereafter, each wavelet domain signal is uniformly quantized by the quantizer 120 to a level of 2 ^M ≦ (0, 4, 8, 16, 32, 64, 128, 256,...).

Step 103) On the decoder 200 side, the encoded Key frame X _{2n ± 1} input from the encoder 100 is decoded using an intra-frame decoder.

Step 104) On the decoder 200 side, the auxiliary information generator 210 creates auxiliary information of the Wyner-Ziv frame. Therefore, the estimated value of the Wyner-Ziv frame signal X _{2n in} units of blocks using motion compensation interpolation, average interpolation, or extrapolation shown in the technique described in Non-Patent Document 6 above.

を作成する。次式は式（１）に示す動きベクトルMVを用いて作成した推定値の例である。

Create The following expression is an example of an estimated value created using the motion vector MV shown in Expression (1).

それぞれＸ_2n-1，Ｘ_2n+1並びに両信号を用いて作成しており、前向き予測、後ろ向き予測、両方向予測と呼ぶ。

They are created using X _2n-1 , X _{2n + 1 and} both signals, respectively, and are called forward prediction, backward prediction and bidirectional prediction.

  Next, in DWT220,

に、それぞれＬレベルの離散時間ウェーブレット変換

Respectively, L-level discrete-time wavelet transform

を施す。それぞれの変換後の第ｓステージにおけるサブバンド

Apply. Subband in the sth stage after each conversion

の値を、

The value of

と定義する。これらは、補助情報として用いられる。

It is defined as These are used as auxiliary information.

Step 105) In the encoder 100, for the Wyner-Ziv frame, the subband LL _L of the lowest frequency band that is uniformly quantized by the quantizer 120 in Step 102 is divided into bit planes by the demultiplexer 130. Then, it is sent to the Slepian-Wolf encoder 140.

Step 106) In the decoder 200, for the subband LL _L of the lowest frequency band of the Wyner-ziv frame,

を補助情報として、再構成器２６０でLL_Lを再構成する。このLL_Lが再構成されるまでのSlepian-Wolf符号器１４０／Slepian-Wolf復号器２４０の処理は、従来のウェーブレット領域DVCと同じである。ここで、再構成された信号をLL'_Lと定義する。s＝Lとする。

As the auxiliary information, to reconstruct the LL _L at reconstructor 260. Processing Slepian-Wolf encoder 140 / Slepian-Wolf decoder 240 until the LL _L is reconstructed is the same as the conventional wavelet domain DVC. Here, the reconstructed signal is defined as LL ′ _L. Let s = L.

Step 107) In the decoder 200, the subbands LH _s , HL _s , and HH _{s in} the middle and high frequency bands of the Wyner-Ziv frame are divided into rectangular small blocks, and entropy coding or Wyner− is performed for each small block. Perform Ziv encoding. The mode switching unit 270 determines whether to perform entropy coding or Wyner-Ziv coding mode switching by the low-frequency band decoded subband LL _{s ′} , the auxiliary information generator 210, and the DWT 220. Auxiliary information

を用いて、以下に示す２つの指標により行う。

The following two indices are used.

Mode switching in the mode determiner 270 is performed for each block, which uses band correlation of wavelet transform, and a small block with a subband LL _s is a subband of an adjacent Key frame. a higher correlation with small blocks LL _s (a low), Wyner-Ziv frames subbands LH _s, HL _s, HH _s also spatially corresponding small block of subband LH _s adjacent Key frame, It uses the property of high (low) correlation between small blocks corresponding to HL _s and HH _s in space. FIG. 7 shows the spatial correspondence of small blocks.

(A) MAD (Mean Absolute Difference)
It is an index representing the spatial smoothness of the signal in the block B of the decoded subband LL _'s, is calculated by the following equation.

ここで、LL'_s（ｘ，ｙ）は、復号済みサブバンドで、（ｘ，ｙ）は小ブロック内の座標を表している。但し、

Here, LL ′ _s (x, y) is a decoded subband, and (x, y) represents the coordinates in the small block. However,

であり、Ｎは小ブロック内の信号の数を表す。

N represents the number of signals in the small block.

(B) SAD (Sum of Absolute differences)
The second index is an index representing the correlation between the decoded subband LL's and the auxiliary information, and is calculated by the following equation.

式（１１）は、前向き、後向き、両方向予測を基に生成された補助情報の中で、最も良い推定値を与える補助情報を判定する指標となる。

Expression (11) serves as an index for determining auxiliary information that gives the best estimated value among auxiliary information generated based on forward, backward, and bidirectional prediction.

  In the mode determiner 270, the entropy encoder 150 performs the entropy encoding using the two indexes of the above formulas (10) and (11), or the Slepian-Wolf encoder 140 performs the Slepian-Wolf encoding. The determination of whether or not to perform is performed as follows.

  (A) If MAD <μSAD (where μ is a constant), entropy coding is performed.

(B) When mad ≧ Μsad, Slepian-Wolf encoding is performed. When Slepian-Wolf encoding is performed, auxiliary information that minimizes Equation (11) is used as auxiliary information. For example, when SAD = SAD _f , as auxiliary information of LH _s , HL _s , HH _s , respectively

を用いる。

Is used.

  The obtained mode switching determination result is sent to the encoder 100 through the feedback channel.

Step 108) In the encoder 100, for the high frequency band subbands LH _s , HL _s and HH _s in the s-th stage of the Wyner-Ziv frame, based on the mode switching determination result sent from the decoder 200, Entropy encoding by the entropy encoder 150 or Slepian-Wolf encoding by the Slepian-Wolf encoder 140 is performed for each block. As entropy coding, Huffman coding or arithmetic coding is used. LDPC or turbo coding is used as Slepian-Wolf coding.

Step 109) In the decoder 200, for the subbands LH _s , HL _s , and HH _s in the high frequency band in the s-th stage of the Wyner-Ziv frame, the intra-coded block is subjected to entropy decoding in the entropy decoder 290. . The Slypian-Wolf decoder 240 executes a Slepian-Wolf decoding process on the Wyner-Ziv encoded block. This Slepian-Wolf decoding process uses the same method as the conventional wavelet domain DVC. The above decoding process is performed on all the small blocks in the sth stage.

  Step 110) The IDWT 280 performs one level inverse discrete wavelet transform. Let s = s−1.

  Step 111) Repeat the processing in step 110 and subsequent steps until the original image is decoded.

  Note that the operation of each component of the encoder and decoder shown in FIG. 4 can be constructed as a program, and can be installed and executed in a computer used as the encoder and decoder.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a system for encoding / decoding a moving image frame using a wavelet region.

It is a principle block diagram of this invention. It is a figure for demonstrating the principle of this invention. It is a block diagram of the wavelet area | region DVC. It is a block diagram of the wavelet area | region DVC in one embodiment of this invention. It is a sequence chart of encoding and decoding in one embodiment of the present invention. It is a figure which shows 2 level wavelet transformation in one embodiment of this invention. It is mode determination based on LL _s subband in the s-th stage in one embodiment of the present invention. It is a block diagram of Distributed Source Coding. It is a block diagram of pixel region DVC.

Explanation of symbols

11 Conventional Intra Frame Encoder 12 Quantizer 13 Slepian-Wolf Encoder 21 Conventional Intra Frame Decoder 22 Description or Interpolator 24 Reconstructor 31 DWT
32 Quantization unit 33 Subband bit plane division unit 34 Slepian-Wolf encoder 35 Encoding means 41 Slepian-Wolf decoder 42 Reconstructor 43 IDWT
44 Decoding means 45 Auxiliary information generator 46 DWT
100 Encoder 110 DWT
120 Quantizer 130 Demultiplexer 131 Turbo Encoder 132 Buffer 133 Turbo Decoder (Slepian-Wolf Decoder)
140 Slepian-Wolf encoding means, Slepian-Wolf encoder 150 Entropy encoding means, entropy encoder 200 Decoder 210 Auxiliary information generating means, Auxiliary information generator 220 DWT
240 Slepian-Wolf decoding means, Slepian-Wolf decoder 250 Multiplexer 260 Reconstruction means, reconstructor 270 Mode determination means, Mode determiner 280 IDWT
290 Entropy Decoder

Claims (10)

An image encoder for encoding a moving image frame using a wavelet domain Distributed Video Coding (DVC),
Encoding means for encoding the input Key frame;
DWT (Discrete Wavelet Transform) that transforms the input Wyner-Ziv frame into the wavelet domain,
Quantizing means for quantizing the wavelet domain signal and dividing the subband of the lowest frequency band into bit planes;
Entropy encoding means for performing entropy encoding of the output from the quantization means based on the mode switching determination result from the decoder;
Based on the mode switching determination result from the decoder, Slepian-Wolf encoding means for performing Wyner-Ziv encoding on the bit plane acquired by the quantization means;
A demultiplexer that distributes the output from the quantization means to either the entropy encoding means or the Slepian-Wolf encoding means based on the mode determination result obtained from the decoder,
The Slepian-Wolf encoding means is
When the decoder has auxiliary information that is four subbands obtained by performing discrete wavelet transform on forward prediction, backward prediction, and bidirectional prediction of the Wyner-Ziv frame, As a result of calculating SAD (Sum of Absolute Differences), which is an index indicating the correlation between the decoded subband LL _{s of the} frequency band and each auxiliary information, auxiliary information that gives the smallest value of SAD among those SADs Means for obtaining and encoding from the decoder;
The demultiplexer
Obtained from the decoder, the SAD and MAD (Mean Absolute Difference) of the decoded subband LL _s is the case of MAD <μSAD (where, mu is a constant), instructs the encoding to the entropy coding means In the case of MAD ≧ μSAD, the image encoder includes means for instructing encoding to the Slepian-Wolf encoding means for performing Wyner-Ziv encoding. An image decoder for decoding a moving image frame using a wavelet domain DVC,
Decryption means for decrypting the input Key frame;
For the Wyner-Ziv frame, auxiliary information generating means for generating auxiliary information that is the subband LL _L of the lowest frequency of the s-stage by performing s times DWT conversion on the estimated value by the forward prediction and the backward prediction in both directions ,
Reconstructing means for reconstructing the subband of the lowest frequency band of the Wyner-Ziv frame using the auxiliary information;
The subbands LH _s , HL _s , and HH _s in the middle and high frequency bands in the wavelet stage s of the Wyner-Ziv frame are divided into small rectangular blocks, and entropy coding or Wyner-Ziv coding is performed for each small block. Mode determination means for determining whether to perform the mode determination, and sending the mode determination result to the encoder;
The Wyner-Ziv subbands high and high frequency bands in the wavelet first s stage frame LH _s, and HL _s, entropy decoding unit entropy decodes the coded blocks in the frame of the sub-band HH _s in the high frequency band ,
And Slepian-Wolf decoding means for Slepian-Wolf decoding the Wyner-Ziv encoded block,
The Slepian-Wolf decoding means is:
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction and bi-directional prediction of the Wyner-Ziv frame, the low frequency band divided into small rectangular blocks has been decoded. As a result of calculating SAD, which is an index indicating the correlation between subband LL _s and each auxiliary information, including means for decoding using auxiliary information from which SAD having the smallest value among those SADs is obtained,
The mode determination means includes
When the MAD and the SAD of the decoded subband LL _s are MAD <μSAD (where μ is a constant), entropy coding is performed, and when MAD ≧ μSAD, Wyner-Ziv coding is performed. picture decoder characterized in that it comprises means for determining the. An image encoder for encoding a moving image frame using a wavelet domain DVC,
Encoding means for encoding the input Key frame;
DWT (Discrete Wavelet Transform) that transforms the input Wyner-Ziv frame into the wavelet domain,
Quantizing means for quantizing the wavelet domain signal and dividing the subband of the lowest frequency band into bit planes;
Entropy encoding means for performing entropy encoding of the output from the quantization means based on the mode switching determination result from the decoder;
Based on the mode switching determination result from the decoder, Slepian-Wolf encoding means for performing Wyner-Ziv encoding on the bit plane acquired by the quantization means;
A demultiplexer that distributes the output from the quantization means to either the entropy encoding means or the Slepian-Wolf encoding means based on the mode determination result obtained from the decoder,
The Slepian-Wolf encoding means is
When the decoder has auxiliary information that is four subbands obtained by performing discrete wavelet transform on forward prediction, backward prediction, and bidirectional prediction of the Wyner-Ziv frame, As a result of calculating the decoded subband LL _{s of the} frequency band and the MSE of each auxiliary information, auxiliary information that obtains the smallest MSE (Mean Square Error) among those MSEs is obtained from the decoder and encoded. Including means for performing
The demultiplexer
If the variance σ of the decoded subband LL _{s in} the low frequency band obtained from the decoder and the MSE are σ <μMSE (where μ is a constant), the entropy encoding means is instructed to perform encoding. , Σ ≧ μMSE, an image encoder comprising means for instructing encoding to the Slepian-Wolf encoding means for performing Wyner-Ziv encoding. An image decoder for decoding a moving image frame using a wavelet domain DVC,
Decryption means for decrypting the input Key frame;
For the Wyner-Ziv frame, auxiliary information generating means for generating auxiliary information that is the subband LL _L of the lowest frequency of the s-stage by performing s times DWT conversion on the estimated value by the forward prediction and the backward prediction in both directions ,
Reconstructing means for reconstructing the subband of the lowest frequency band of the Wyner-Ziv frame using the auxiliary information;
The middle and high frequency band subbands LH _s and HL _s and the high frequency band subband HH _s in the wavelet stage s of the Wyner-Ziv frame are each divided into rectangular small blocks, and entropy coding or , Mode determination means for determining whether to perform Wyner-Ziv encoding and sending a mode determination result to the encoder;
The Wyner-Ziv subbands high and high frequency bands in the wavelet first s stage frame LH _s, and HL _s, entropy decoding unit entropy decodes the coded blocks in the frame of the sub-band HH _s in the high frequency band ,
And Slepian-Wolf decoding means for Slepian-Wolf decoding the Wyner-Ziv encoded block,
The Slepian-Wolf decoding means is:
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction and bi-directional prediction of the Wyner-Ziv frame, the low frequency band divided into small rectangular blocks has been decoded. As a result of calculating the MSE of the subband LL _s and each auxiliary information, including means for decoding using the auxiliary information that can obtain the smallest MSE among those MSEs,
The mode determination means includes
If the variance σ of the decoded subband LL _{s in} the low frequency band and the MSE are σ <μMSE (where μ is a constant), entropy coding is performed, and if σ ≧ μMSE, Wyner-Ziv picture decoder characterized in that it comprises means for determining as to perform encoding. An image encoding method for encoding a moving image frame using a wavelet domain DVC,
An encoding step in which the encoding means encodes the input Key frame;
A wavelet transform step in which DWT transforms the input Wyner-Ziv frame into a wavelet domain;
A quantization means for quantizing the signal in the wavelet domain and dividing a subband of the lowest frequency band into bit planes;
A demultiplexer that distributes the output from the quantization means to either the entropy encoding means or the Slepian-Wolf encoding means based on the mode determination result from the decoder;
An entropy encoding means for performing entropy encoding of the output from the quantization means;
Slepian-Wolf encoding means performs a Slepian-Wolf encoding step of performing Wyner-Ziv encoding on the bit plane acquired by the quantization means based on the mode switching determination result from the decoder. ,
In the sorting step,
When the decoder has auxiliary information that is four subbands obtained by performing discrete wavelet transform on forward prediction, backward prediction, and bidirectional prediction of the Wyner-Ziv frame, get the SAD is an index representing a correlation of the decoded subband LL _s and the respective auxiliary information frequency band, MAD and the SAD of the decoded sub-band LL is, MAD <μSAD (but, mu is a constant) In this case, the entropy encoding means is instructed to encode, and in the case of MAD ≧ μSAD, the Slepian-Wolf encoding means for performing Wyner-Ziv encoding is instructed to encode ,
In the Slepian-Wolf encoding step,
An image encoding method characterized in that auxiliary information for obtaining a SAD having the smallest value among the SADs is acquired from the decoder and encoded. An image decoding method for decoding a moving image frame using a wavelet domain DVC,
A decoding step in which the decoding means decodes the input Key frame;
Auxiliary information generation means generates auxiliary information that is the subband LL _L of the lowest frequency of the s-th stage by performing DWT transformation on the estimated value of the forward prediction and the backward prediction for the Wyner-Ziv frame s times. An auxiliary information generation step;
Reconstructing means for reconstructing a subband of the lowest frequency band of the Wyner-Ziv frame using the auxiliary information;
The mode determination means divides the subbands LH _s and HL _s of the medium and high frequency band and the subband HH _s of the high frequency band in the wavelet s stage of the Wyner-Ziv frame into small rectangular blocks, and Determining whether to perform entropy encoding or Wyner-Ziv encoding, and sending a mode determination result to the encoder;
Entropy decoding means, the Wyner-Ziv subbands high and high frequency bands in the wavelet first s stage frame LH _s, HL _s, entropy decoding the intra-coded blocks of the subband HH _s in the high frequency band Entropy decoding step,
Slepian-Wolf decoding means performs a Slepian-Wolf decoding step of Slepian-Wolf decoding a Wyner-Ziv encoded block;
In the mode determination step,
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction and bi-directional prediction of the Wyner-Ziv frame, the low frequency band divided into small rectangular blocks has been decoded. get the subband LL _s and SAD is an index representing a correlation of the respective auxiliary information, MAD and the SAD of the decoded subband LL _s is, MAD <If μSAD (but, mu is a constant), entropy If MAD ≧ μSAD, it is determined that Wyner-Ziv encoding is performed .
In the Slepian-Wolf decoding step,
Decoding is performed using auxiliary information from which the SAD having the smallest value among the SADs is obtained.
An image decoding method characterized by the above. An image encoding method for encoding a moving image frame using a wavelet domain DVC,
An encoding step in which the encoding means encodes the input Key frame;
A wavelet transform step in which DWT transforms the input Wyner-Ziv frame into a wavelet domain;
A quantization means for quantizing the signal in the wavelet domain and dividing a subband of the lowest frequency band into bit planes;
An entropy encoding unit performs entropy encoding of an output from the quantization unit based on a mode switching determination result from a decoder; and
Slepian-Wolf encoding means, based on the mode switching determination result from the decoder, Slepian-Wolf encoding step for performing Wyner-Ziv encoding on the bit plane acquired by the quantization means;
A demultiplexer performs a distribution step of distributing the output from the quantization means to either the entropy encoding means or the Slepian-Wolf encoding means based on the mode determination result acquired from the decoder,
In the sorting step,
When the decoder has auxiliary information, which is four subbands obtained by performing discrete wavelet transform on forward prediction, backward prediction, and bidirectional prediction of the Wyner-Ziv frame, a small rectangular block is output from the decoder. the MSE of the divided low frequency band decoded subband LL _s and each of the auxiliary information to obtain the result of calculation, the variance sigma and the MSE of the decoded subband LL _s is, σ <μMSE (where, mu Is a constant), the encoding is instructed to the entropy encoding means, and in the case of σ ≧ μMSE, the encoding is instructed to the Slepian-Wolf encoding means for performing Wyner-Ziv encoding,
In the Slepian-Wolf encoding step,
Auxiliary information for obtaining the smallest MSE (Mean Square Error) in the MSE is obtained from the decoder and encoded.
An image encoding method characterized by the above. An image decoding method for decoding a moving image frame using a wavelet domain DVC,
A decoding step in which the decoding means decodes the input Key frame;
Auxiliary information generation means generates auxiliary information that is the subband LL _L of the lowest frequency of the s-th stage by performing DWT transformation on the estimated value of the forward prediction and the backward prediction for the Wyner-Ziv frame s times. An auxiliary information generation step;
Reconstructing means for reconstructing a subband of the lowest frequency band of the Wyner-Ziv frame using the auxiliary information;
The mode determination means divides the subbands LH _s and HL _s of the medium and high frequency band and the subband HH _s of the high frequency band in the wavelet s stage of the Wyner-Ziv frame into small rectangular blocks, and Determining whether to perform entropy encoding or Wyner-Ziv encoding, and sending a mode determination result to the encoder;
Entropy decoding means, the Wyner-Ziv subbands high and high frequency bands in the wavelet first s stage frame LH _s, HL _s, entropy decoding the intra-coded blocks of the subband HH _s in the high frequency band Entropy decoding step,
Slepian-Wolf decoding means performs a Slepian-Wolf decoding step of Slepian-Wolf decoding a Wyner-Ziv encoded block ;
In the mode determination step,
When there is auxiliary information that is four subbands obtained by discrete wavelet transform of forward prediction, backward prediction and bi-directional prediction of the Wyner-Ziv frame, the low frequency band divided into small rectangular blocks has been decoded. Gets the results of calculating the subband LL _s and MSE of each of the auxiliary information, the variance σ and the MSE of the decoded subband LL _s is the case σ of <μMSE (where, mu is a constant), entropy coding If σ ≧ μMSE, it is determined that Wyner-Ziv encoding is performed ,
In the Slepian-Wolf decoding step,
Decoding using auxiliary information that provides the smallest MSE among the MSEs,
An image decoding method characterized by the above. An image encoding program for causing a computer to function as each means constituting the encoder according to claim 1 . An image decoding program for causing a computer to function as means for constituting the decoder according to claim 2 .

Images