JP4515378B2 - Image decoding apparatus, program, and computer-readable recording medium - Google Patents

Description

  The present invention relates to an image decoding apparatus, a program, and a computer-readable recording medium, and in particular, multi-viewpoint image encoding, that is, a system that needs to encode each other independently, and loss of radio, the Internet, etc. BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding apparatus and program for restoring a series of images (still images and moving images) that are divided and encoded separately in a generated network communication system, and a computer-readable recording medium.

  In recent years, there has been a shift from analog signal systems to digital signal systems, and the demand for digital images has increased. However, since a digital image has a huge amount of data as it is, an encoding technique for efficiently compressing the image is important, and an international standard algorithm is widely used.

  As an example, MPEG-2 (“Information technology-Generic coding of movie pictures and associated audio information: Video,” ISO / IEC 13818-2, May 1996) is used for digital TV. H.264 (“Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (IUT-T Rec. H.264-ISO / IEC 14496-10 AVC),” Joint Video Team of ISO / IEC MPEG & ITU -T, 2003) has been decided to be used.

  The specific compression algorithm of these image compression algorithms is such that motion compensation (MC) is first performed on the encoder side to eliminate temporal signal redundancy, and then discrete cosine transform ( Entropy coding is performed after spatial signal redundancy is eliminated by frequency conversion such as DCT). The decoder side performs decoding by performing the reverse process.

  However, in recent years, the standardized moving image compression algorithm has a high compression efficiency, but the amount of computation has become enormous. For example, H.264 requires a large amount of arithmetic processing, and it is known that the processing arithmetic amount increases by about 5 to 10 times compared to MPEG-2. This large amount of arithmetic processing is necessary for complicated inter-frame prediction, variable macroblock size optimization, and the like, and is necessary on the encoder side.

  On the other hand, for example, there is a demand for less processing on the encoder side even if arithmetic processing on the decoder side is increased in applications such as sensor camera and mobile phone video processing. One system that satisfies this requirement and has shifted the high-load computation processing on the encoder side to the decoder side is Distributed Source Coding (DSC).

  As shown in FIG. 7, DSC is a system in which a plurality of correlated information sources are distributed and encoded without observing each other, and the separately encoded data is collectively decoded on the receiving side. It is. DSC was originally established by the Slepian-Wolf theorem (D. Slepian and JK Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inform. Theory, vol. 19, no. 4, pp.471-480, 1973) and the Wyner-Ziv theorem established by Wyner and Jiv (A. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,” IEEE Trans. Inform Theory, vol. 22, no1, pp.1-10, 1976), which is an extension of these theorems to actual systems. The Slepian-Wolf theorem gives an allowable compression rate region that can be decoded without distortion when the two information sources are encoded separately without observing each other. The Wyer-Ziv theorem is A rate distortion region is given when distortion occurs in one information source. According to these theorems, the compression limit when coding separately without observing each other is within the range of the Slepian-Wolf and Wyner-Ziv theorem, but when observing each other and coding. Proven to be equal. In recent years, research has been conducted on the extent to which the limits given by the Slepian-Wolf theorem and the Wyner-Ziv theorem are approached using Turbo codes (for example, see Non-Patent Document 1 and Non-Patent Document 2).

  There are various DSC configuration methods. For example, as shown in FIG. 8, a method using a turbo code for encoding and decoding a Wyner-Ziv frame has been proposed. This method can reduce the processing on the encoder side and further improves the coding efficiency by creating supplementary information from the Key frame (see, for example, Non-Patent Document 3).

However, in the creation of auxiliary information from the Key frame, the amount of information processing increases because of the structure in which all information is once decoded and then auxiliary information is recreated. In addition, when suitable supplementary information cannot be created, it is necessary to retransmit parity bits and the like, which cannot be used in a large-scale distribution system using a network or a system that requires real-time performance.
J. Bajcsy and P. Mitran, “Coding for the Slepian-Wolf problem with turbo codes,” in Proc. IEEE Global Communications Conf., Vol.2, 2001, pp.1400-1404 A. Aaron and B. Girod, “Compression with side information using turbo codes,” in Proc. IEEE Data Compression Conf., 2002, pp.252-261 B. Girod, A. Aaron, S. Rane and D. Rebollo- Monedero, “Distributed video coding,” Proceedings of the IEEE, Special Issue on Video Coding and Delivery, vol. 93, no. 1, pp. 71-83 , January 2005

  As described above, DSC is expected to be effective as a coding for a best-effort communication system such as multi-view video coding or IP network. However, even when a turbo encoder is used or MPEG-2 is used, decoding processing and partial encoding processing are required to create supplementary information, and the amount of calculation processing is increasing.

  The present invention has been made in view of the above points, and by using a scalability function that has been attracting attention in recent years, a supplementary information is created to construct a DSC system and provide a high-quality moving image. An object is to provide an image decoding apparatus, a program, and a computer-readable recording medium.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 1 ) is an image decoding apparatus for decoding a plurality of separately encoded image data sequences,
The data string A of the plurality of data strings is divided into two paths, the first path is decoded and output, and the low frequency component of the data string of the second path is the first auxiliary information, the high frequency First decoding means for decoding the component as second auxiliary information;
Auxiliary information creating means for performing motion compensation on the first and second auxiliary information and outputting them as third and fourth auxiliary information, respectively;
A second decoding means for decoding a data string B different from the data string A among the plurality of data strings using the third auxiliary information;
It is determined whether the fourth auxiliary information is likely using the probability distribution or the correlation value of the signal. If it is judged that the fourth auxiliary information is likely, the reconstruction is performed by adding to the image decoded by the second decoding means and outputting it. Means.

The present invention (Claim 2 ) is an image decoding apparatus for decoding a plurality of separately encoded image data sequences,
The data string A of the plurality of data strings is divided into two paths, the first path is decoded and output, and the low frequency component of the data string of the second path is the first auxiliary information, the high frequency First decoding means for decoding the component as second auxiliary information;
Auxiliary information creating means for performing motion compensation on the first and second auxiliary information and outputting them as third and fourth auxiliary information, respectively;
High frequency component estimation means for compensating for the shortage of the high frequency component of the fourth auxiliary information output from the auxiliary information creation means, and outputting as fifth auxiliary information;
A second decoding means for decoding a data string B different from the data string A among the plurality of data strings using the third auxiliary information;
It is determined whether the fifth auxiliary information is likely using the probability distribution or the correlation value of the signal. If it is determined that the fifth auxiliary information is likely, the reconstruction is performed by adding the image decoded by the second decoding means and outputting it. Means.

According to the present invention (Claim 3 ), in the image decoding device according to Claim 1 or 2 , the data string A is data encoded by JPEG2000, which is an international standard.

The present invention (Claim 4 ) is an image decoding program that causes a computer to function as the image decoding apparatus according to any one of Claims 1 to 3 .

The present invention (Claim 5 ) is a computer-readable recording medium storing a program that causes a computer to function as the image decoding device according to any one of Claims 1 to 3 .

  As described above, according to the present invention, it is possible to construct a DSC that effectively uses the JPEG2000 international standard algorithm, and the encoding efficiency is higher than that of a general JPEG2000 that does not perform prediction on the decoder side. Improvement in image quality can be expected.

  In addition, there is a demerit that the high-frequency component of the supplementary image is insufficient due to the structure of this system, but by introducing a time resolution expansion process that compensates for it, high frequency component estimation is performed by nonlinear processing. It is possible to extend the temporal resolution of a moving image having a band component. An efficient DSC using an existing system can be configured by expanding the time resolution using a multi-component conversion mechanism that can be implemented in JPEG2000 Part 2 and using a scalability function.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 shows a system configuration example that can be realized by the present invention.

  This figure shows the principle of creating a Wyner-Ziv frame and a Key frame.

  On the encoder side, the Key frame is compressed by the compression device 3 using a code string compressed by embedded coding, for example, a compression algorithm established in an international standard such as JPEG2000. On the decoding side, the Key frame is decompressed by the decompressing device 4 using the same compression algorithm.

  Also, in the compressor 1 of the Winer-Ziv frame on the encoder side, channel coding such as LDPC code or Turbo code is performed, and only the redundant information to be created is transmitted. That is, the compression rate is determined by how much redundant information is added. In the decoder-side Winer-Ziv frame decompression device 2, the supplementary information a created from the Key frame by the supplementary information creation device 5 described later is used to decompress using the sum-product method or the like. The auxiliary information a is created by motion prediction or filter processing from the auxiliary information b extracted without performing calculation or the like from the embedded encoded key frame. Further, in the Winer-Ziv frame, an image expanded using the auxiliary information a is reconstructed by using the auxiliary information c configured by a component exclusive of the auxiliary information a by the reconstruction device 6. Thus, in the present invention, it is possible to create a high-quality image.

[First Embodiment]
FIG. 3 is a basic configuration diagram of the DSC system according to the first embodiment of the present invention.

  In the figure, the encoder side encodes multiple data strings (Key frame and Wyner-Ziv frame), and the decoder side receives the encoded multiple data strings and decodes the Wyner-Ziv frame from the Key frame. Shows the system to do.

  The system shown in FIG. 1 includes compression devices 1 and 2, expansion devices 7 and 8, and auxiliary pixel creation device 9.

  The compression device 1 has a Wyner-ziv frame encoding unit 10, the compression device 3 has a Key frame encoding unit 11, the decompression device 7 is a Wyne-Ziv decoding unit 12, and the decompression device 8 is a Key frame decoding unit. 13, the extraction unit 14, and the auxiliary pixel creation device 9 includes a partial decoding unit 15 and a motion compensation unit 16.

  Hereinafter, each component will be described.

  First, the information source is divided into a Key frame and a Wyner-Ziv frame. Each of the compression devices 1 and 3 is compressed by the encoding units 10 and 11 separately.

  The encoding unit 10 in the Wyner-Ziv frame uses an error correction code and generates a parity bit. Only the parity bits are sent to the decompression device 7. In other words, this is equivalent to compressing the message data k into redundant data m using an error correction code. The closer the error correction encoding efficiency R is to 1, the higher the compression rate. The encoding unit 11 in the Key frame performs embedded encoding such as JPEG2000 and sends it to the decompression device 8.

  In the key frame decompression device 8, the encoded key frame is divided into two passes, and in one pass, decoding is performed by the decoding unit 13 and output as a decoded image of the key frame. The other path is sent to the extraction unit 14 and a part of the data string encoded by the encoding unit 11 is extracted. The extracted data is input to the partial decoding unit 15 of the auxiliary pixel creation device 9, and the low frequency component is decoded and sent to the motion compensation unit 16. The motion compensation unit 16 performs motion compensation on the sent low-frequency portion by linear filter processing. At this time, motion compensation is performed using low frequency components such as a Key frame (t) and a Key frame (t-1), and auxiliary information is created. The created auxiliary information is sent to the decoding unit 12 of the expansion device 7 of the Wyner-Ziv frame. In addition, although the example which performs motion compensation using a linear filter is shown in said motion compensation part 16, it is not limited to this example, If it is a technique in which motion compensation is possible, it will use other techniques. Also good.

  In the decoding unit 12, decoding processing is performed using the auxiliary information output from the motion compensation unit 16 and the parity bits generated by the encoding unit 10, and a Wyner-Ziv frame is output.

  By the above operation, the Wyner-Ziv frame has the same effect as that obtained by inter-frame decoding, and it can be expected that the compression efficiency is improved and the robustness is improved because the method uses an error correction code.

[Second Embodiment]
FIG. 4 is a basic configuration diagram of a DSC system according to the second embodiment of the present invention.

  In the figure, a plurality of data strings (Key frame and Wyner-Ziv frame) are encoded on the encoder side, and a plurality of data strings are received on the decoder side, and the auxiliary pixel creation device 105 receives the Wyner-Ziv frame from the Key frame. 2 shows a system in which auxiliary information is created by the reconstruction device 106.

  The system shown in the figure includes compression apparatuses 101 and 102, expansion apparatuses 103 and 104, an auxiliary pixel creation apparatus 105, and a reconstruction apparatus 106.

  The compression apparatus 101 includes a Wyer-Ziv frame encoding unit 20, and the compression apparatus 102 includes a Key frame encoding unit 21. The decompressor 103 includes a Wyer-Ziv frame decoder 22, and the decompressor 104 includes a Key frame decoder 23 and an extractor 24. The auxiliary pixel creation device 105 includes a partial decoding unit 25 and a motion compensation unit 26. The reconstruction device 106 includes an auxiliary information determination unit 27 and a synthesis unit 28.

The present embodiment is different from the first embodiment in that a reconstruction device 106 is included. In the first embodiment, the low-frequency component is extracted from the Key frame to reduce the amount of calculation and increase the coding efficiency, but there is a possibility that a Wyner-Ziv frame lacking the high-frequency component is created. . Therefore, in this embodiment, by introducing the reconstruction device 106, the low frequency component is decoded by the decoding unit 22 of the expansion device 103 of the Wyner-Ziv frame, and the high frequency component that is lacking by the reconstruction device 106. It supplements.

  In the first embodiment, the partial decoding unit 15 of the auxiliary pixel generation device 9 has only been sent to the motion compensation unit 16 in the low-frequency component, but in this embodiment, the partial decoding unit 25 of the auxiliary pixel generation device 105. The high frequency component can also be sent to the motion compensation unit 26. The motion compensation unit 26 performs motion compensation on the low frequency components, and sends auxiliary information to the decoding unit 22 as in the first embodiment. On the other hand, it is more difficult to increase the accuracy of motion compensation for high frequency components than for low frequency components, leading to a decrease in coding efficiency. Therefore, after motion compensation is performed by the motion compensation unit 26, the auxiliary information determination unit 27 of the reconstruction device 106 determines whether the auxiliary information is likely. Judgment of plausibility is made using signal probability distribution, correlation value, and the like. If it is determined to be plausible, the signal is added to the signal decoded by the decoding unit 22 by the combining unit 28 of the reconstruction device 106, and a Wyner-Ziv frame is output.

  As an example of the above, FIG. 5 shows a DSC configuration using LDPC code and JPEG2000.

  First, the key frame (odd frame) on the encoding side is encoded by JPEG2000. On the other hand, the W-Z frame (even frame) is subjected to post-quantization so that only the low-frequency component of the image remains after being subjected to Mallet division by wavelet transform (DWT). The quantized DWT coefficient is converted to a Gray code conversion (a conversion in which the hamming distance between adjacent decimal numbers is also 1 in a binary number: Gray, F. “Pulse Code Communication.” United States Patent Number 2632058. March 17, 1953) and encoded by a Low Density Parity Check (LDPC) encoder (RG Gallager, “Low density parity check codes,” in Research Monograph series. Cambridge, MIT Press, 1963). Only the syndrome bits created by the LDPC code are transmitted in W-Z Frames.

  In the configuration shown in FIG. 5, the wavelet transformer (DWT) uses only wavelet transform and the quantizer (Q) to make only low frequency components, and the gray code encoder performs gray coding. This improves the efficiency of motion prediction on the decoder side.

  In the decoding process, after the Key frame (odd frame) is decoded by the JPEG2000 decoder, auxiliary information (Side Information) necessary for decoding the W-Z frame (even frame) is created. Auxiliary information, wavelet transform (DWT), quantization (Q), and Gray code conversion (Gray code converter) are applied to the prediction signal created from the decoded Key frame by linear prediction. Is created and sent to the LDPC decoder.

  In the LDPC decoder, the error of the auxiliary information is corrected by the sum-product decoding method using the received syndrome bits. In other words, this is equivalent to correcting a signal from which the predicted image has been removed on the decoder side. The signal decoded by the sum-product decoding method is subjected to inverse gray code transformation, inverse quantization, and inverse wavelet transformation, reconstructed with an intermediate signal created from the Key frame, and the decoding process of the W-Z frame is completed.

  Note that the wavelet transform and quantization processing on the decoder side can be omitted by using the JPEG2000 scalability function. In that case, an intermediate signal is created for the extracted low-frequency components, and gray transform is performed. Sent to the LDPC decoder. The intermediate image can be created using multicast component conversion of JPEG 2000 Part 2.

[Third Embodiment]
FIG. 6 is a basic configuration diagram of a DSC system according to the third embodiment of the present invention.

  In the figure, the encoder side encodes a plurality of data strings (Key frame and Wyner-Ziv frame), and the decoder side receives the plurality of data strings and creates auxiliary information of the Wyner-Ziv frame from the Key frame. Indicates the system.

  The system shown in the figure includes compression devices 201 and 202, decompression devices 203 and 204, an auxiliary pixel creation device 205, a reconstruction device 206, and a high frequency component estimation device 207.

  The compression device 201 includes a Wyner-Ziv frame encoding unit 30, the compression device 202 includes a Key frame encoding unit 31, and the decompression device 203 includes a Wyner-Ziv frame decoding unit 32. 204 includes a key frame decoding unit 33 and an extraction unit 34, an auxiliary pixel generation device 205 includes a partial decoding unit 35 and a motion compensation unit 36, and a high frequency component estimation device 207 includes a high frequency component estimation unit 37. The reconstruction device 206 includes an auxiliary information determination unit 38 and a synthesis unit 39.

  The configuration shown in FIG. 6 is different from the second embodiment in that a high frequency component estimation device 207 is included. In the second embodiment, a high frequency component is sent from the Key frame to the motion compensation unit 36 separately from the low frequency component, but the high frequency component is insufficient due to motion compensation (such as filter processing). Therefore, in the present embodiment, the high frequency component estimation device 207 is introduced to compensate. For this purpose, a high-frequency component estimation device that is compatible with the encoding unit 31 and performs time resolution expansion processing is used. The high-frequency component estimation device 207 upsamples the number of samples (frame rate) in the time axis direction of the input low-resolution video, removes unnecessary high-frequency components (imaging components) by a filter, The input low-resolution moving image is subjected to high-frequency prediction in the time axis direction using a nonlinear prediction method.

  In addition, the operations of the first to third embodiments described above can be constructed as a program, and can be installed and executed on a computer that performs decoding processing of a plurality of encoded data strings input from the encoder side. It is.

  The constructed program can be stored in a portable storage medium such as a hard disk device or a flexible disk / CD-ROM, installed in a computer, executed, or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a moving image generation technique, particularly a technique for decoding an encoded moving image signal.

It is a principle block diagram of this invention. 2 is a system configuration example that can be realized by the present invention. 1 is a basic configuration diagram of a DSC system in a first embodiment of the present invention. It is a basic block diagram of the DSC system in the 2nd Embodiment of this invention. It is a structural example of the DSC system in the 2nd Embodiment of this invention. It is a basic block diagram of the DSC system in the 3rd Embodiment of this invention. It is a conceptual diagram of Distributed Source Coding (DSC). It is an example of a system configuration using a turbo code for encoding / decoding.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Encoding means, compression apparatus 2 2nd decoding means, decompression apparatus 3 Encoding means, compression apparatus 4 1st decoding means, expansion apparatus 5 Auxiliary information creation means, auxiliary information creation apparatus 6 Reconstruction apparatus 7 Expansion apparatus 8 Decompression device 9 Auxiliary pixel creation device 10 Encoding unit 11 Encoding unit 12 Decoding unit 13 Decoding unit 14 Extraction unit 15 Partial decoding unit 16 Motion compensation unit 20 Encoding unit 21 Encoding unit 22 Decoding unit 23 Decoding unit 24 Extraction unit 25 Partial decoding unit 26 Motion compensation unit 27 Auxiliary information determining unit 28 Combining unit 30 Encoding unit 31 Encoding unit 32 Decoding unit 33 Decoding unit 34 Extraction unit 35 Partial decoding unit 36 Motion compensation unit 37 High frequency component estimation unit 38 Auxiliary information determination 39 Combining unit 101 Compression device 102 Compression device 103 Expansion device 104 Expansion device 105 Auxiliary pixel creation device 106 Reconstruction device 201 Compression device 202 Compression device 203 Expansion device 204 expansion apparatus 205 auxiliary pixel producing apparatus 206 reconstructor 207 high-frequency component estimating unit

Claims (5)

An image decoding device for decoding a plurality of separately encoded image data sequences,
The data string A of the plurality of data strings is divided into two paths, the first path is decoded and output, and the low frequency component of the data string of the second path is the first auxiliary information, high First decoding means for decoding the band component as second auxiliary information;
Auxiliary information creating means for performing motion compensation on the first and second auxiliary information and outputting them as third and fourth auxiliary information, respectively;
A second decoding means for decoding a data string B different from the data string A among the plurality of data strings using the third auxiliary information;
Whether the fourth auxiliary information is likely is determined using the probability distribution or correlation value of the signal. If it is determined that the fourth auxiliary information is likely, the fourth auxiliary information is added to the image decoded by the second decoding means and output. Reconstruction means;
An image decoding apparatus comprising: An image decoding device for decoding a plurality of separately encoded image data sequences,
The data string A of the plurality of data strings is divided into two paths, the first path is decoded and output, and the low frequency component of the data string of the second path is the first auxiliary information, high First decoding means for decoding the band component as second auxiliary information;
Auxiliary information creating means for performing motion compensation on the first and second auxiliary information and outputting them as third and fourth auxiliary information, respectively;
High frequency component estimation means for compensating for the shortage of the high frequency component of the fourth auxiliary information output from the auxiliary information creation means and outputting as fifth auxiliary information;
A second decoding means for decoding a data string B different from the data string A among the plurality of data strings using the third auxiliary information;
Whether the fifth auxiliary information is likely is determined using the probability distribution or correlation value of the signal. If it is determined that the fifth auxiliary information is likely, it is added to the image decoded by the second decoding means and output. Reconstruction means;
An image decoding apparatus comprising: The data string A is data encoded by the international standard JPEG2000.
The image decoding apparatus according to claim 1 or 2 . Computer
An image decoding program that causes the image decoding apparatus according to any one of claims 1 to 3 to function. Computer
A computer-readable recording medium storing a program that functions as the image decoding device according to any one of claims 1 to 3 .

Images