KR100523930B1 - Apparatus for compressing/decompressing multi-viewpoint image - Google Patents

Abstract

Multi-view images are compressed and reconstructed so that they can be transmitted with a minimum amount of data while maintaining better image quality.

The multi-view image compression is performed by encoding the center image encoder while estimating and compensating for the movement of the center image, generating a central image data stream, and generating a reconstructed central image, and the left image encoder and the right image encoder respectively. In addition to estimating and compensating the movement of the left image and the right image, the reference image is restored while performing the estimation of disparity and the compensation of the disparity. The reconstruction of the multiview image is performed by the central image decoder. Decode the data stream and compensate for movement to restore the center image, and the left and right image decoders decode and decode the left and right images, respectively, and move the decoded data and the center image restored by the center image decoder to the reference image. And recover by compensating for the variation.

Description

Apparatus for compressing / decompressing multi-viewpoint image}

The present invention provides a multiview that compresses and reconstructs multi-viewpoint images captured by a plurality of cameras having different viewpoints to have a minimum bit rate while maintaining better image quality. TECHNICAL FIELD The present invention relates to an apparatus and method for compressing and restoring an image, and more particularly, to an apparatus for compressing and restoring a multiview image, which compresses and decompresses an image of three views.

The stereoscopic imaging systems of U.S. Pat.Nos. 5,612,735 and 5,619,256, which are widely known as multiview profiles of ISO / IEC13818 Moving Pictures Expert Group (MPEG-2), are images of two views, that is, images of the left view (hereinafter, 'left image'). And the image of the right view (hereinafter, referred to as 'right image') to implement a 3D image.

According to the stereoscopic image system, one image among the left image and the right image, for example, the left image is set as a reference image, and the stereo encoder module moves estimation with respect to the left image set as the reference image. ME), Moving Compensator (MC), Discrete Cosine Transform (DCT), Inverse DCT (IDCT), Quantization, Dequantization, Variable Length Coder (VLC), Variable Length Decoder (VLD) MPEG-2 video compression is performed in a general manner through a block that performs such a function.

The 3D encoder module performs motion estimation and motion compensation on the right image, which is an image of another viewpoint, and performs compression by disparity estimation (ME) and disparity compensation (MC) with reference to the left image. do. That is, a disparity estimation is performed by using the left image as a reference image to obtain a disparity vector, a motion vector is obtained by performing motion estimation on the basis of the disparity vector and the same right image before or after, and the motion vectors of the disparity vector and the motion vector are obtained. Compress using the better of the results.

The data streams generated by compressing the left and right images by the stereo encoder module should be transmitted to the stereo decoder module based on Temporal Scalability (TS) of MPEG-2, so that the base layer and the enhancement layer are enhanced. layer), a data stream compressed with the left image is transmitted to the base layer, and a data stream compressed with the right image is transmitted to the enhancement layer.

Here, the data stream transmitted to the enhancement layer should be restored by referring to the data stream of the base layer. When there is no data stream of the base layer, the data stream transmitted to the enhancement layer cannot be restored.

This conventional technique will be described in detail with reference to the drawings of FIG. 1.

1 is a block diagram showing the configuration of a conventional multiview image compression / restore apparatus described in US Pat. Nos. 5,612,735 and 5,619,256. As shown in the drawing, a conventional multi-view image compression / restore apparatus includes a stereo encoder module 100 that compresses a left image and a right image, and data of a left image and a right image compressed by the stereo encoder module 100. It consists of a three-dimensional decoder module 200 for receiving and restoring a stream transmitted through a transmission cable or wireless.

The stereoscopic encoder module 100 includes a left image motion compensation DCT encoder 110 having a base layer of MPEG-2, a motion and variance compensation DCT encoder block 120, and a system multiplexer 130. The motion and disparity compensation DCT encoder block 120 includes a disparity estimator 121, a disparity compensator 123, and a right image motion compensation DCT encoder 125 composed of a base layer of MPEG-2.

The stereo decoder module 200 includes a system demultiplexer 210, a left image motion compensation DCT decoder 220 composed of a base layer of MPEG-2, and a motion and disparity compensation DCT decoder block 230. The motion and disparity compensation DCT decoder block 230 includes a right image motion compensation DCT decoder 231 composed of a base layer of MPEG-2, and a disparity compensator 233.

The conventional multi-view image compression / restore device having such a configuration performs compression and reconstruction by setting the left image as a reference image. The left image, which is the reference image, is the left image motion compensation DCT encoder of the stereoscopic encoder module 100. The encoder 110 encodes according to the MPEG-2 standard algorithm, outputs the encoded data stream to the system multiplexer 140, and generates a reconstructed left image for encoding of the next image and the right image. The data is stored in a built-in memory and then output to the shift estimator 121 and the shift compensator 123 of the motion and shift compensation DCT encoder block 120.

The right image is input to the disparity estimator 121 and the right image motion compensation DCT encoder 125 of the motion and disparity compensation DCT encoder block 120. The disparity estimator 121 is the right image and the left image motion compensation DCT encoder. A disparity vector is obtained by comparing the reconstructed left image input from 110, and the disparity vector is transferred to the disparity compensator 123 and the right image motion compensation DCT encoder 125.

The disparity compensator 123 generates a compensation image for the next right image by using the disparity vector received from the disparity estimator 121 and the reconstructed left image input from the left image motion compensation DCT encoder 110. The generated compensation image is transmitted to the motion compensation DCT encoder 125 of the right image.

The motion compensation DCT encoder 125 of the right image encodes the disparity vector received from the disparity estimator 121 in the same manner as the motion vector of the MPEG-2 standard. In addition, the right image motion compensation DCT encoder 125 obtains a difference image between the compensation image generated by the disparity compensator 123 and the right original image, encodes the encoded image using an MPEG-2 encoding algorithm using DCT, and encodes the encoded data. The stream is output to the system multiplexer 130.

The system multiplexer 130 conforms to the standards of the MPEG-2 system, and encodes the left and right video data streams respectively input from the left image motion compensation DCT encoder 110 and the right image motion compensation DCT encoder 125. They are multiplexed and transmitted to the system demultiplexer 210 of the stereoscopic decoder module 200 through a predetermined transmission medium.

The system demultiplexer 210 of the stereo decoder module 200 demultiplexes the multiplexed data stream received from the system multiplexer 130 to restore the data stream of the left image and the data stream of the right image, and restore the left image. The data stream and the right image data stream are respectively input to the left image motion compensation DCT decoder 220 and the right image motion compensation DCT decoder 230 of the motion and disparity compensation DCT decoder block 230.

Then, the left image motion compensation DCT decoder 220 decodes the received left image data stream through an algorithm that satisfies the MPEG-2 standard, restores the left image, outputs the reconstructed left image to the outside, and distorts it. Input to the compensator 233.

The motion compensation DCT decoder 231 of the right image of the motion and disparity compensation DCT decoder block 230 decodes the data stream of the right image received from the system demultiplexer 210 to obtain a disparity vector and a disparity compensator 233. ) To obtain the difference image between the compensation image and the original image with reference to the left image generated as described below.

The disparity vector obtained by the right image motion compensation DCT decoder 231 is input to the disparity compensator 233. The disparity compensator 233 is reconstructed previously by the disparity vector and the left image motion compensation DCT decoder 220. A compensation image is generated with reference to the left image, and the generated compensation image is input to the right image motion compensation DCT decoder 231, and the right image motion compensation DCT decoder 231 is a compensation image input from the disparity compensator 233. The right image is reconstructed by synthesizing the obtained difference image.

However, the above-described conventional technique is a compression / restore system of two-view images that can be applied only to two images of a left image and a right image, and when the number of images to be compressed / restored to a multiview is increased, The compression / restore system has to cope with further, which causes data redundancy between each time point, resulting in waste of data amount.

In addition, since the MPEG-2 standard is used as it is, only one of the results of the motion estimation and the disparity estimation is coded.

Therefore, an object of the present invention is to compress and reconstruct an image of at least three views, and to expand to multiple multiples of three, such as six and nine, based on the compression and reconstruction of the three-view image as a basic structure. It is to provide a compression / restoration device.

Another object of the present invention is to provide a multi-view image compression / restore apparatus and method for compressing and reconstructing a multi-view image by using motion estimation and disparity estimation simultaneously.

An advantage of the multi-view image compression / restoration apparatus of the present invention having such a purpose is that it can be extended to multiples of three. When the conventional compression / restoration apparatus is expanded, the left image or the right image is compressed using the disparity estimation and the disparity compensation based on the reference image. For example, if a 12-view image is compressed using an existing binocular system, six MPEG base layer coding and six MPEG enhancement layer coding are performed, and data of enhancement layer coding is larger than the data amount of the base layer coding. Small amount The three-view image compression apparatus, which is the basis of the present invention, performs four MPEG base layer codings on a 12-view image, and performs coding similar to eight MPEG enhancement layer codings, but with higher performance. You will have a smaller amount of data.

To this end, the multi-view image compression apparatus of the present invention encodes the central image while generating the central image data stream while estimating the movement of the input central image and compensating for the movement. Provides a reconstructed center image as a reference image, and the left image encoder estimates the movement of the input left image and compensates for the movement, and estimates the variation and the shift by referring to the reconstructed center image provided by the center image encoder as the reference image. While performing the compensation, the left image is encoded to generate a left image data stream, and the right image encoder estimates the movement of the input right image and compensates for the movement, and restores the center image generated by the center image encoder. Encoding the right image while performing disparity estimation and disparity compensation The right video data stream is generated.

The central image encoder generates a difference image by subtracting a motion compensation value to a center image to which a subtractor is input, quantizes the generated difference image by the DCT unit, the quantization unit by changing the quantization coefficient according to the bit rate control signal, The VLC unit performs VLC according to a pre-stored VLC table and outputs it to the outside through a buffer, and generates a bit rate control signal to match a target bit rate according to the data stream of the central image stored in the buffer to adjust the quantization coefficient of the quantization unit. The motion compensation block is adaptively changed, and the motion compensation block performs motion estimation and motion compensation of the center image using the output signal of the center image and the quantization unit, and provides a motion compensation value to the subtractor. The reference image is provided to the right image encoder.

The motion compensation block is an inverse quantizer dequantizes the output signal of the quantization unit, an IDCT unit IDCT, and an adder adds a motion compensation value to the output signal of the IDCT unit to generate a reconstructed center image, and generates the generated reconstruction The frame memory stores the center image and provides the left image encoder and the right image encoder as reference images, and the motion estimator estimates the movement of the center image with reference to the reconstructed center image stored in the frame memory. The motion compensation unit generates a motion compensation value according to the motion estimation value estimated by the motion estimation unit based on a center image, and provides the generated motion compensation value to the subtractor and the adder.

In the left image encoder and the right image encoder, a subtractor generates a difference image by subtracting the motion and disparity compensation values from the left image or the right image, the DCT unit DCT the generated difference image, and the quantization unit quantizes the bit rate control signal. Quantize by changing the coefficients, VLC unit VLC according to the pre-stored VLC table, and then output to the outside through the buffer, bit rate control according to the data rate of the left or right image stored in the buffer to control the bit rate A signal is generated to adaptively change the quantization coefficients of the quantization unit, and a motion / variation compensation block includes motion estimation and motion compensation with the left or right image, the output signal of the quantization unit, and the reconstructed center image of the central image encoder. Perform shift estimation and shift compensation to provide motion and shift It is characterized by providing a phase value.

The motion / disparity compensation block is generated by inverse quantization of the output signal of the quantization unit by inverse quantization, by the IDCT unit by IDCT, and by an adder by adding a motion / disparity compensation value to the frame. The memory is stored and the motion / disparity estimator estimates the motion and the shift of the left or right image by the left or right reconstructed image stored in the frame memory and the reconstructed center image of the central image encoder. The motion compensator generates motion and compensation values according to the motion / disparity estimation values estimated by the motion estimator based on the right reconstructed image and the reconstructed center image of the center image encoder, and subtracts the generated motion and disparity compensation values. And an adder.

The multi-view image reconstruction apparatus of the present invention decodes the data stream of the central image inputted by the central image decoder and compensates for the movement to generate a reconstructed central image, and decodes each of the left image inputted by the left image decoder and decodes the same. Compensate for movement and variance of the data and the reconstructed central image provided by the central image decoder as a reference image to generate a reconstructed left image, and the right image decoder decodes the input right image, respectively, and decodes the decoded data and the central image. The reconstructed right image is generated by compensating for the movement and the shift as the reference image of the reconstructed center image provided by the decoder.

The central image decoder is configured to VLD the data stream of the compressed central image, the inverse quantization unit inverse quantization, the IDCT unit IDCT, and the motion compensation block in the output signal of the IDCT unit to compensate for the movement to generate a reconstructed central image Characterized in that.

The motion compensation block generates a reconstructed center image by adding a motion compensation value to an output signal of the IDCT unit, stores the generated reconstructed center image in a frame memory, and provides the reconstructed center image to the left image decoder and the right image decoder. The motion compensation unit detects motion from the restored center image stored in the frame memory, generates a motion compensation value, and inputs the generated motion compensation value to the adder.

The left image decoder and the right image decoder include a VLD portion of a compressed left image or a right image, a VLD portion of the compressed image, an inverse quantization portion of the output signal of the VLD, an IDCT portion, and an IDCT portion, an output signal of the IDCT portion, and the central image decoder. The motion / disparity compensation block compensates for the movement and the shift by using the reconstructed center image provided by the reconstructed center image to generate the reconstructed left image or the right image.

The motion / disparity compensation block, the adder adds a motion compensation value to the output signal of the IDCT unit to generate a reconstructed left image or a right image, the frame memory stores the output signal of the adder, the reconstructed left image stored in the frame memory Alternatively, the motion / disparity compensation unit detects motion and variation from the right image and the reconstructed center image of the central image decoder to generate motion / disparity compensation values and input the generated motion / disparity compensation values to the adder. .

delete

delete

Hereinafter, a multiview image compression / restore apparatus of the present invention will be described in detail with reference to the accompanying drawings of FIGS. 2 to 11.

2 is a diagram showing the configuration of a preferred embodiment for compressing a three-view image in the multi-view image compression apparatus of the present invention. As shown in the drawing, the three-view compression apparatus of the present invention includes a center image encoder 300, a left image encoder 310, and a right image encoder 320, which generate data streams of a center image, a left image, and a right image, respectively. .

The central image encoder 300 estimates the movement of the input central image (ME) and compensates the movement (MC) while encoding the central image to generate a central image data stream and reconstructing the central image. The image is provided as a reference image to the left image encoder 310 and the right image encoder 320. The left image encoder 310 estimates (ME) the movement of the input left image, compensates the movement (MC), and shifts the image by referring to the reconstructed center image input from the center image encoder 300 as a reference image. A left image data stream is generated by encoding a left image while performing estimation (DE) and disparity compensation (DC). The right image encoder 320 estimates the movement of the input right image (ME), compensates the movement (MC), and estimates the variation by referring to the reconstructed center image input from the center image encoder 300 as a reference image. (DE) and disparity compensation (DC) are performed to generate the right image data stream by encoding the right image.

The three-view image compression apparatus of the present invention can remove temporal and spatial redundancy of adjacent images, thereby reducing the amount of data.

In general, the MPEG2 encoding method consists of a combination of an encoding method (Intracoding: I picture) in the spatial domain and an encoding method (Intercoding: P picture and B picture) in the time domain.

The encoding method on the spatial domain is a method of compressing and transmitting the original image through VT and quantization and VLC.

The encoding method in the temporal domain reconstructs the I image compressed in the spatial domain through an inverse quantizer and IDCT, stores the reconstructed image in the frame memory, and compares the motion with the image currently input through the motion estimation unit. It estimates and encodes the difference image with the original image by compensating for the spatial movement, that is, the movement of the current input image by the estimated movement. In this case, the motion estimation method is referred to as a B picture in the case of including both a P picture in the case of forward estimation and a forward estimation and backward estimation.

Therefore, the motion prediction and compensation of the P picture and the B picture are affected according to the precision of the picture coded with the I picture, and the decoding process of reconstructing the coded picture must first reconstruct the I picture correctly before the B signal is transmitted. The picture and P picture are correctly restored.

In the multi-view image compression apparatus of the present invention, in the encoding of a multi-view image, as shown in FIG. 3, the central image encoder 300 encodes the input central image into an I image and then a B image, Repeating the encoding with the B picture and the P picture, and providing the reconstructed center image restored when encoding with the B picture, the B picture and the P picture to the left image encoder 310 and the right image encoder 320. The left image encoder 310 and the right image encoder 320 encode the left image and the right image, respectively, as I images, and then restore the center image provided by the center image encoder 300 as a reference image. The encoding is repeated for the B picture, the B picture and the P picture with reference.

In addition, in the multi-view image compression apparatus of the present invention, as shown in FIG. 4, the central image encoder 300 encodes the input central image into an I image and then encodes the B image, the B image, and the P image. Repeating the operation, and providing the restored center image restored to the B image, the B image, and the P image to the left image encoder 310 and the right image encoder 320, and the left image encoder 310 and The right image encoder 320 encodes the input left image and the right image into an I image, respectively, and then encodes the P image while referring to the reconstructed center image provided by the center image encoder 300 as a reference image. Repeat what you do.

The multi-view image reconstruction apparatus of the present invention to be described later decodes in the order of encoding by the multi-view image compression apparatus of the present invention, that is, decodes the multi-view image in the order shown in FIGS. Restore

FIG. 5 is a block diagram illustrating a detailed configuration of the central image encoder 300 of FIG. 2. As shown therein, a subtractor 400 for generating a difference image of the center image by subtracting a motion compensation value from the input central image, a DCT unit 410 for DCT the output signal of the subtractor 400, and the DCT A quantization unit 420 for quantizing the output signal of the unit 410 while changing the quantization coefficient according to the bit rate control signal, and VLC for outputting the output signal of the quantization unit 420 according to a pre-stored Variable Length Coding (VLC) table. A VLC unit 430, a buffer 440 for storing a data stream of a center image which is an output signal of the VLC unit 430, and a data stream of a center image stored in the buffer 440 so as to meet a target bit rate. Motion estimation and motion compensation of the center image using a bit rate control unit 450 for generating a rate control signal and adaptively changing the quantization coefficients of the quantization unit 420 and output signals of the center image and the quantization unit 420. To do The motion compensation block 460 provides a motion compensation value to the subtractor 400 and generates a reconstructed center image by restoring the center image to provide the reference image to the left image encoder 310 and the right image encoder 320 as a reference image. It consists of

The motion compensation block 460 may include an inverse quantizer 461 for inversely quantizing the output signal of the quantizer 420, an IDCT unit 462 for IDCT the output signal of the inverse quantizer 461, and An adder 463 for generating a reconstructed center image by adding a motion compensation value to the output signal of the IDCT unit 462, and storing the reconstructed center image generated by the adder 463, and storing the left image encoder 310 and A frame memory 464 provided as a reference image to the right image encoder 320, a motion estimator 465 estimating a motion of the input central image by referring to a reconstructed center image stored in the frame memory 464; Based on the reconstructed central image stored in the frame memory 464, a motion compensation value is generated according to the motion estimation value estimated by the motion estimator 465, and the generated motion compensation value is subtracted from the subtractor 400 and the adder. Provided to 463 Jikim comprises a compensator (466).

In the central image encoder 300 of the present invention configured as described above, the subtractor 400 subtracts the motion compensation value provided by the motion compensation block 460 from the input central image to generate the difference image of the central image. The difference image of the center image generated by the subtractor 400 is DCT by the DCT unit 410, is quantized by the quantization unit 420 according to the bit rate control signal, and then VLC is output by the VLC unit 430. The VLC data is stored in the buffer 440 and then output as a data stream of the center image.

Here, the VLC unit 430 performs VLC on the macroblock of the P picture which reduces the correlation between frames through forward predicted according to the VLC table as shown in FIG. 6 and bidirectional predicted. VLC is performed according to a VLC table as shown in FIG.

In addition, the bit rate control unit 450 generates a bit rate control signal according to the data stream of the central image stored in the buffer 440, and the quantization unit 420 receives data input from the DCT unit 410 according to the generated bit rate control signal. The target bit rate is adjusted by quantizing while controlling the bit rate.

The inverse quantization unit 461 of the motion compensation block 460 dequantizes the output signal of the quantization unit 420, and the IDCT unit 462 performs IDCT output. The output data of the IDCT unit 462 is added to the motion compensation value by the adder 463 to generate a reconstructed center image. The reconstructed center image is stored in the frame memory 463, and then the left image encoder 310 and The right image encoder 320 is provided as a reference image.

Here, in order to encode the input central image into a B image, a B image, and a P image, at least two frames of a reconstruction center image are required, and the frame memory 464 stores a capacity to store at least two frames of the reconstruction center image. Have

The motion estimation unit 465 estimates the motion of the input central image as a reference image from the reconstructed center image stored in the frame memory 464, and inputs the estimated motion to the motion compensator 466. Then, the motion compensator 466 generates a motion compensation value using the reconstructed center image and the motion estimation value stored in the frame memory 464, and the generated motion compensation value is input to the adder 463 to provide the IDCT unit ( The addition and reconstruction of the center image is generated to the output signal of 462, and the motion compensation value is input to the subtractor 400 to generate the difference image of the center image.

FIG. 8 is a block diagram illustrating a detailed configuration of the left image encoder 310 and the right image encoder 320 of FIG. 2. A subtractor 500 for generating a difference image by subtracting a motion and disparity compensation value to the left image or the right image input as shown in the drawing, a DCT unit 510 for DCT the output signal of the subtractor 500; A quantization unit 520 which quantizes the output signal of the DCT unit 510 while changing the quantization coefficient according to a bit rate control signal, and a VLC unit which VLCs the output signal of the quantization unit 520 according to a pre-stored VLC table ( 530, a buffer 540 for storing a data stream of a left image or a right image which is an output signal of the VLC unit 530, and a target bit rate according to a data stream of a left image or a right image stored in the buffer 540. A bit rate control unit 550 for adaptively changing the quantization coefficients of the quantization unit 520 by generating a bit rate control signal so as to correspond to, and a reconstructed center image provided by the center image decoder 300 as the reference image. Motion / disparity compensation block 560 which provides motion and disparity compensation values to the subtractor 500 by performing motion estimation, motion compensation, and disparity estimation and disparity compensation using an image or a right image and an output signal of the quantization unit 520. It consists of

The motion / disparity compensation block 560 includes an inverse quantization unit 561 for inverse quantization of the output signal of the quantization unit 520, and an IDCT unit 562 for IDCT the output signal of the inverse quantization unit 561. And an adder 563 which adds a motion / disparity compensation value to the output signal of the IDCT unit 562 to generate a reconstructed left image or reconstructed right image reconstructing a left image or a right image, and the adder 563 A frame memory 564 that stores the generated left image or the right image, and a left image or a restored right image stored in the frame memory 564 and a center image of the center image encoder 300. A motion / variation estimator 565 that estimates the motion and the shift of the right image, and the motion / variation estimated by the motion estimator 565 based on the restored left image or the restored right image stored in the frame memory 564. Estimated value Generated according to the motion compensation and values and is configured to generate a motion compensation and the shift value as a motion / disparity compensator 566 to provide the subtracter 500 and the adder 563.

The left image encoder 310 or the right image encoder 320 of the present invention configured as described above subtracts the motion and disparity compensation values provided by the motion / disparity compensation block 560 to the input left image or the right image. Subtracts and generates a difference image of the left image or the right image, the DCT unit 510 DCTs the generated difference image of the left image or the right image, and the quantization unit 520 quantizes the signal according to the bit rate control signal, The VLC unit 530 performs a VLC according to a pre-stored VLC table, and outputs the data stream of the left image or the right image, which is the VLC data of the VLC unit 530, after being stored in the buffer 540.

In addition, the bit rate controller 550 generates a bit rate control signal according to the data stream of the left image or the right image stored in the buffer 540, and the generated bit rate control signal is input from the DCT unit 510. While controlling the bit rate of the data, it is quantized to meet the target bit rate.

The inverse quantization unit 561 of the motion / variation compensation block 560 dequantizes the output signal of the quantization unit 520, the IDCT unit 562 IDCT, and the data IDCT in the IDCT unit 562 is an adder. A motion / displacement compensation value is added to the data 563 and stored in the frame memory 563 as a reconstructed left image or a reconstructed right image. The motion / disparity estimator 565 estimates the motion and the shift of the reconstructed left image or the reconstructed right image stored in the frame memory 564 as a reference image based on the reconstructed center image input from the central image encoder 300, The estimated motion and the shift are input to the motion / disparity compensator 566. Then, the motion / disparity compensator 566 generates a motion and disparity compensation value using the left or right reconstructed image and the motion and disparity estimation value stored in the frame memory 564, and adds the generated motion and disparity compensation value. In addition to adding to the output signal of the IDCT unit 562 to the input signal (563) to compensate for the movement and the shift of the left image or the right image input to the subtractor 500 and to generate a difference image.

Like the frame memory 464, the frame memory 564 may have a capacity to store a restored left image or a restored right image of at least two frames.

9 is a diagram illustrating an embodiment of restoring a three-view image in a multi-view image restoration apparatus according to the present invention. As shown in the drawing, the multi-view image reconstruction apparatus of the present invention includes a center image decoder 600, a left image decoder 610, and a right image decoder 620.

The central image decoder 600 decodes the data stream of the input central image and compensates for the movement to generate a reconstructed central image, and provides the center image decoder 610 and the right image decoder 620. The left image decoder 610 decodes the input left image, respectively, and compensates for movement and variation based on the decoded data and the reconstructed center image provided by the center image decoder 600 to generate a reconstructed left image. . The right image decoder 620 generates a reconstructed right image by decoding the input right image, respectively, and compensates for the motion and the disparity based on the decoded data and the reconstructed center image provided by the center image decoder 600. .

FIG. 10 is a detailed block diagram illustrating a configuration of the central image decoder 600 of FIG. 9. As shown therein, a VLD unit 700 for VLD data stream of the central image, an inverse quantizer 710 for inversely quantizing the output signal of the VLD unit 700, and an output of the inverse quantizer 710 An IDCT unit 720 for IDCT the signal and a motion compensation block 730 for compensating for the motion in the output signal of the IDCT unit 720 to generate a reconstructed center image.

The motion compensation block 730 stores an adder 731 for generating a reconstructed center image by adding a motion compensation value to the output signal of the IDCT unit 720, and stores the reconstructed center image generated by the adder 731. A motion compensation value is generated by detecting motion using a frame memory 733 provided to the left image decoder 610 and the right image decoder 620 and a reconstructed center image stored in the frame memory 733. The motion compensation unit 735 inputs a motion compensation value to the adder 731.

In the central image decoder 600 of the present invention configured as described above, the VLD unit 700 VLDs the data stream of the input central image, and the dequantized unit 710 dequantizes the decoded signal, and the IDCT unit ( 720 outputs the IDCT.

Data output from the IDCT unit 720 is input to the adder 731 of the motion compensation block 730, and a motion compensation value is added and a reconstructed center image is generated. The reconstructed center image is a frame memory 733. The motion compensation unit 735 generates a motion compensation value and provides it to the adder 731 as a reconstructed center image stored in the left and right image decoders 610 and 620 after being stored in the. Done.

FIG. 11 is a detailed block diagram illustrating a configuration of the left image decoder 610 or the right image decoder 620 of FIG. 9. As shown therein, a VLD unit 800 for VLD data streams of a left image or a right image, an inverse quantizer 810 for inversely quantizing an output signal of the VLD unit 800, and the inverse quantizer 810 IDCT unit 820 to IDCT the output signal, the output signal of the IDCT unit 820 and the reconstructed center image provided by the center image decoder 600 to compensate for the movement and the shifted left image or The motion / variation compensation block 830 generates a right image.

The motion / disparity compensation block 830 includes an adder 831 for adding a motion compensation value to the output signal of the IDCT unit 820 to generate a reconstructed left image or a reconstructed right image, and an output of the adder 831. A motion / variation compensation value by detecting a motion and a shift by a frame memory 833 storing a signal, a restored left image or a restored right image stored in the frame memory 833, and a restored center image of the center image decoder 600. And a motion / disparity compensator 835 for generating a and inputting the generated motion / disparity compensation value to the adder 831.

In the left image decoder 610 or the right image decoder 620 of the present invention configured as described above, the VLD 800 VLDs a data stream of an input left or right image, and decodes the decoded signal 810. Inverse quantization, the IDCT unit 820 is output by IDCT.

The data output from the IDCT unit 820 is input to the adder 831 of the motion / disparity compensation block 830, and the motion / disparity compensation value is added to generate a left image or a right image, and the generated left side of the restoration. The image or the right image is stored in the frame memory 833, and the motion / disparity compensator 735 generates the motion and the disparity compensation values to the adder 831 to the restored left image or the right image stored in the frame memory 733. Will be provided.

As described in detail above, the present invention can be extended by maintaining the maximum image quality at the minimum bit rate without overlapping data than the conventional compression / restoration apparatus. In addition, since the central image encoder and decoder are completely compatible with the codec of the MPEG-2 base layer, the present invention can be independently applied to a digital television receiver, a digital camcorder, an information terminal, and the like.

1 is a block diagram showing the configuration of a conventional multi-view video compression / restoration device,

2 is a view showing the configuration of a preferred embodiment for compressing a three-view image in the multi-view image compression apparatus of the present invention,

3 and 4 are views showing the format and correlation of compressing an image of three views in the present invention,

5 is a block diagram showing a detailed configuration of the central image encoder of FIG.

6 is a view showing a VLC table according to a macroblock type of a P picture in a multi-view video compression apparatus of the present invention.

7 is a view showing a VLC table according to a macroblock type of a B picture in the multi-view video compression apparatus of the present invention.

FIG. 8 is a block diagram illustrating a detailed configuration of the left image encoder and the right image encoder of FIG. 2.

9 is a view showing the configuration of a preferred embodiment for restoring a three-view image in the multi-view image restoration apparatus of the present invention,

FIG. 10 is a block diagram illustrating a detailed configuration of the central image decoder of FIG. 9.

FIG. 11 is a block diagram illustrating a detailed configuration of a left image and a right image decoder of FIG. 9.

Explanation of symbols on the main parts of the drawings

300: center image encoder 310: left image encoder

320: Right image encoder 400, 500: Subtractor

410, 510: DCT unit 420, 520: quantization unit

430, 530: VLC section 440, 540: buffer

450, 550: bit rate control unit 460, 730: motion compensation block

461, 561, 710, 810: Inverse quantization unit 462, 562, 720, 820: IDCT unit

463, 563, 731, 831: Adders 464, 564, 733, 833: Frame memory

465: motion estimation unit 466, 735: motion compensation unit

560, 830: motion / disparity compensation block 565: motion / disparity estimator

566, 835: motion / disparity compensator 600: central image decoder

610: left image decoder 620: right image decoder

700, 800: VLD part

Claims (12)

A center image encoder for estimating the motion of the input center image and compensating the motion, encoding the center image to generate a center image data stream, and providing a reconstructed center image as a reference image; A left image encoder encoding an input left image to generate a left image data stream; And A right image encoder which encodes an input right image to generate a data stream of the right image; Consists of, Each of the left image encoder and the right image encoder; A subtractor for generating a difference image by subtracting a motion and disparity compensation value from a left image or a right image, DCT unit for DCT the output signal of the subtractor, A quantization unit for quantizing the output signal of the DCT unit while changing a quantization coefficient according to a bit rate control signal; A VLC unit which VLCs the output signal of the quantization unit according to a pre-stored VLC table; An inverse quantizer for inversely quantizing the output signal of the quantizer; IDCT unit for IDCT the output signal of the inverse quantization unit; An adder for generating a reconstructed image of a left image or a right image by adding a motion / disparity compensation value to the output signal of the IDCT unit; A frame memory for storing a left or right reconstructed image generated by the adder; A motion / variation estimator for estimating a motion and a shift of a left image or a right image from a left or right reconstructed image stored in the frame memory and a reconstructed center image of the central image encoder Based on the left or right reconstructed image stored in the frame memory and the reconstructed center image of the center image encoder, a motion and a compensation value are generated according to the motion / disparity estimation value estimated by the motion estimator, and the generated motion and disparity compensation are generated. And a motion / disparity compensation unit for providing a value to the subtractor and the adder. The method of claim 1, wherein the central image encoder; A subtractor for generating a difference image by subtracting a motion compensation value to an input central image; A DCT unit to DCT the output signal of the subtractor; A quantizer for quantizing the output signal of the DCT unit while changing a quantization coefficient according to a bit rate control signal; A VLC unit which VLCs an output signal of the quantization unit according to a pre-stored VLC table; A buffer for storing a data stream of a central image which is an output signal of the VLC unit; A bit rate controller for adaptively changing a quantization coefficient of the quantization unit by generating a bit rate control signal in accordance with a target bit rate according to the data stream of the central image stored in the buffer; And Motion estimation and motion compensation of the center image are performed using the output signal of the center image and the quantization unit to provide a motion compensation value to the subtractor and to provide a reconstructed center image as a reference image to the left image encoder and the right image encoder. Multi-view image compression device, characterized in that consisting of compensation blocks. The method of claim 2, wherein the motion compensation block; An inverse quantizer for inversely quantizing the output signal of the quantizer; IDCT unit for IDCT the output signal of the inverse quantization unit; An adder for generating a reconstructed center image by adding a motion compensation value to an output signal of the IDCT unit; A frame memory for storing the restored center image generated by the adder and providing the restored center image as a reference image to the left image encoder and the right image encoder; A motion estimator for estimating the motion of the center image by referring to the restored center image stored in the frame memory; And And a motion compensation unit generating a motion compensation value according to the motion estimation value estimated by the motion estimation unit based on the reconstructed central image stored in the frame memory and providing the generated motion compensation value to the subtractor and the adder. Multi-view video compression device. The method of claim 1, wherein the left image encoder and the right image encoder are each; A buffer for storing a data stream of a left image or a right image which is an output signal of the VLC unit; And Multi-view image compression, characterized in that it further comprises a bit rate control unit for adaptively changing the quantization coefficient of the quantization unit by generating a bit rate control signal according to the data rate of the left image or the right image stored in the buffer Device. delete A central image decoder for decoding a data stream of an input central image and compensating for motion to generate a reconstructed central image; A left image decoder which decodes an input left image and generates a reconstructed left image; It consists of a right image decoder to decode the input right image to generate a restored right image, The left image decoder and the right image decoder are respectively; A VLD unit which VLDs a data stream of a compressed left or right image, An inverse quantization unit for inversely quantizing the output signal of the VLD unit; IDCT unit for IDCT the output signal of the inverse quantization unit; An adder for generating a reconstructed left image or right image by adding a motion / disparity compensation value to the output signal of the IDCT unit; A frame memory for storing an output signal of the adder; Motion and shift compensation values are detected by restoring left or right images stored in the frame memory and restoring center images of the center image decoder to generate motion / disparity compensation values and input the generated motion / disparity compensation values to the adder. Multi-view image reconstruction device, characterized in that consisting of / variance compensation. 7. The apparatus of claim 6, wherein the central image decoder; A VLD unit configured to VLD a data stream of the compressed central image; An inverse quantizer for inversely quantizing the output signal of the VLD unit; IDCT unit for IDCT the output signal of the inverse quantization unit; And And a motion compensation block for compensating for the motion in the output signal of the IDCT unit and generating a reconstructed center image. The method of claim 7, wherein the motion compensation block; An adder for generating a reconstructed center image by adding a motion compensation value to an output signal of the IDCT unit; A frame memory for storing the restored center image generated by the adder and providing the restored center image to the left image decoder and the right image decoder; And And a motion compensator for generating a motion compensation value by detecting a motion with a reconstructed center image stored in the frame memory and inputting the generated motion compensation value to the adder. delete delete delete delete