JPH11113026A - Device and method for encoding and decoding stereoscopic moving image high efficiency - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve predictive efficiency and to perform encoding by adding a search range of a motion vector which shifts a search center by half screen to a range which makes a position that is the same with a coded image of a reference image in the case of detecting a motion vector of a P or B frame. SOLUTION: A motion vector detecting part 7 performs block matching of an inter-frame coded frame (P and B frames) between block image data sent from a block division processing part 6 and a reference image stored in frame memory 14 in the search range that is defined in each block and detects a motion vector whose finite difference becomes a minimum. Encoding parts 6 to 15 perform coding that conforms to an MPEG system and find a motion vector by adding a motion vector search range which shifts a search center by half screen to a motion vector search range which makes a position that is the same with a coded image in a reference image a center when it has detected a motion vector in the P or B frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION In order to simplify a stereoscopic video encoding apparatus using a parallax image, and to utilize the correlation between the parallax images, two left and right input parallax images are matched with a high-efficiency coding algorithm. One of two parallax images is delayed by one image period, synthesized into one image, and a motion vector search range is added to increase the prediction efficiency, and a three-dimensional moving image high-efficiency coding apparatus and decoding for decoding the same are provided. Device.

[0002]

2. Description of the Related Art A method using a parallax image as a method for reproducing a three-dimensional moving image is widely used because the use of glasses makes the apparatus relatively simple. When recording left and right parallax images on a medium or the like, if two moving images are simply recorded, the recording time will be halved. There has been proposed a method of synthesizing a single analog moving image and recording it on a disk. In addition, there is also a device in which two input disparity signals are alternately thinned out on a frame or field basis to form one moving image signal, which is compressed by one encoding device to simplify the device. They are simply thinned out to simplify the device.

FIG. 3 shows an example of a conventional three-dimensional video coding apparatus.
This will be described below. The two parallax signals L and R are switched by a switching signal for each frame (or field) created by the switching signal creating unit 41 using the synchronization signal as an input, and the two parallax signals are switched by the video switching unit 42 for each frame. A / D as one thinned video signal
The data is converted into digital data by the conversion unit 43 and
Supplied to The encoding unit 44 is supplied with the output of the motion compensation unit 45 and performs encoding, and the motion vector and the encoded data are recorded on the disk 47 via the synthesis buffer unit 46. In this coding, since the input image is an image signal in which left and right parallax images are alternately arranged in the time direction, correlation in the time direction used in the MPEG coding is deteriorated, so that efficient coding can be expected. Did not.

[0004]

The two parallax signals to be inputted are alternately thinned out on a frame or field basis to form a single moving picture signal, which is compressed by a single coding apparatus to simplify the apparatus. Planning is done. This is to simplify the apparatus by simply thinning out. In a stereoscopic video encoding device using a parallax image, a device that expands a search range of a motion vector to left and right parallax images by a simple method based on MPEG, and performs encoding with higher prediction efficiency, Never seen before. The present invention solves this problem,
By expanding the search range, encoding can be performed with higher prediction efficiency.

[0005]

SUMMARY OF THE INVENTION The present invention relates to a stereoscopic high-efficiency encoding apparatus using a parallax image, wherein one of the left and right parallax images input simultaneously to the encoding apparatus is delayed by one image period. A delay device for causing a delay, an image processing unit for combining the image delayed by one image period with the other undelayed image in the left and right or up and down directions in one image, and combining the images by the image processing unit And an encoding unit for encoding the image. The encoding unit performs encoding in accordance with the MPEG system. In the case of detecting a motion vector of a P frame or a motion vector of a B frame, a reference unit is used. A motion vector search range obtained by moving the search center for a half screen to the motion vector search range centered on the same position as the encoded image in the image is obtained. Providing stereoscopic videos high-efficiency encoding apparatus that.

According to the present invention, there is provided a stereoscopic image high-efficiency coding apparatus using a parallax image, wherein a delay device for delaying one of left and right parallax images input simultaneously to the coding apparatus by one image period, An image processing unit that divides the image delayed by one image period by the delay device and the other undelayed image into left and right or top and bottom within one image and synthesizes the image, and encodes the image synthesized by the image processing unit The encoding unit performs encoding in accordance with the MPEG system, and in the case of detecting a P-frame motion vector, a motion vector search range centered on the same position as the encoded image in the reference image. Is added to the motion vector search range that has moved the search center for a half screen to obtain a motion vector, and in the case of detecting a B-frame motion vector, the reference image On a screen where a parallax image of the same time exists in the screen, a motion vector range obtained by moving the search center for a half screen to the motion vector search range centered on the same position as the encoded image in the reference image is obtained. In a screen where there is no parallax image at the same time in the reference image, stereoscopic video high-efficiency coding in which a motion vector is obtained in a motion vector search range centered on the same position as the encoded screen in the reference image Provide equipment.

According to the present invention, there is provided a high-efficiency stereoscopic video decoding apparatus for decoding a bit stream encoded by the high-efficiency stereoscopic video encoding apparatus according to claim 1 or 2, wherein the bit stream is extracted by a decoding unit. The image obtained by combining the two parallax images on one screen is separated into two parallax images by an image separation unit, and the parallax images that have not passed through the delay during encoding are provided with a delay unit having a delay of one image unit. A high-efficiency stereoscopic video decoding apparatus that reproduces parallax images at the same time on the left and right is provided.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the three-dimensional moving picture high efficiency coding apparatus of the present invention will be described below with reference to FIG.
The input left and right parallax image signals are A / D converted by the A / D converters 1 and 2, respectively, and are converted into digital image data. One L-system signal is, for example, 1
The signal is delayed by the unit image period delay unit 3 and input to the image processing unit 4. The other signal, for example, the signal of the R system is input to the image processing unit 5 as it is. Each of the input image data is separately sub-sampled by 、,
The data is written to the memories of the image processing units 4 and 5.

The read operation is first switched from reading the subsampled image data for one line from the left eye image memory at the rate of the image data before subsampling, and then reading the subsampled one line data from the right eye image memory. To be one line of data.
By performing this for all the lines, the images are synthesized so that the parallax image data of the left and right halves is included in one image.

As described above, by subsampling the image data in half and synthesizing the same, the synthesized image has the same number of pixels as the input image, so that the load of the encoding process is not increased. . Of course, if the encoding processing capability is provided, it is possible to perform processing with twice the number of pixels. As a result, the left-eye and right-eye images are arranged in each of the left and right halves of the video in one image unit, and the video delayed by one image unit is synthesized into one video and supplied to the next encoding units 6 to 15. Is done. Here, an example has been described in which an image is divided into left and right portions in one image and then synthesized, but the present invention is not limited to this.
The left-eye and right-eye images may be combined into upper and lower images in one image in association with the upper and lower images.

[0011] Digital image data divided into blocks by the block division processing unit 6 is first supplied to the encoding units 6 to 15 by the motion compensation inter-frame encoding method. In the intra-coded frame (I frame), the difference is not taken, but is supplied to the orthogonal transform processing unit 9 as it is to perform the orthogonal transform. The inter-frame coded frames (P frame, B frame) are determined by the motion vector detection unit 7 between the block image data from the block division processing unit 6 and the reference image recorded in the frame memory 14 in each block. In the search range obtained, block matching is performed to detect a motion vector with the smallest difference.

The difference between the image data motion-compensated by the vector and the input block image data is calculated by a subtraction unit 8,
The difference value is supplied to the orthogonal transform unit 9 to perform orthogonal transform. The output of the orthogonal transformer 9 is quantized by the quantizer 10 and supplied to the variable-length encoder 15. The I frame and the P frame are also supplied to a local decoding unit for creating a reference screen. Additional information such as a motion vector and a prediction mode of the motion vector detecting unit 7 is supplied to the multiplexing unit 16 and multiplexed with the encoded image data.

The local decoding unit includes an inverse quantization unit 11,
It is composed of an inverse orthogonal transform unit 12 and an adder unit 13, is inversely quantized by the inverse quantizer 11, and its output is subjected to inverse orthogonal transform by the inverse orthogonal transform unit 12. The prediction data is added to the difference data (P frame) by the adder 13 while the intra frame (I frame) is left as it is, and the frame memory 1
4 is stored as a reference image.

The data quantized by the quantization unit 10 is subjected to variable length coding by a variable length coding unit 15 and
Are multiplexed with additional information such as a motion vector from the motion vector detecting section 7 and output as an MPEG stream. The MPEG stream is supplied to a recording medium, for example, a disk 17, and recorded. On the disk 17 on which the output stream of this encoding device was recorded, the encoding was performed by increasing the search range by expanding the search range of the stereoscopic image data using a parallax image using one encoding device. The signal will be recorded. Therefore, the recorded disk medium 17 can reproduce a stereoscopic image for a long time by the high-efficiency encoding described above. Here, the parallax image delayed in the input processing of the parallax image, in this case, an index indicating that the signal is an L-system signal, is assigned to this MPEG.
Write to the user data area of the stream to indicate that the L system signal has been delayed.

Next, an embodiment of the high-efficiency stereoscopic video decoding apparatus according to the present invention will be described below with reference to FIG.
The coded stream supplied from the disk 20 or the like is decoded by the variable length decoding unit 21 and
, And inversely orthogonally transformed by the inverse orthogonal transform unit 23, and the intra-coded frame (I frame) passes through the adder 24 and is recorded in the frame memory 25.

A one-way predicted frame (P frame) in the inter-coded frame is stored in a frame memory 2 according to a prediction mode and a motion vector from the variable length decoding unit 21.
5 is decoded by performing processing such as addition with the I frame or P frame recorded in the frame memory 5 and is recorded in the frame memory 25. The bidirectional inter-frame prediction frame (B frame) is subjected to processing such as addition according to the prediction mode from the I frame and the P frame recorded in the frame memory 25 according to the prediction mode and the motion vector from the variable length decoding unit 21. Go and decrypt. The image data decoded in this way has two delays different by one image period.
One parallax image is synthesized on one screen.

An image synthesized on one screen is separated into two images by an image separation unit 26. If the image data is subsampled at the time of recording, the separated left and right parallax images are oversampled by the oversampling units 27 and 28, respectively. In order to match the delay between the left and right parallax images, for example, the R-system image is output with a delay of one image period in the inserted delay unit 29. Here, an index for indicating the parallax image delayed in the input processing of the parallax image is read out from the user data area of the MPEG stream, and an L-system oversampling unit without a delay unit is read in accordance with the index signal. 27. The output signal of the oversampling unit 27 and the output signal of the delay unit 29 are output as reproduction signals whose timings match between the left and right parallax images L and R.

Next, a motion vector detection method and a search area in a motion vector search will be described with reference to the drawings. 4 and 5 show that the predicted frame configuration is I
The search area in the motion vector search of the B frame when encoding by BPBPBP... Is defined as a left-eye image (Ln).
This is shown separately for the region and the right-eye image (Rn + 1) region. FIG. 4 shows a case where the left-half image (Ln) of the parallax image and the right-eye image (Rn + 1) are combined in the right half of the coded image shown in the center in the left half. Here, the search area is changed between when the coded block of the coded image is in the left half area and when it is in the right half area.

In the case of detecting the motion vector of the B frame, on the screen shown on the left side in FIG. 4 where a parallax image of the same time (n) exists in the reference image, the same position as the encoded image in the reference image is obtained. That is, the motion vector of the encoded image Ln is obtained by adding the motion vector range moved by the search center (search center) for a half screen to the motion vector search range centered on the same position as the encoded image of the reference image. I made it. Similarly, the same time (n + 1) in the reference image
In the screen shown on the right side in FIG. 5 in which the parallax image exists, the motion vector range obtained by moving the search center (the center of the search) for a half screen to the motion vector search range centered on the same position as the encoded image in the reference image To obtain the motion vector of the encoded image Rn + 1.

Since the reference image Rn + 2 in FIG. 4 is too far in time from the coded image Ln, only the reference image Ln + 1 is used in the search area without using it in the search area. Similarly, in the reference image of FIG. 5, the reference image Ln-1 is not used in the search area, and only the reference image Rn is used in the search area. This is to reduce the amount of calculation by reducing the search area of the part with poor prediction, compared to the case where the left and right are searched in the same manner to obtain a motion vector.

FIGS. 6 and 7 show the same predicted image structure as that shown in FIGS. 4 and 5 when PBP when encoding by IBPBPBP.
It shows a search area for a motion vector search of a frame. In this case, a motion vector search for searching a similar search area is performed regardless of the block position of the encoded screen.

That is, the encoding is performed in accordance with the MPEG system, and in the detection of the motion vector of the P frame, the encoded image of the reference image is set within the motion vector search range centered on the same position of the encoded image Ln-1 of the reference image. The motion vector of the coded screen Ln + 1 is obtained by adding a motion vector search range that has moved the search center for a half screen of Rn. Similarly, a motion vector search range obtained by moving the search center for the half screen of the reference image encoded screen Ln-1 to the motion vector search range centered on the same position of the reference image encoded screen Rn is added. Screen Rn + 2
Motion vector is obtained.

FIGS. 8 and 9 show that the predicted frame structure is
It shows a search area of a motion vector search of a P frame when encoding by IPPPPP. That is, the encoding is performed in accordance with the MPEG system, and in the motion vector detection of the P frame, the motion vector search is performed by moving the search center for a half screen within the motion vector search range centered on the same position as the encoded image in the reference image. Added motion range to obtain motion vector.

FIGS. 4 and 5 show that the predicted frame structure is
The search area for the motion vector search of the B frame when encoding by IBPBPBP.
) Area and a right-eye image (Rn + 1) area, respectively. The search area is different when the coded block of the coded image is in the left half area and in the right half area. Has been described, however, regardless of whether or not a parallax image at the same time exists in the reference image, the same position as the encoded image in the reference image is centered in the case of detecting the motion vector of the B frame. A motion vector search range in which the search center has been moved by half a screen may be added to the motion vector search range described above, and the motion vector may be obtained without reducing the search area.

Next, the motion vector detection of the B frame in the case where the motion vector is obtained without reducing the search area will be described below with reference to FIGS. FIG.
Are the reference images (Ln-1, Rn) shown on the left side of FIG. 4 and the reference image (Ln) shown on the right side of FIG.
+1, Rn + 2) as a search area, add a motion vector search range obtained by moving the search center by half a screen to a motion vector search range centered on the same position as the encoded image in the reference image on both the left and right sides. A motion vector was obtained. Similarly, the encoded image R shown in FIG.
The reference image (Ln-1, Rn-1) shown on the left side of FIG.
n) and the reference image (Ln + 1, Rn + 2) shown on the right side of FIG.
Is used as a search area, and on both the left and right sides, a motion vector search range obtained by moving the search center by half a screen is added to a motion vector search range centered on the same position as the encoded image in the reference image to obtain a motion vector. . In the case where the coded block of the coded image is in the left half region and in the right half region, the case where the search is performed without reducing the search region without changing the search region is more effective in reducing the code amount. Has a great effect.

[0026]

According to the three-dimensional image coding apparatus of the present invention, a one-system coding apparatus encodes three-dimensional image data by using a parallax image and expanding a search range to increase prediction efficiency. Can be done. In the three-dimensional image decoding apparatus of the present invention, the decoding part of the apparatus is widely used MPE.
Since a general-purpose LSI compliant with G can be used, a stereoscopic image decoding device can be manufactured at low cost.

[Brief description of the drawings]

FIG. 1 is a diagram showing an embodiment of a three-dimensional image high efficiency coding apparatus according to the present invention.

FIG. 2 is a diagram illustrating an embodiment of a stereoscopic image decoding device according to the present invention.

FIG. 3 is a diagram illustrating an example of a conventional stereoscopic image encoding device.

FIG. 4 is a diagram illustrating a frame configuration of an encoding unit according to the present invention when the encoding unit is IBPB.
FIG. 10 is a diagram illustrating a search area for detecting a motion vector of a B frame at the time of PBP.

FIG. 5 is a diagram illustrating a frame configuration of an encoding unit according to the present invention when the encoding unit is IBPB;
FIG. 10 is a diagram illustrating a search area for detecting a motion vector of a B frame at the time of PBP.

FIG. 6 is a diagram illustrating a frame configuration of an encoding unit according to an embodiment of the present invention;
FIG. 10 is a diagram showing a search area for detecting a motion vector of a P frame at the time of PBP.

FIG. 7 is a diagram illustrating a frame configuration of an encoding unit according to an embodiment of the present invention.
FIG. 10 is a diagram showing a search area for detecting a motion vector of a P frame at the time of PBP.

FIG. 8 is a diagram illustrating a frame configuration of an encoding unit according to the present invention when IPPP is used;
FIG. 10 is a diagram illustrating a search area for motion vector detection of a P frame at the time of PPP.

FIG. 9 is a diagram illustrating an example in which the frame configuration of the encoding unit according to the present invention is IPPP.
FIG. 10 is a diagram illustrating a search area for motion vector detection of a P frame at the time of PPP.

[Explanation of symbols]

 1, 2 A / D conversion unit 3, 29 delay unit 4, 5 image processing unit 6 block division processing unit 7 motion vector detection unit 8 subtraction unit 9 orthogonal transformation unit 10 quantization unit 11, 22 inverse quantization unit 12, 23 Inverse orthogonal transformer 13, 24 Adder 14, 25 Frame memory 15 Variable length encoder 16 Multiplexer 17, 20 Disk (recording medium) 21 Variable length decoder 26 Image separator 27, 28 Oversampler L Left System disparity signal R Right system disparity signal

Claims (7)

[Claims] 1. A stereoscopic image high-efficiency encoding device using a parallax image, comprising: a delay device for delaying one of left and right parallax images input simultaneously to the encoding device for one image period; and the delay device. And an image processing unit that divides the image delayed by one image period and the other undelayed image into left and right or top and bottom within one image and combines them, and encoding that encodes the image combined by the image processing unit The encoding unit performs encoding conforming to the MPEG system, and in the case of detecting a motion vector of a P frame or a motion vector of a B frame, the encoding unit has the same position as the encoded image in the reference image. Stereoscopic motion characterized by obtaining a motion vector by adding a motion vector search range that has moved the search center by half a screen to a motion vector search range centered on Image height efficiency coding apparatus. 2. A high-efficiency stereoscopic image encoding apparatus using a parallax image, comprising: a delay device for delaying one of left and right parallax images input simultaneously to the encoding device by one image period; and the delay device. And an image processing unit that divides the image delayed by one image period and the other undelayed image into left and right or top and bottom within one image and combines them, and encoding that encodes the image combined by the image processing unit The encoding unit performs encoding conforming to the MPEG system, and in the case of detecting a motion vector of a P frame, a half of the motion vector search range centered on the same position as the encoded image in the reference image. A motion vector is obtained by adding a motion vector search range that has moved the search center for the screen, and in the case of detecting a motion vector of a B frame, the same is included in the reference image. On the screen where the temporal disparity image exists, a motion vector range obtained by moving the search center for a half screen to the motion vector search range centered on the same position as the encoded image in the reference image is obtained to obtain a motion vector. In a screen in which no parallax image of the same time exists in an image, a motion vector is obtained in a motion vector search range centered on the same position as the coded screen in the reference image, and a three-dimensional moving image high-efficiency coding apparatus characterized in that . 3. The three-dimensional moving picture high efficiency coding apparatus according to claim 1, wherein the number of pixels at the time of combining the two parallax images into one image is reduced to 入 力 of the input image. A three-dimensional moving picture high-efficiency coding apparatus characterized by being dropped and synthesized. 4. The stereoscopic moving picture high-efficiency encoding apparatus according to claim 1, wherein an index for indicating a parallax image delayed in the input processing of the parallax image includes user data of an MPEG stream. A three-dimensional moving picture high-efficiency coding apparatus characterized by writing in an area. 5. A stereoscopic video efficient decoding apparatus for decoding a bit stream encoded by the stereoscopic video efficient encoding apparatus according to claim 1 or 2, wherein the bit stream is extracted by a decoding unit. An image in which two parallax images are combined on one screen is separated into two parallax images by an image separation unit, and a parallax image that does not pass through a delay at the time of encoding is passed through a delay unit having a delay of one image unit. A high-efficiency stereoscopic video decoding apparatus for reproducing parallax images at the same time on the left and right. 6. A stereoscopic image high-efficiency encoding method using a parallax image, wherein one of left and right parallax images input simultaneously to the encoding device is delayed by one image period, and is delayed by one image period. And one of the non-delayed images is divided into left and right or up and down in one image, and the combined image is encoded, the combined image is encoded, and the encoding is performed according to the MPEG method. A motion vector is obtained by adding a motion vector search range obtained by moving the search center by half a screen to a motion vector search range centered on the same position as the encoded screen in the reference image,
In the frame motion vector detection, on a screen where a parallax image at the same time exists in the reference image, the search center is moved by a half screen to a motion vector search range centered on the same position as the encoded image in the reference image. A three-dimensional moving image high-efficiency encoding method, wherein a motion vector is obtained by adding a motion vector range. 7. A stereoscopic video high-efficiency decoding method for decoding a bit stream encoded by the stereoscopic video high-efficiency encoding method according to claim 6, wherein two parallax images synthesized on one screen are combined. A stereoscopic video high-efficiency decoding method, characterized in that a parallax image which has been separated and not passed through a delay at the time of encoding is delayed by one image unit, and the left and right parallax images are reproduced at the same time.