JP2000165909A - Method and device for image compressing processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently predict a motion. SOLUTION: Concerning the block of a current position, which is a processing object, in a right eye image, the object block of a reference left eye image is obtained. There is a parallax between right and left eye images. In the parallel style of arranging two cameras parallel, for example, an object watched in the right eye image looks moving to the left side in the left eye image without fail. Then, the search range of the correspondent block is limited to the left side of the correspondent position.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image compression process for a 3D (three-dimensional) image including a right-eye image and a left-eye image, and more particularly to an improvement in association between images.

[0002]

2. Description of the Related Art In moving image processing consisting of continuous images,
Images are obtained at short time intervals. Therefore, there is considerable correlation between images that are close in time. Therefore, for image data that is close in time, the amount of data can be reduced by using this as difference data. Further, changes in an image over time often appear as movements in the image. Therefore, the correspondence between the images is performed using the motion vector, and the efficiency when the difference data is obtained is improved.

For example, a block of a predetermined size is set for an image of interest, the most similar block is searched for in an adjacent image, and difference data is obtained between the corresponding blocks. The correspondence between the two blocks is converted into data as a motion vector.

As described above, by using the motion vector, the efficiency of the image compression processing can be improved. Therefore, the motion compensation process using the motion vector
It is widely used for compression processing of moving images such as MPEG.

[0005] Further, 3D displays and the like are becoming widespread, and 3D moving images are also widely handled. This 3D image is created using a right-eye image and a left-eye image captured by two cameras having normal parallax. Then, even in such a 3D image, image processing using the same motion vector as described above is performed.

[0006]

Here, conventionally, 3D
No special consideration is given to motion compensation in the image. Therefore, the detection of the motion vector has not been performed sufficiently efficiently.

The present invention has been made in view of the above problems, and has as its object to perform image compression processing more efficiently on 3D images.

[0008]

SUMMARY OF THE INVENTION The present invention provides an image compression processing method and apparatus for compressing a 3D image including a right-eye image and a left-eye image, and a method for determining a corresponding portion between the right-eye image and the left-eye image. In addition, a search range for finding a corresponding portion is determined in consideration of a direction based on a parallax between a right-eye image and a left-eye image.

As described above, in the present invention, a search range for determining a corresponding portion is determined in consideration of directions based on parallax between a right-eye image and a left-eye image. The images in the right-eye image and the left-eye image have a certain relationship based on the line of sight. For example, in the case of three-dimensionalization using a parallel type or a three-dimensional type using an intersection type to give depth, the object in the right-eye image is always located at the position moved leftward in the left-eye image. Therefore, it is possible to limit the search target range when detecting the corresponding portion, and it is possible to perform an efficient search.

For example, when searching for a corresponding block in the left-eye image for a predetermined portion (block) of the right-eye image, the search range is limited to the left side of the directly corresponding position, thereby making the search range small and efficient. Search can be performed. Also, as the search range becomes smaller, the change width (dynamic range) of the motion vector indicating the corresponding portion also becomes smaller. Therefore, the table used when encoding the motion vector can be made small.

When a corresponding portion between the right-eye image and the left-eye image is obtained, it is preferable to obtain a motion vector based on the corresponding portion and use the obtained motion vector in image compression. is there. In this way, highly efficient image compression can be achieved using motion vectors.

In addition, it is preferable that the search is performed such that the origin is set as the starting point and the distance is gradually increased from the origin within the obtained search range. When the search is performed by software, the search time can be shortened by performing the search in such a search order, and the processing speed can be improved.

[0013]

Embodiments of the present invention (hereinafter referred to as embodiments) will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the overall configuration of an image compression processing apparatus according to the present invention.

A video 1 input (eg, a left channel for a left eye image), which is an image signal for the base layer, from one camera is first input to the A / D converter 10.
Here, it is converted into digital data. Next, the data is input to the format converter 12, where the data format is converted so as to be compatible with the subsequent digital image data processing. The image data that has been subjected to the format conversion is input to the screen rearranging unit 14 and rearranged according to the type of screen (frame). That is, in MPEG, depending on a frame, an I picture for intra-frame coding, a P picture for forward prediction (prediction from a preceding frame), and a bi-directional prediction inserted between I and P pictures. Since there is a B picture to be coded, rearrangement is performed to delay the B picture compared to the I and P pictures that refer to the coding.

The rearranged image data is supplied to a subtractor 16.
, Where the difference from the preceding image is calculated. This difference is made between the reference macroblock and the corresponding macroblock based on a comparison for each macroblock with the image of the reference frame. The obtained difference data is subjected to a discrete cosine transform in a DCT (discrete cosine transform) unit 18, quantized to predetermined bits by a quantizer 20, variable-length coded by a variable-length coding unit 22, and temporarily stored in a buffer 24. After being memorized,
Output as encoded data. The buffer 24 provides the free space information to the quantizer 20 via the rate control unit 26, and the quantizer 20 controls the number of quantization bits based on the free space information.

The output of the quantizer 20 is returned by the inverse quantizer 28 and the inverse DCT unit 30 to data before being subjected to discrete cosine transform and quantization. Then, this data is supplied to the video memory 34 via the adder 32. The adder 32 adds image data when the output of the inverse DCT unit 30 is difference data, and the output of the adder 32 is image data. Then, the decoded image data is supplied to the video memory 34 and stored.

The data in the video memory 34 is supplied to a motion compensation prediction unit 36. The current image data from the screen rearranging unit 14 is also supplied to the motion compensation prediction unit 36, where the comparison is performed for each macroblock,
A motion vector is calculated. Here, in some cases, it is better to perform motion prediction, but in other cases, it is better to perform intra-frame coding. Therefore, the motion compensation prediction unit 36 also determines this prediction mode. Then, the data on the motion vector and the prediction mode is
To supply. The variable length encoding unit 22 also encodes the supplied motion vector and prediction mode.

The output of the motion compensation prediction unit 36 is
8, and is supplied to the adder 32.
When the output from the inverse DCT unit 30 is difference data, it is supplied to the adder 32. In the case of a B picture, there is a possibility of bidirectional prediction, and the switch 38 is controlled accordingly. Image data including one frame of motion compensation prediction output from the motion compensation prediction unit 36 is supplied to the subtractor 16. Therefore, in the adder 32, reconstruction of the image data including the motion compensation prediction is performed. In addition,
When intra-frame encoding is performed, such as when the input data is an I picture, the adder 3
No. 2 is bypassed, so it does not become difference data. At this time, the switch 38 is opened, and the adder 32 processes the output from the inverse DCT unit 30 as it is.

A video 2 input, which is an output of an enhancement layer from another camera, is similarly processed by an A / D converter 50 and a format converter 52. Then, the timing with the video 1 input is directly supplied to the subtracter 54 via the delay buffer 53 for adjustment without rearranging the screens. The output of the subtractor 54 is output as coded data via a DCT unit 56, a quantizer 58, a variable length coding unit 60, and a buffer 62. The output of the quantizer 58 is supplied to a motion compensation prediction unit 72 via an inverse quantizer 64, an inverse DCT unit 66, an adder 68, and a video memory 70. The motion compensation prediction unit 72 is also supplied with the image data from the format conversion unit 52, where the motion prediction is performed. The motion vector and the prediction mode of the prediction result are supplied to the variable length coding unit 60. The image data based on the prediction is supplied to the subtractor 54. Further, the image data based on the prediction is supplied to an adder 68 via a switch 74, and is added to the difference data so that the video memory 70
Is stored as image data. A switch 76 that bypasses the subtractor 54 is also provided. The motion compensation prediction unit 72 is supplied with image data from the video memory 34 in the base layer, and the motion compensation prediction unit 72 performs prediction using the image data in the base layer. Note that the rate control unit 78 controls the number of quantization bits in the quantizer 58 according to the remaining memory capacity of the buffer 62.

FIG. 2 shows a motion compensation prediction unit 3 of the base layer.
6 is shown. As described above, the current (current) input is input to the two vector detection units 110 and 112. The vector detection unit 110 performs forward prediction (forward) with reference to a preceding frame, and the vector detection unit 112 performs backward prediction (back) with reference to a subsequent frame. The vector detection unit 110 supplies the vector information and the evaluation value to the prediction mode determination unit 114. Here, the evaluation value indicates the absolute value sum of the difference between the corresponding blocks for each pixel. Then, the prediction mode determination unit 114 determines a prediction mode based on the vector evaluation value.

The vector detectors 110 and 112 output prediction data which is difference data of the corresponding block obtained from the current input and the reference input. This prediction data is supplied to the selector 118 via the adder 116 and directly. Then, based on the prediction mode determined by the prediction mode determination unit 114, the selector 118 performs forward prediction data from the vector detection unit 110, backward prediction data from the vector detection unit 112, and bidirectional prediction data from the adder 116. Select and output. Note that the bidirectional prediction data from the adder 116 is supplied to the evaluation value calculation unit 119, where the evaluation value of bidirectional prediction is calculated, and this is supplied to the prediction mode determination unit 114. Therefore, the prediction mode determination unit 114 can determine three predictions.

FIG. 3 shows the motion compensation prediction unit 7 of the reference layer.
2 is shown. As described above, the motion compensation prediction unit 72 includes the three vector detection units 120, 122, and 124.
have. This is because image data of three frames of (i) the nearest base layer, (ii) the base layer immediately after, and (iii) the closest enhancement layer are input as reference inputs. Three vector detectors 120, 1
22 and 124 obtain vector information, vector evaluation values and prediction data, respectively. Then, the vector evaluation value is sent to the prediction mode determination unit 126, where the prediction mode is determined.

The prediction data from the vector detectors 120, 122, and 124 are supplied to three adders 128, 130, and 13 respectively.
After the addition is performed two by two in the selector 2, the selector 13
4 is supplied. Also, the vector detection units 120 and 12
The prediction data from 2 and 124 are supplied to selector 13
4 is supplied. Then, the selector 134 outputs one of the six input signals according to the determination result of the prediction mode determination unit 126.
Choose one. In this way, prediction data for selecting one of the three or two of the three as described above is obtained.

The outputs of the adders 128, 130, and 132 are supplied to evaluation value calculation units 129, 131, and 133, where the evaluation values of bidirectional prediction are calculated. 126. Therefore, the prediction mode determination unit 126 can determine the prediction mode.

FIG. 4 shows reference frames in the motion compensation prediction units 36 and 72. As described above, the prediction of the base layer is the same as that of the related art. The I picture has no prediction, the P picture has forward prediction, and the B picture has bidirectional prediction.
On the other hand, in the enhancement layer, image data of the base layer is also referred to. That is, the motion compensation prediction unit 72 includes:
Three frames of image data are supplied: (i) the closest base layer a, (ii) the base layer immediately after, and (iii) the closest enhancement layer c. If the picture is a P picture, one of the three directions is selected, and if the picture is a B picture, two of the three directions are selected for prediction.

[Detection of Motion Vector] As described above, the motion compensation prediction unit 72 that processes the image data of the reference layer detects the motion vector using the image data of the base layer. Here, the base layer and the reference layer are
It corresponds to a left-eye image and a right-eye image. Therefore, there is a parallax between the two images based on the camera position.

Here, as shown in FIG. 5, there is a parallel type of camera arrangement, and this parallel type makes the viewing directions of the two cameras parallel.

Also, there are two types of 3D images: (A) three-dimensionalization for giving depth to the screen, and (B) three-dimensionality for making the screen appear to protrude from the screen.

In the parallel type, as shown in FIG. 6, the object in the left-eye image moves leftward in the right-eye image regardless of the depth in the image at the time of shooting. Further, when there are a plurality of objects and there is a difference in the depth, the difference between the left and right images is larger for the nearer object.

On the other hand, when the image is projected on the screen, the position of the object to be projected on the screen changes depending on whether the object is to be provided with depth or to be protruded. That is, in the case of three-dimensionalization having depth, the target in the projection of the left-eye image viewed by the left eye is shifted to the left as compared to the target of the right-eye image viewed by the right eye. On the other hand, in the case of three-dimensional projection in which the object is projected, the object in the projection of the left-eye image viewed by the left eye is shifted to the right side as compared with the object of the right-eye image viewed by the right eye.

In this embodiment, this information is used for detecting a motion vector. That is, in the case of three-dimensionalization having depth, the object of the right-eye image is
Be sure to be in the position to the left. Therefore, as shown in FIG. 8, the search target range for comparison when detecting a motion vector is limited. That is, the right-eye image is the current image, and when searching for the corresponding block in the left-eye image referred thereto, the search range is limited to a predetermined range on the left side from the same position on the screen.

For example, conventionally, one macroblock in the target right-eye image is moved ± 31 pixels left and right and ± 16 pixels up and down around the same position of the left-eye image to find a corresponding block. Wanted a vector. On the other hand, in the present embodiment, only the left portion is to be searched. That is, the range is limited to only the range of 0 to 31. Thus, the search range can be reduced, and the search time can be reduced. Also, the accuracy of the search remains the same.

Also, instead of limiting the search range,
Simply moving to the left can improve search accuracy. Further, the search range may be moved to the left so that the right side slightly remains, or the search range may be reduced so that the right side slightly remains.

In the normal case, one of the three-dimensional methods (A) and (B) is adopted, and it is rare that both are adopted. Therefore, the moving direction may be determined according to the three-dimensional method. That is, in the case of (A), the search range is moved to the left as described above, and in the case of (B), the search range is moved to the right.

"Encoding of motion vector data" As described above, encoding is also performed on a motion vector of a macroblock. In the present embodiment, the size of the motion vector is also limited by limiting the search range.
Therefore, this is used to improve the efficiency of the code.

Macroblock (MB) to be encoded
In coding the motion vector (MV), the difference value ΔMV from the motion vector MV ′ of the immediately preceding macroblock is used.
= (MV−MV ′) according to the table.

At the time of encoding, the detection range of the motion vector is determined for each frame, and the information is also encoded. For example, if the detection range is -16 to +15 pixels, the motion vector is -16 to +15 in the table.
Are converted to the codes respectively assigned. here,
The code length is assigned such that the closer the value of the motion vector is to 0, the shorter the code length becomes. At this time, the range of ΔMV is −31
To +31. This is also encoded using a table in the same manner, but -16 → -1, 0, + 1 → 15,
The same code is assigned to + 15 → + 31, −, −31 → −17. In this way, by using the same code according to the size of the data, the code can be used effectively.

In this embodiment, the search range is, for example, 0 to
Limited to 31 pixels. Therefore, the vector value is always -1.
By adding an offset of 6, the ΔMV becomes -16 to
It can be handled as a value in the range of +15. Therefore, encoding can be performed using only the above-mentioned table of -16 to +15. Further, it is not necessary to determine at the time of reproduction whether the encoding uses the large or small table.

Further, the table itself can be changed to make it more efficient. For example, if the range is 0 to 15, the vector difference value is converted from -15 to +15. By allocating predetermined codes to the vector difference values 0 to 15 and allocating the same codes to 1 to 15 to -15 to -1 as shown in Table 1, the table can be made smaller. Further, since the number of data is small, the number of bits of the code to be allocated may be small, and the average code length can be shortened.

[0041]

[Table 1] As described above, in the present embodiment, since the search range can be limited, the fluctuation range of the motion vector can be reduced. Therefore, it is possible to reduce the number of bits to be allocated in encoding and to reduce the size of the table for allocation.

"Search Order" The motion compensation prediction unit 72 as described above can be constituted by hardware. In this case, hardware is prepared to perform a predetermined search. Therefore, the processing amount within a certain time is restricted by hardware, but the search order is not affected if the processing amount is within the range of the restriction.

On the other hand, when the search is performed by software, the overall processing speed changes depending on whether the optimum vector can be detected quickly. Therefore, by ending the search immediately when an appropriate motion vector is detected, the processing speed can be improved.

Therefore, in the motion compensation predictor 72 according to the present embodiment, as shown in FIG. 9, a search may be performed starting from the origin and gradually becoming farther from the origin within the search range. In particular, FIGS. 9A and 9B
As shown in the figure, the spiral type is biased to one side. Also,
As shown in FIG. 9C, it is also preferable to divide the area according to the distance from the origin and to search in order for each area.

[0045]

As described above, in the present invention, a search range for finding a corresponding portion is determined in consideration of the direction based on the parallax between the right-eye image and the left-eye image. Therefore, it is possible to limit the search target range when detecting the corresponding portion, and it is possible to perform an efficient search.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of an image compression processing device according to the present invention.

FIG. 2 is a block diagram showing a configuration of a motion compensation prediction unit 36.

FIG. 3 is a block diagram illustrating a configuration of a motion compensation prediction unit 72.

FIG. 4 is a diagram showing a reference frame in motion prediction.

FIG. 5 is an explanatory diagram of three-dimensionalization.

FIG. 6 is an explanatory diagram showing movement of an object in an image at the time of shooting.

FIG. 7 is an explanatory diagram showing movement of a target in an image at the time of projection.

FIG. 8 is a diagram showing a search range.

FIG. 9 is a diagram showing a search order.

[Explanation of symbols]

10, 50 A / D converter, 12, 52 format converter, 14 screen rearranging unit, 16, 54 subtractor,
18, 56 DCT unit, 20, 58 quantizer, 22,
60 variable length coding unit, 24, 62 buffers, 26,
78 rate controller, 28, 64 inverse quantizer, 30,
66 inverse DCT unit, 32, 68 adder, 34, 70
Video memory, 36, 72 Motion compensation prediction unit.

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Akiomi Kunisaka 2-5-2-5 Keihanhondori, Moriguchi-shi, Osaka Sanyo Electric Co., Ltd. (72) Osamu Ota 2-chome Keihanhondori, Moriguchi-shi, Osaka No.5-5 Sanyo Electric Co., Ltd. F term (reference) 5C059 KK15 MA02 MA03 MA05 MA23 NN01 NN03 NN08 NN11 NN34 PP04 UA02 UA34 5C061 AA29 AB04 AB08 AB24

Claims (4)

[Claims] An image compression processing method for compressing a three-dimensional image including a right-eye image and a left-eye image, the method comprising: determining a corresponding portion between the right-eye image and the left-eye image; An image compression processing method comprising: determining a search range for obtaining a corresponding portion in consideration of a direction based on the search range. 2. An image compression processing apparatus for compressing a three-dimensional image including a right-eye image and a left-eye image, wherein a parallax between the right-eye image and the left-eye image is determined when a corresponding portion between the right-eye image and the left-eye image is obtained. An image compression processing apparatus for determining a search range for obtaining a corresponding part in consideration of a direction based on the search result. 3. The apparatus according to claim 2, wherein when a corresponding portion between the right-eye image and the left-eye image is obtained, a motion vector is obtained based on the corresponding portion.
An image compression processing device characterized in that the obtained motion vector is used at the time of image compression. 4. The apparatus according to claim 2, wherein a search is performed within the determined search range, starting from the origin determined for each pixel as a starting point and gradually moving away from the origin. Image compression processing device.