JP2001285895A - Image data coder and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the image quality of three-dimensional image data with a desired data quantity at an area close to a desired view point in the case of coding the three-dimensional image data. SOLUTION: Parallax information is detected from right and left eye images, Remote near information of the respective right and left eye images is detected depending on the magnitude of the obtained parallax information. The image area is divided into an area close to the view point and an area remote from the view point, and coding parameters (a quantization value and a motion retrieval range) are set and controlled to decrease a data compression rate for the near area and to increase the data compression rate for the remote area on the contrary.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image data encoding apparatus and method for encoding a plurality of types of multi-view image data such as three-dimensional image data.

[0002]

2. Description of the Related Art Conventionally, when encoding three-dimensional image data, one of the image data, for example, a right channel (hereinafter referred to as "Rch") image data is referred to, and the other data, for example, a left channel (hereinafter referred to as "Rch"). The image data is encoded.

In general, when processing a three-dimensional image, since the left image and the right image have a high correlation, Rch encoding can be performed using the Lch encoding result.

As shown in FIG. 1, an Lch image and Rc
The image of h has a high correlation, and when coding the portion A of Rch in the figure, the most similar portion B is detected in the area C near the position A ′ corresponding to A, and A ′ and B Displacement (M
V) and the difference value (AB).

Therefore, encoding of a three-dimensional image is performed according to FIG. In FIG. 2, Lch image data is encoded by an Lch encoding unit, and Rch image data is encoded by correlating with decoded image data obtained by decoding the encoding result of the Lch encoding unit. After that, Lch, R
The image code data coded for each channel is multiplexed by the multiplexing unit and output as Lch and Rch coded data.

When the MPEG standard is used for encoding, a multi-view image profile (Multi-view) is used.
Profile), and the encoding is performed according to FIG. In FIG. 3, the Lch image data and the Rch image data are DCT-transformed and quantized. Thereafter, the image data is encoded by a variable length encoding unit, multiplexed by a multiplexing unit, and output as Lch and Rch image code data. The image data quantized by the quantization unit is decoded (inverse quantization, inverse DCT transform) and written to the frame memory. The motion compensator for detecting the motion (motion vector) of the inter-frame data and the parallax compensator for detecting the disparity (disparity vector) between the Lch image data and the Rch image data convert the decoded image data corresponding to the input image data into the decoded image data. read out.

After that, the difference between the input image data and the corresponding decoded image data is calculated and supplied to the DCT unit. Further, the quantization control unit outputs L, R output from the multiplexing unit.
The quantization values of the Lch and Rch quantization units are controlled by the ch image code data.

[0008]

Generally, in the case of a three-dimensional image, as shown in FIG. 4, an area close to the viewpoint has a large movement between data that changes with time and is sharper.
The code amount increases.

However, as is apparent from FIG.
In the quantization control, the quantization value, which is an encoding parameter, is variable-length coded, and the quantization value is appropriately controlled from the multiplexed Lch and Rch image code data to a desired data amount. That is, the output Lch,
When the Rch image code data amount is equal to or more than the desired data amount, the quantization value in the quantization unit is increased to increase the data compression degree.

When performing motion compensation for detecting the motion (motion vector) of image data between frames, motion compensation is performed from a fixed motion search range.

For this reason, when image data in a region close to the viewpoint is encoded, since the motion is large, the search range is insufficient with a fixed motion search range, and the encoded data amount of the difference value increases. Also, since the image is sharper, the amount of coded data increases.

Therefore, in the quantization value control, the quantization value of the region close to the viewpoint becomes large with a desired Lch and Rch image code data amount, which causes a problem that the image quality becomes low.

[0013]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems of the prior art, the present invention firstly encodes a second image data by referring to the first image data. An apparatus for encoding eye image data,
Perspective information detecting means for obtaining perspective information of an image from the first image data and the second image data; and first encoding of the first image data based on the perspective information obtained by the perspective information detecting means. First encoding parameter setting means for setting a parameter, second encoding parameter setting means for setting a second encoding parameter of the second image data in consideration of the perspective information, It is characterized by having a first encoding unit for encoding and a second encoding unit for encoding the second image data. Instead of setting coding parameters (motion search range and quantization value) only with fixed values or coded and multiplexed image code data as in the related art, the coding is performed in consideration of the perspective information of the image. By setting the optimization parameter, it is possible to improve the image quality of an area close to the viewpoint.

Here, the perspective information detecting means detects the perspective information of the image based on the parallax information of the first image data and the second image data, and a first encoding parameter setting means and a second encoding parameter setting means. The means sets a motion search range parameter, which is an encoding parameter, based on the perspective information, and sets a quantization value, which is an encoding parameter, based on the perspective information and the coded image code data. Further, the first image data and the second image data can be Rch image data and Lch image data.

Further, the present invention is a method of encoding a plurality of multi-view image data by encoding a second image data with reference to the first image data, wherein
A perspective information detecting step of obtaining perspective information of an image from the image data and the second image data, and a first encoding parameter of the first image data based on the perspective information obtained in the perspective information detecting step. A first encoding parameter setting step to be set; and a second encoding parameter setting step for setting the second encoding parameter of the second image data in consideration of the perspective information.
An encoding parameter setting step, a first encoding step of encoding the first image data, and a second encoding step of encoding the second image data are provided. Also according to the present invention, instead of setting coding parameters (motion search range and quantization value) only with fixed values or coded and multiplexed image code data as in the related art, the perspective information of the image is considered. By setting the encoding parameters in this way, it is possible to improve the image quality of an area close to the viewpoint.

Here, the perspective information detecting step detects the perspective information of the image based on parallax information between the first image data and the second image data, and sets a first encoding parameter setting step and a second encoding parameter. The parameter setting step sets a motion search range parameter, which is an encoding parameter, based on the perspective information, and is encoded with the perspective information.
A quantum value which is an encoding parameter is set by the multiplexed image code data.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings, taking as an example a case where two types of multi-view image data (Lch image data and Rch image data) are encoded based on the MPEG standard.

FIG. 5 is a block diagram showing the overall configuration of the image data encoding apparatus according to this embodiment. A preprocessing unit 1, a perspective information detecting unit 2, an Lch encoding parameter setting unit 3, an Lch encoding unit 4, an Rch encoding parameter setting unit 5, an Rch encoding unit 6, and a multiplexing unit 7 are provided. Image data and Rch image data are input and compression-encoded.

The pre-processing unit 1 encodes the image data in accordance with the MPEG standard.
A macroblock of 16 pixels is formed. The perspective information detecting unit 2 detects perspective information of the image from the Lch image data and the Rch image data, and supplies the perspective information of the Lch and Rch image data to the Lch and Rch encoding parameter setting units 3 and 5. I do.

The Lch and Rch encoding parameter setting units 3 and 5 respectively calculate Lch and Rch based on the perspective information detected by the perspective information detecting unit 2 and the data amount of the Lch and Rch encoded data multiplexed by the multiplexing unit 7. The coding parameters are fed back to the coding units 4 and 6 of the respective Rch. Further, the Rch encoding unit 6 encodes the Rch image data with reference to the Lch image code data encoded by the Lch encoding unit 4. Therefore, the perspective information detecting unit 2 supplies the disparity vector to the Lch encoding unit 4 and the Lch decoding image data to the Rch encoding unit 6.

FIG. 6 is a block diagram showing the configuration of the perspective information detector 2.

First, the input Lch image data and Rc
The disparity detection unit 11 searches the h image data for a macroblock of the Lch image data that best matches the macroblock of the Rch image data, and detects a disparity vector. Next, the distance determination unit 12 determines Lch, Rc for each macroblock according to the magnitude of the disparity vector as shown in FIG.
h Perspective information is assigned to each of the image data.

FIG. 8 shows an Lch encoding parameter setting unit 3
FIG. Motion search range setting unit 2
In FIG. 1, the motion search range setting parameter is increased for a macroblock close to the viewpoint with respect to the input Lch perspective information, and conversely, the motion search range is set for a distant macroblock. Set a smaller setting parameter. In addition, the quantization value setting unit 22 determines a quantization value so that the input Lch and Rch encoded data amounts become a desired data amount, and calculates the quantization value for each macroblock from the input perspective information. Perform weighting.

The configuration of the Rch encoding parameter setting section 5 shown in FIG. 5 is the same as that of the above-described Lch encoding parameter setting section 3, and therefore, the description thereof is omitted.

FIG. 10 is a block diagram showing the configuration of the Lch encoder 4. According to the MPEG standard, forward prediction encoding, backward prediction encoding, and bidirectional prediction encoding are performed. For this reason, the difference calculator 31 calculates a difference from a macroblock that has been searched for as a best match in a temporally later screen or a temporally earlier screen. In the case of an intra macroblock, the difference is not supplied to the DCT unit 32 without performing the difference calculation.

The difference value or the intra macroblock image data is converted into a DC by the DCT unit 32 in units of 8 × 8 pixel blocks.
The DCT coefficient obtained by the T-transform is supplied to the quantization unit 33. The quantization unit 33 quantizes using the quantization value supplied by the Lch encoding parameter setting unit 3 and supplies the result to the variable length encoding unit 34. The variable-length encoding unit 34 performs variable-length encoding in which the number of consecutive zero data and the level of non-zero data are coded together.

On the other hand, the quantized DCT coefficients are also supplied to an inverse quantization unit 35 and an inverse DCT unit 36 for decoding, and the decoded image data is stored in a frame memory 37. If the decoded image data is the difference amount in the inverse quantization unit 35 and the inverse DCT unit 36, the decoded image data in the frame memory 37 is stored after being added by the adder 38. The decoded image data decoded in this manner is transmitted to the motion compensation unit 39 and R
This is supplied to the channel encoding unit 6.

The motion compensator 39 supplies the macroblock that best matches the current macroblock (image data to be encoded) by the Lch encoding parameter setting unit 3 based on the input image data to be encoded and the decoded image data. A search is performed using the obtained motion search range parameter to detect a motion vector. The detected motion vector is supplied to the variable length coding unit 34.

A corresponding macroblock specified by a motion vector is output from the decoded image data in the frame memory 37, and a difference from the current macroblock is calculated by a differentiator 31.

FIG. 11 is a block diagram showing the configuration of the Rch encoder 6. The configuration of the Rch encoding unit 6 is the
It is almost the same as the ch encoding unit 4, but the Rch encoding unit 6
In the same manner as in the Lch encoding unit 4, compression encoding is performed by using a motion vector and a difference value between temporally preceding and succeeding image data. And encoding based on the difference value.
Therefore, when the decoded image data decoded by the inverse quantization unit 35 and the inverse DCT unit 36 is a parallax difference amount, the decoded image data in the frame memory 37 is stored after being added by the adder 42. The decoded image data decoded in this way is supplied to the parallax compensation unit 43.

The disparity compensator 43 supplies the disparity vector supplied from the perspective information detector 2 to the variable length encoder 34. Further, a corresponding macroblock specified by a disparity vector is output from the decoded image data in the frame memory 37,
The difference from the current macroblock is calculated by the differentiator 41. Thereafter, similarly to the Lch encoding unit 4, the signal is supplied to a differentiator 31 for performing forward prediction encoding and backward prediction encoding.

FIG. 12 shows a flow of processing in the encoding method according to the present embodiment. As a whole configuration,
Preprocessing step 51, perspective information detection step 52, Lc
h coding parameter setting step 53, Lch coding step 54, Rch coding parameter setting step 5
5, an Rch encoding step 56, and a multiplexing step 57. The processing of each step is the same as that of the above-described image data encoding apparatus, and thus the description thereof is omitted.

In the present embodiment, two types of multi-view image data are used. However, it is needless to say that the present invention can be applied to any coding method for coding any n types of multi-view image data. A method other than the MPEG method is also possible.

[0034]

As described above, when encoding a plurality of types of multi-view image data according to the present invention, it is possible to perform encoding without deteriorating the image quality of a region close to the viewpoint in a desired data amount. It becomes possible.

[Brief description of the drawings]

FIG. 1 shows Lch image data and Rch of three-dimensional image data.
FIG. 4 is an explanatory diagram illustrating a relationship between image data.

FIG. 2 is an overall configuration block diagram of a conventional technique.

FIG. 3 is a configuration block diagram of a conventional encoding device.

FIG. 4 is a diagram for explaining a problem of the related art.

FIG. 5 is an overall configuration block diagram of an image data encoding device according to an embodiment of the present invention.

FIG. 6 is a block diagram illustrating a configuration of a perspective information detection unit in FIG. 5;

FIG. 7 is an explanatory diagram of assignment of perspective information.

FIG. 8 is a block diagram illustrating a configuration of an encoding parameter setting unit in FIG. 5;

FIG. 9 is an explanatory diagram of encoding parameter setting.

FIG. 10 is a block diagram illustrating a configuration of an Lch encoding unit in FIG. 5;

FIG. 11 is a block diagram illustrating a configuration of an Rch encoding unit in FIG. 5;

FIG. 12 is a diagram illustrating a processing procedure of an image data encoding method according to the embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 1 Preprocessing unit, 2 Perspective information detection unit, 3 Lch coding parameter setting unit, 4 Lch coding unit, 5 Rch coding parameter setting unit, 6 Rch coding unit, 7 multiplexing unit, 11 Parallax detection unit, 12 Perspective Determination unit, 21 motion search range setting unit, 22 quantization value setting unit, 31 difference unit, 32 DCT unit, 33 quantization unit, 34 variable length coding unit, 35 inverse quantization unit, 36 inverse DCT unit, 37
Frame memory, 38 adder, 39 motion compensator, 4
1 Differentiator, 42 Adder, 43 Parallax Compensator, 51 Preprocessing Step, 52 Perspective Information Detection Step, 53 L
ch coding parameter setting step, 54 Lch coding step, 55 Rch coding parameter setting step, 56 Rch coding step, 57 multiplexing step

 ──────────────────────────────────────────────────続 き Continued on front page F-term (reference) BC02 BC08 BD01

Claims (8)

[Claims] An apparatus for encoding a plurality of multi-view image data by encoding a second image data while referring to a first image data, wherein the first image data and the second image data are encoded. Perspective information detecting means for obtaining perspective information of an image from the image data of the first, and a first code for setting a first encoding parameter of the first image data in consideration of the perspective information obtained by the perspective information detecting means. Encoding parameter setting means, second encoding parameter setting means for setting a second encoding parameter of the second image data in consideration of the perspective information, and the first image according to the first encoding parameter. Image data encoding apparatus comprising: first encoding means for encoding data; and second encoding means for encoding the second image data according to the second encoding parameter. Location. 2. The apparatus according to claim 1, wherein the perspective information detecting means includes a first image data and a second image data.
An image data encoding device for detecting perspective information based on disparity information of the image data. 3. The apparatus according to claim 1, wherein the first coding parameter setting means and the second parameter setting means determine a motion between image data frames, which is a coding parameter, based on the perspective information. A motion search range parameter for detection is set, and based on the perspective information and the image code data encoded by the first encoding unit and the second encoding unit, an encoding parameter, An image data encoding device, wherein a quantization value parameter is set. 4. The apparatus according to claim 1, wherein the first image data and the second image data are right-eye (right channel) image data and left-eye (left channel) image data. An image data encoding device, characterized in that: 5. A method of encoding a plurality of multi-view image data by encoding a second image data while referring to the first image data, wherein the first image data and the second image data are encoded. A perspective information detecting step of obtaining perspective information of an image from image data; and a first encoding parameter for setting a first encoding parameter of the first image data in consideration of the perspective information obtained in the perspective information detecting step. A setting step; a second encoding parameter setting step of setting a second encoding parameter of the second image data in consideration of the perspective information; and encoding the first image data according to the first encoding parameter. A first encoding step of encoding the image data, and a second encoding step of encoding the second image data according to the second encoding parameter. Over data encoding method. 6. The image data according to claim 5, wherein the perspective information detecting step detects perspective information based on disparity information between the first image data and the second image data. Encoding method. 7. The method according to claim 5, wherein the first encoding parameter setting step and the second parameter setting step include a step of determining a motion between image data frames, which is an encoding parameter, based on the perspective information. A motion search range parameter at the time of detection, a quantization value at the time of encoding as an encoding parameter, based on the perspective information and the image code data encoded at the first encoding step and the second encoding step. An image data encoding method comprising setting parameters. 8. The method according to claim 5, wherein the first image data and the second image data are right (right channel) image data and left (left channel) image data. An image data encoding method characterized by the above-mentioned.