JPH09116930A - Method for converting two-dimensional video into three-dimensional video - Google Patents

Abstract

PROBLEM TO BE SOLVED: To stereoscopically view a subject moving to the left and a subject moving to the right by selecting one of an input video and a delay video as a picture element for the left eye, and selecting the other as the picture element for the right eye for the respective picture elements based on discrimination information and generating a video for the left eye and a video for the right eye. SOLUTION: A right/left video selection circuit 3 is controlled based on a delay quantity calculation circuit 6. When the code of delay quantity, which is obtained by the delay quantity calculation circuit 6, is positive, a right eye video selection circuit 33 selects the delay video (the output of an interpolation circuit 31), and a left eye video selection circuit 32 selects a through video. When the code of the delay quantity, which is obtained by the delay quantity calculation circuit 6, is negative, the right eye video selection circuit 33 selects the through video and the left eye video selection circuit 32 selects the delay video (the output of the interpolation circuit 31). Thus, the plural subjects different in moving directions can stereoscopically be viewed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention
The present invention relates to a method of converting a 3D image.

[0002]

2. Description of the Related Art As a method of converting a two-dimensional image into a three-dimensional image, a main image signal and a sub-image signal delayed in time with respect to the main image signal are generated based on the two-dimensional image signal. A method of outputting one as a left-eye video signal and the other as a right-eye video signal is known. However, with this method, in the case of an image in which there are a subject moving to the left and a subject moving to the right, it is difficult to stereoscopically view these subjects moving in different directions.

[0003]

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method for converting a two-dimensional image into a three-dimensional image capable of stereoscopically viewing both a left-moving subject and a right-moving subject. And

[0004]

A method of converting a two-dimensional image into a three-dimensional image according to the present invention relates to the size and direction of the movement of a subject for each area larger than a pixel unit based on a two-dimensional input image. The first step of detecting information, based on the information about the magnitude and direction of the movement of the subject detected for each area, which of the left-eye image and the right-eye image should be the delayed image for each pixel is determined. The second step of generating the identification information shown and calculating the delay amount for each pixel, based on the delay amount calculated for each pixel,
The third step of generating a delayed image delayed by the delay amount corresponding to each pixel with respect to the input image, and the input image for each pixel based on the identification information generated for each pixel And a delayed image, one of which is selected as a pixel for the left eye and the other of which is selected as a pixel for the right eye, thereby providing a fourth step of generating a left-eye image and a right-eye image. .

In the second step, for example, based on the information on the magnitude and direction of the motion of the subject detected for each area, the delayed video for each area must be either left-eye video or right-eye video. Generating the first identification information indicating the frequency and calculating the delay amount for each field for each area, and the first identification information generated for each area and the delay for each field calculated for each area Based on the amount, second identification information indicating whether the delayed image should be a left-eye image or a right-eye image for each pixel is generated, and a delay amount smaller than a field unit is calculated for each pixel. Those having a step of performing are used. In this case, in the third step, for example, by interpolating an image delayed for each field with respect to the input image, the delayed image delayed by the delay amount corresponding to each pixel with respect to the input image. Is generated.

According to the present invention, it is possible to provide images with different delay amounts for each pixel as left-eye images and right-eye images according to the direction and size of the movement of the subject in each area. Therefore, it is possible to stereoscopically view a plurality of subjects having different movement directions. Also, when the delay amount is changed in units of large areas, distortion may occur at the boundaries of the areas. According to the present invention, the amount of delay can be gently changed between small areas, so that distortion can be reduced.

[0007]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the configuration of a 2D / 3D image conversion device for converting a 2D image into a 3D image.

The two-dimensional input video signal is sent to the motion vector detection circuit 1, the delayed video generation circuit 2 and the left and right video selection circuit 3. The input image sent to the left and right image selection circuit 3 will be referred to as a through image.

The delayed video generation circuit 2 comprises a plurality of field memories 21, four field memories 21 in this example. In the first-stage field memory 21, the video one field before the through video is stored. These field memories 21 are connected in series,
Every time a video signal of a new field is input to the field memory 21 of the first stage, the contents of the field memory 21 of the preceding stage are transferred to the field memory 21 of the succeeding stage.

Therefore, at the time when a video signal of a new field is input to the 2D / 3D video converter, the first
In the field memory 21 in the second row, the video one field before the through video is stored. In the field memory 21 in the second row, the video two fields before the through video is stored. In the field memory 21 in the third row, the video three fields before the through video is stored. Field memory 2 in the 4th row
In No. 1, an image of four fields before the through image is stored. The output image of each field memory 21 is
It is an image delayed by the field unit with respect to the through image. The output image of each field memory 21 is sent to the left and right image selection circuit 3.

Video writing and video reading to and from each field memory 21 are controlled by the field memory control circuit 4 based on commands from the CPU 5.

The left / right video selection circuit 3 includes a left-eye video output terminal OL and a right-eye video output terminal OR, and one of the through video and the delayed video is output to one of the output terminals for each pixel. To the other output terminal.

The left and right video selection circuit 3 includes an interpolation circuit 31, a left-eye image selection circuit 32 and a right-eye image selection circuit 33 for generating a video delayed by a unit smaller than a field unit from the through video. There is.

The interpolation circuit 31 includes a first selection circuit 301, a second selection circuit 302, a first multiplier 303, and a second multiplier 30.
4 and an adder 305. First selection circuit 30
1 is a through image, an output image of the first-stage field memory 21, an output image of the second-stage field memory 21, an output image of the third-stage field memory 21, and a fourth-stage field memory 21. One of the output images of is selected and output.

The second selection circuit 302 includes an output video from the first-stage field memory 21, an output video from the second-stage field memory 21, an output video from the third-stage field memory 21, and a fourth-stage. One of the output images of the field memory 21 is selected and output.

The first multiplier 303 multiplies the output of the first selection circuit 301 by a coefficient (1-K). The second multiplier 304 multiplies the output of the second selection circuit 302 by the coefficient K. The adder 305 outputs the output of the first multiplier 303 and the second multiplier 30.
4 and the output of 4 are added.

The left-eye image selection circuit 32 selects one of the through image and the output of the interpolation circuit 31 and supplies it to the left-eye image output terminal OL. The right-eye image selection circuit 33 selects one of the through image and the output of the interpolation circuit 31, and outputs the right-eye image output terminal O.
Supply to R.

The left / right image selection circuit 3 includes a delay amount calculation circuit 6
Is controlled based on the The delay amount calculation circuit 6 includes a CPU 5
The delay amount is calculated for each pixel in units smaller than the field unit based on the delay amount for each of the plurality of divided areas sent from the unit.

In each field of the input video, the delay amount calculation circuit 6 outputs the delay amount for each pixel. The delay amount is composed of a code (positive / negative sign) indicating the direction of movement of the subject and a delay amount calculated based on the magnitude of the movement of the subject. The delay amount is composed of an integer part representing the delay amount in field units and a decimal part representing the delay amount in units smaller than the field unit. In this example, the sign is positive when the subject is moving from left to right, and is negative when the subject is moving from right to left. Further, the delay amount is determined so as to decrease as the movement of the subject increases.

A signal representing the integer part of the delay amount is sent to the first selection circuit 301 and the second selection circuit 302 as a control signal. When the integer part of the delay amount is 0, the first selection circuit 301 selects the through image, and the second selection circuit 302 selects the output image (1 field delayed image) of the first stage field memory 21. When the integer part of the delay amount is 1, the first selection circuit 301 selects the output image (1 field delay image) of the first-stage field memory 21, and the second selection circuit 302 sets the second-stage field memory. 21 output video (2 field delay video) is selected.

When the integer part of the delay amount is 2, the first selection circuit 301 selects the output video (2 field delay video) of the field memory 21 of the second stage, and the second selection circuit 3
02 is the output video (3
Field delay video). The integer part of the delay amount is 3
In the case of, the first selection circuit 301 selects the output image of the third stage field memory 21 (three-field delay image), and the second selection circuit 302 outputs the output image of the fourth stage field memory 21 (four fields). Delayed video) is selected. When the integer part of the delay amount is 4, the first selection circuit 30
1 selects the output image (4 field delay image) of the 4th stage field memory 21, and the 2nd selection circuit 302 selects the output image (4 field delay image) of the 4th stage field memory 21.

The signal representing the fractional part of the delay amount is used as a signal representing the coefficient K, and the first multiplier 303 and the second multiplier 30 are used.
4 is supplied. Therefore, the adder 305 outputs a video delayed from the through video by a delay amount of a unit smaller than the field unit represented by the integer part and the decimal part (hereinafter referred to as a delayed video).

The signal indicating the sign of the delay amount obtained by the delay amount calculation circuit 6 is directly used for the right-eye image selection circuit 33.
Sent as a control signal. On the other hand, a signal indicating the sign of the delay amount obtained by the delay amount calculation circuit 6 is sent to the left-eye video selection circuit 32 as a control signal via the inverter 7. That is, the signal for which the sign of the delay amount obtained by the delay amount calculation circuit 6 is inverted is supplied to the left-eye video selection circuit 32.

When the sign of the delay amount obtained by the delay amount calculation circuit 6 is positive, the right-eye image selection circuit 33 selects the delayed image (output of the interpolation circuit 31) and the left-eye image selection circuit 32 selects the through image. Select. When the sign of the delay amount obtained by the delay amount calculation circuit 6 is positive, the right-eye image selection circuit 33 selects the through image, and the left-eye image selection circuit 32 selects the delayed image (output of the interpolation circuit 31). .

FIG. 2 shows, from the motion vector in the horizontal direction for each divided area detected by the motion vector detection circuit 1,
The flow until the delay amount for each pixel is calculated is shown.

The input image is, for example, as shown in FIG.
An image in which there are stationary trees, mountains, and houses in the upper part, a subject of a car moving to the right on the lower left side, and a subject of a person moving to the left on the lower right side. Suppose The moving speed of a car is assumed to be faster than the moving speed of a person.

As shown in FIG. 2A, the motion vector detection circuit 1 has, for example, four image fields in one field.
A horizontal motion vector is detected for each of the divided areas E11, E12, E21, E22. As shown in FIG. 2A, the horizontal motion vector detected in the upper left area E11 and the upper right area E12 of the divided areas is zero. Of the divided areas, the lower left area E21
The direction of the horizontal motion vector detected at is rightward. The direction of the horizontal motion vector detected in the lower right area E22 is leftward, and the size thereof is smaller than the size of the horizontal motion vector detected in the lower left area.

As shown in FIG. 2B, the CPU 5 calculates the delay amount in units of fields for each area based on the motion vector of each area. In this example, the delay amount D11 in the upper left region and the delay amount D12 in the upper right region are both 0. The delay amount D21 in the lower left area is +1.
The delay amount D22 in the lower right area is -3.

The delay amount calculation circuit 6 has delay amounts D11, D12, D21, D for each area obtained by the CPU 5.
Based on No. 22, the delay amount for each pixel is calculated as follows.

First, the entire image area of one field is shown in FIG.
As shown in (c), 9 blocks B11, B12, B13,
It is divided into B21, B22, B23, B31, B32, and B33. The delay amount of each pixel in the upper left block B11 is
The delay amount D11 for the upper left area E11 calculated by the CPU 5 is determined. Also, block B13 on the upper right
The delay amount of each pixel inside is determined as the delay amount D12 for the upper right area E12 calculated by the CPU 5.

Further, the delay amount of each pixel in the lower left block B31 is calculated by the CPU 5 and the lower left area E2 is calculated.
A delay amount D21 for 1 is determined. Also, the delay amount of each pixel in the lower right block B33 is determined as the delay amount D22 for the lower right area E22 calculated by the CPU 5.

Upper left block B11 and upper right block B
The delay amount of each pixel in the block B12 between 13 and 13 is obtained by weighted averaging the delay amounts D11 and D12 according to the position of the pixel.

Upper left block B11 and lower left block B
The delay amount of each pixel in the block B21 between the block 31 and 31 is calculated by weighted averaging the delay amounts D11 and D21 according to the position of the pixel.

Upper right block B13 and lower right block B
The delay amount of each pixel in the block B23 between 33 and 33 is obtained by weighted averaging the delay amounts D12 and D22 according to the position of the pixel.

The lower left block B31 and the lower right block B
The delay amount of each pixel in the block B32 between 33 and 33 is obtained by weighted averaging the delay amounts D21 and D22 according to the position of the pixel.

The delay amount of each pixel of the central block B22 is determined by the delay amount (weighted average of D11 and D12) of the block B12 according to the position of the pixel,
Delay determined for block B32 (D21 and D
22 and the weighted average). As a result, the delay amount for each pixel is obtained as shown in FIG.

In FIG. 2D, a through image is output to the left-eye image output terminal OL and the right-eye image output terminal OR for the pixel having the delay amount of 0. In FIG. 2D, for a pixel having a positive delay amount, a through image is sent to the left-eye image output terminal OL, and a delayed image corresponding to the delay amount is sent to the right-eye image output terminal OR. In FIG. 2D, for pixels with a negative delay amount, a through image is sent to the right-eye image output terminal OR, and a delayed image corresponding to the delay amount is sent to the left-eye image output terminal OL.

As a result, the input image shown in FIG.
An image for the left eye and an image for the right eye are obtained as shown in FIG.
In other words, the display position of the subject of the car is that the right-eye image is on the left side with respect to the left-eye image, so when observing the left-eye image only with the left eye and the right-eye image only with the right eye,
It seems to pop out forward. Further, the display position of the subject of a person moving in the opposite direction of the automobile also appears to project forward because the right-eye image is on the left side with respect to the left-eye image.

[0040]

According to the present invention, it is possible to stereoscopically view both a subject moving leftward and a subject moving rightward.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a 2D / 3D video conversion device.

FIG. 2 is an explanatory diagram showing a flow from a horizontal motion vector for each divided area detected by a motion vector detection circuit to calculation of a delay amount for each pixel.

FIG. 3 is a schematic diagram showing an example of an input image and a left-eye image and a right-eye image obtained therefrom.

[Explanation of symbols]

 1 Motion Vector Detection Circuit 2 Delayed Video Generation Circuit 3 Left and Right Video Selection Circuit 4 Field Memory Control Circuit 5 CPU 6 Delay Amount Calculation Circuit 7 Inverter 21 Field Memory 31 Interpolation Circuit 32 Left Eye Video Selection Circuit 33 Right Eye Video Selection Circuit 301, 302 Selection circuit 303, 304 Multiplier 305 Adder

Claims (3)

[Claims] 1. A first step of detecting, based on a two-dimensional input image, information about the magnitude and direction of movement of a subject for each area larger than a pixel unit, and the magnitude of movement of the subject detected for each area. Second step of generating identification information indicating whether the delayed image should be the left-eye image or the right-eye image for each pixel based on the information on the height and direction and calculating the delay amount for each pixel , A third step of generating a delayed image delayed by a delay amount corresponding to each pixel based on the delay amount calculated for each pixel, and a third step of generating a delayed image for each pixel. Further, based on the identification information, one of the input image and the delayed image is selected as the pixel for the left eye and the other is selected as the pixel for the right eye for each pixel, whereby the image for the left eye and the image for the right eye are selected. The fourth method of converting steps, the 3-dimensional image a two-dimensional image and a to generate and. 2. The second step is to set a delayed image for each region into a left-eye image and a right-eye image based on the information about the magnitude and direction of the motion of the subject detected for each region. Generating the first identification information indicating the frequency and calculating the delay amount for each field for each area, and the first identification information generated for each area and the delay for each field calculated for each area Based on quantity
Generating second identification information indicating whether the delayed image should be the left-eye image or the right-eye image for each pixel, and calculating a delay amount of a unit smaller than a field unit for each pixel. The method of converting a 2D image into a 3D image according to claim 1. 3. In the third step, the delayed image delayed by the delay amount corresponding to each pixel with respect to the input image by interpolating the image delayed for each field with respect to the input image. The method for converting a 2D image into a 3D image according to claim 2, wherein