JPH10108220A - Device for converting two-dimensional image into three-dimensional image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a stereoscopic effect fixed regardless of the kind and the size of a three-dimensional monitor by deciding the delay amount of a second image as against a first image based on the pixel number converting value of a parallactic amount selected by means of an input means and a movement vector in the horizontal direction, which is obtained from the first image. SOLUTION: Phase control circuits 14 and 15 move the display positions of the inputted images in the horizontal direction by deviating the phase of an inputted image signal. The phase deviating amount and direction are controlled by a memory control circuit 24. Data required for detecting the movement vector is transmitted from a movement vector detecting circuit 16 to CPU 20 and a field memory 11 calculates a delay amount based on the movement vector. First and second dip switches 41 and 42 provided in an operation and display part 23 connected to CPU 20 set the suitable image element number converting value corresponding to a monitor condition regardless of the monitor condition such as the kind and the size, etc., of the three-dimensional monitor so as to make the stereoscopic effect fixed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for converting a two-dimensional video output from a VTR, a video camera or the like, or transmitted by a CATV broadcast, a TV broadcast, etc., into a three-dimensional video.

[0002]

2. Description of the Related Art As a conventional method for converting a two-dimensional image into a three-dimensional image, there is the following method. That is, in the case of a two-dimensional image in which an object moving from left to right is reflected,
In this method, the original two-dimensional image is used as a left-eye image, and an image several frames before the left-eye image is used as a right-eye image. In this case, a parallax is generated between the left-eye image and the right-eye image. By displaying these images almost simultaneously on the screen, the moving object is raised forward with respect to the background.

A video several frames before the left-eye video is obtained by storing an original two-dimensional video in a field memory and reading it out with a delay of a predetermined number of fields.

In such a method, when the delay amount of one of the left-eye image and the right-eye image is constant, the parallax increases as the moving object moves faster, and the stereoscopic effect changes. It becomes difficult to see three-dimensional images.

[0005] In order to keep the parallax constant, the present applicant has devised that the faster the moving object moves, the smaller the amount of delay of one of the left-eye image and the right-eye image is. In this way, a relatively new field is presented as a delayed image for a fast moving image, and a relatively old field is presented as a delayed image for a slow moving image.

[0006]

The method of changing the delay amount according to the movement of a moving object has the following problems. In other words, even with the same three-dimensional video signal obtained by the above conversion method, if the type, size, and the like of the three-dimensional monitor are different, the parallax changes, and the stereoscopic effect changes.

According to the present invention, the stereoscopic effect can be kept constant regardless of the type and size of the three-dimensional monitor.
It is an object of the present invention to provide a device for converting a three-dimensional image into a three-dimensional image.

[0008]

An apparatus for converting a two-dimensional image into a three-dimensional image according to the present invention includes a first image and a first image which are delayed from the two-dimensional image using a field memory. By generating the second image, a predetermined parallax angle is obtained for each condition of various three-dimensional monitors that can be used to display the three-dimensional image in an apparatus that converts a two-dimensional image into a three-dimensional image. The pixel number conversion value of the amount of parallax is determined in advance, and the input means for allowing the operator to select the pixel number conversion value of the parallax amount according to the conditions of the three-dimensional monitor used, and the input means Means for determining a delay amount of the second video with respect to the first video based on the selected pixel number conversion value of the parallax amount and a horizontal motion vector obtained from the first video. Features.

[0009] As an input means, a pixel number conversion value of the amount of parallax corresponding to the condition of the three-dimensional monitor used for displaying a three-dimensional image is provided to the operator in accordance with the size of the three-dimensional monitor. What is selected is used.

[0010] As input means, a pixel number conversion value of the amount of parallax according to the condition of the three-dimensional monitor used for displaying a three-dimensional image is operated in accordance with the type and size of the three-dimensional monitor. Is selected by the user.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

[1] Description of the configuration of the 2D / 3D conversion device

FIG. 1 shows the configuration of a 2D / 3D conversion device for converting a two-dimensional image into a three-dimensional image.

The 2D / 3D converter generates a parallax by generating a left-eye image and a right-eye image by a field delay method, and shifts the phase to both or one of the generated left-eye image and right-eye image. To change the positional relationship between the subject and the reference screen surface.

The input terminal 1 receives a two-dimensional video signal a. The two-dimensional video signal a is sent to the motion vector detection circuit 16, the plurality of field memories 11, and the video switching circuit 13, respectively.

As is well known, the motion vector detecting circuit 16 generates data for detecting a motion vector based on a representative point matching method.
The data generated by the motion vector detection circuit 16 is sent to the CPU 20.

The representative point matching method will be briefly described. A plurality of motion vector detection areas are set in the video area of each field, and each motion vector detection area is divided into a plurality of small areas. A plurality of sampling points and one representative point are set in each of the small areas.

The difference (correlation value at each sampling point) between the video signal level at the sampling point in each small area in the current field and the video signal level at the representative point in the corresponding small area in the previous field is determined by each motion vector detection. Required for each area. Then, correlation values between sampling points having the same deviation with respect to the representative point are cumulatively added between small areas in each motion vector detection area.

In each motion vector detection area, the deviation of the point having the smallest correlation accumulated value, that is, the deviation of the point having the highest correlation is determined by the motion vector (movement of the subject) in the motion vector detection area. Is extracted as

The field memory 11 is provided for delaying and outputting the two-dimensional video signal a in field units. The delay amount is set to 0 by the memory control circuit 24.
To a maximum of 60 fields (1 second in the NTSC system).

An output b (a delayed two-dimensional video signal) of the field memory 11 is sent to a video switching circuit 13 and an interpolation circuit 12, respectively. The interpolation circuit 12 generates a vertical interpolation signal for the input signal b. The output c of the interpolation circuit 12 (the vertical interpolation signal of the delayed two-dimensional video signal) is sent to the video switching circuit 13.

Therefore, the input two-dimensional video signal a and the delayed two-dimensional video signal b
And a vertical interpolation signal c of the delayed two-dimensional video signal b is input. The video switching circuit 13 sends a signal b to the left image phase control circuit 14 and the right image phase control circuit 15.
And one of the signals c (sub-video signal) and the signal a (main video signal) are switched and output according to the moving direction of the subject. However, when the delay amount is 0, the signal a is sent to both the left image phase control circuit 14 and the right image phase control circuit 15.

One of the signals b and c is selected based on whether the two-dimensional video signal a is an odd field or an even field. That is, a signal corresponding to the field type (odd field or even field) of the two-dimensional video signal a is selected from the signals b and c. The image switching by the image switching circuit 13 is performed by the CP
Controlled by U20.

Each of the phase control circuits 14 and 15 is provided to shift the display position of the input image in the horizontal direction by shifting the phase of the input image signal. The phase shift amount and the shift direction are controlled by the memory control circuit 24. Left image phase control circuit 14
Is sent to the left image output terminal 2. The output of the right image phase control circuit 15 is sent to the right image output terminal 3.

The CPU 20 controls the memory control circuit 24 and the video switching circuit 13. The CPU 20 includes a ROM 21 for storing the program and the like and a RAM 22 for storing necessary data. Data necessary for detecting a motion vector is sent from the motion vector detection circuit 16 to the CPU 20.

The CPU 20 is connected to an operation / display section 23 having various input means and a display.
The operation / display unit 23 is provided with a three-dimensional monitor so that the stereoscopic effect is constant regardless of the monitor conditions such as the type and size of the three-dimensional monitor.
A first and a second DIP switch 41 for setting a suitable amount of parallax (pixel number conversion value) according to the monitor condition;
42 are provided.

The CPU 20 calculates the motion vector based on
The number of delay fields (delay amount) by the field memory 11 is calculated. That is, in principle, the delay amount is determined so that the delay amount is reduced when the motion vector is large, and is increased when the motion vector is small.

The CPU 20 controls the video switching circuit 13 based on the direction of the motion vector. That is, when the direction of the motion vector is from left to right, the input 2
The two-dimensional video signal a or the delayed two-dimensional video signal b or c is supplied to the phase control circuit 15 for the right eye.
Send to When the direction of the motion vector is from right to left, the input two-dimensional video signal a is converted to the right-eye phase control circuit 15.
Then, the delayed two-dimensional video signal b or c is sent to the left-eye phase control circuit 14.

In this 2D / 3D conversion apparatus, parallax is generated by generating a left-eye image and a right-eye image by a field delay method, and the phase is shifted to both or one of the generated left-eye image and right-eye image. Is performed to change the positional relationship between the subject and the reference screen surface.

[2] Description of a suitable amount of parallax (pixel number conversion value) set by the first and second dip switches 41 and 42

Incidentally, even with the same three-dimensional video signal, the stereoscopic effect differs depending on the monitor conditions, that is, the type of monitor, the size of the monitor, the distance of the observer from the monitor, and the like. Therefore, it is necessary to prevent the stereoscopic effect from changing according to the monitor condition.

Hereinafter, a method for preventing the stereoscopic effect from changing according to monitor conditions will be described with reference to FIG.

A suitable distance between the monitor surface S and the eyes 31 and 32 of the observer is defined as an appropriate viewing distance A [mm]. The distance between the right image R and the left image L of the gazing object on the monitor surface S is defined as a parallax amount B [mm]. The distance between eyes is C [mm]. The suitable viewing distance A is determined by the size of the monitor. In addition, an intersection point P between a line connecting the left eye 31 and the left image L and a line connecting the right eye 32 and the right image R is a stereoscopic position.

The angle formed by the line of sight of the eyes 31L and 31R with respect to the midpoint between the left image L and the right image R on the monitor surface S is θ [r
ad], and the eyes 31L, 31R with respect to the stereoscopic position P
If the angle of the line of sight is α [rad], the parallax angle Δθ
[Rad] is represented by the following equation 1.

[0035]

(Equation 1)

Θ and α are represented by Equations 2 and 3, respectively.

[0037]

(Equation 2)

[0038]

(Equation 3)

Since C: B = (A + D): D, D = A ＝ B / (CB). Therefore, α
Is represented by Equation 4.

[0040]

(Equation 4)

Therefore, the parallax angle Δθ is expressed by the following equation (5).

[0042]

(Equation 5)

Here, assuming that θ and α are sufficiently small, the parallax angle Δθ is expressed by the following equation (6).

[0044]

(Equation 6)

If the horizontal length of the monitor is W [mm] and the number of pixels in the horizontal direction of the monitor is G [pixels], the amount of parallax B
The pixel number conversion value X of [mm] is represented by the following Expression 7.

[0046]

(Equation 7)

By substituting Equation 6 into Equation 7, Equation 8 is obtained.

[0048]

(Equation 8)

.DELTA..theta. Is in radians.
Equation 8 becomes Equation 9.

[0050]

(Equation 9)

In order to make the stereoscopic effect constant regardless of the monitor conditions, it is necessary to keep the parallax angle Δθ (= θ−α) constant at all times.

Assuming that the number of pixels G in the horizontal direction of the monitor is constant, in order to keep the parallax angle Δθ constant, the pixel number conversion value X of the amount of parallax is inversely proportional to the horizontal length W of the monitor, and A must be changed in direct proportion to A. That is, it is necessary to change the pixel conversion value X of the suitable amount of parallax according to the horizontal length W of the monitor and the suitable viewing distance A. When the suitable viewing distance A is determined in advance by the horizontal length W of the monitor, in order to keep the parallax angle Δθ constant, a pixel conversion value X of a suitable parallax amount is obtained for each horizontal length W of the monitor. , A pixel conversion value X corresponding to the horizontal length W of the monitor used
Should be set.

Incidentally, the size of the monitor is generally represented by the length [inch] of the diagonal line of the monitor. However,
Some monitors have different horizontal to vertical ratios. For example, there are monitors having a horizontal to vertical ratio of 4: 3 and monitors having a 16: 9 ratio. When the width-to-length ratio is different, the horizontal length W is different even if the diagonal length of the monitor is the same.

Therefore, in this embodiment, the pixel number conversion value X of the parallax amount for each size of the 4: 3 monitor and 1
The pixel number conversion value X of the parallax amount for each size of the 6: 9 monitor is determined in advance.

A method of obtaining the pixel number conversion value X of the parallax amount for each size for the 4: 3 monitor will be described. First, the horizontal length W of the monitor is obtained for each size. Further, the suitable viewing distance A is determined for each size. Further, a suitable parallax angle Δθ is determined. Then, for each size, the pixel number conversion value X of the parallax amount is calculated based on Expression 9 above.
To determine. Similarly, for the 16: 9 monitor, the pixel number conversion value X of the parallax amount is determined for each size.

The operation / display unit 23 includes a first dip switch 41 for selecting the pixel number conversion value X of the parallax amount for each 4: 3 monitor size determined in this way.
And a second dip switch 42 for selecting a pixel number conversion value X of the parallax amount for each monitor size of 16: 9.

The operator determines the type of monitor to be used (4: 3
The first DIP switch 41 or the second DIP switch 42 is set in accordance with the current monitor or the monitor of 16: 9) and the size (diagonal length [inch] of the monitor). When only one type of monitor is used among the 4: 3 monitor and the 16: 9 monitor, the pixel number conversion value X of the parallax amount for each size for one type of monitor is used. Need only be provided with a dip switch for selecting.

[3] Description of 2D / 3D conversion processing procedure

FIG. 3 shows a 2D / 3D conversion processing procedure by the CPU.

The 2D / 3D conversion processing by the CPU is performed every time the field switching timing of the input video signal a comes.

(1) In step 1, of the plurality of field memories 11, a memory (write memory) for writing the two-dimensional video signal for the current field and a memory for reading the already stored two-dimensional video signal (read memory) ) Is output to the memory control circuit 24. Further, each phase control circuit 14, 1
The data indicating the phase shift amount and the direction due to 5 is output to the memory control circuit 24. Further, a video switching control signal is output to the video switching circuit 13.

The read memory is determined based on the delay amount determined in the previous 2D / 3D conversion processing.
Further, the phase shift amount and the direction by each of the phase control circuits 14 and 15 are determined based on the data that has already been captured and stored in step 2 of the 2D / 3D conversion processing.

The selection of one of the delayed two-dimensional video signals b and c depends on the field type of the two-dimensional video signal b to be read from the field memory 11 and
It is determined based on the field type of the two-dimensional video signal a. Further, the switching between the selected signal b or c and the signal a is determined based on the direction of the horizontal motion vector obtained in the previous 2D / 3D conversion processing.
The switching direction between the selected signal b or c and the signal a is represented by the polarity of the delay amount.

(2) In step 2, the operation / display unit 23
Are input and stored. The various input signals include a signal for setting a phase shift amount and a direction, an automatic / manual mode setting signal indicating whether a delay amount is automatically calculated (automatic mode) or manually set (manual mode), and a first or a second signal. There are a pixel number conversion value X of a suitable parallax amount set by the two dip switches 41 and 42, a delay amount setting signal performed when the manual mode is set, and the like.

(3) In step 3, the previous 2D / 3D
Only reliable motion vectors are extracted based on the reliability results for the motion vectors for each motion vector detection area obtained in step 10 of the conversion process.

(4) In step 4, only the reliable motion vectors extracted in step 3 whose vertical component is smaller than a predetermined value are extracted.

(5) In step 5, the average value of the horizontal component (effective horizontal motion vector) of the reliable motion vector extracted in step 4 is calculated.

(6) In step 6, a delay amount calculation process is performed based on the average value of the effective horizontal motion vectors calculated in step 5 and the suitable pixel number conversion value X of the parallax amount fetched in step 2. Done. The details of the delay amount calculation processing will be described later.

(7) In step 7, it is determined whether the mode is the automatic mode or the manual mode based on the data fetched and stored in step 2.

(8) If it is determined in step 7 that the manual mode has been set, the delay amount is fixed to the set value fetched in step 2 (step 8).

(9) If it is determined in step 7 that the automatic mode is set, the history data used in the delay amount calculation processing in step 6 is updated (step 9).

(10) In step 10, data necessary for motion vector detection is fetched from the motion vector detection circuit 16, and a motion vector for each motion vector detection area is calculated. Further, the reliability of the motion vector is determined for each motion vector detection area based on the average value and the minimum value of the correlation accumulated value for each motion vector detection area. Then, the calculated motion vector and the reliability determination result are stored in the RAM 22.

[4] Description of Delay Calculation Processing

FIG. 4 shows a detailed procedure of the delay amount calculation processing in step 6 of FIG.

First, based on the pixel number conversion value X of the parallax amount taken in step 2 and the average value v (hereinafter referred to as a motion vector average value) of the effective horizontal motion vectors obtained in step 5 above, The first delay amount d1 is obtained (step 21).

The first delay amount d1 is obtained by the following equation (10).

[0077]

(Equation 10)

When the first delay amount d1 is obtained in step 21, the average value of the delay amounts for the ten fields from the current time to the past nine times, and the average value of the delay amounts from the previous time to the past nine times, based on the delay amount history data. Average value of the delay amount for the field,
The average value of the delay amounts for the ten fields from the previous last time to the past nine times is calculated (step 22).

The delay amount history data used in step 22 is the first delay amount d1 obtained in step 21 in the past.

Next, if two or more of the three average values are the same value, the value (multiple values) is selected as the second delay amount d2. Is selected as the second delay amount d2 (step 23).

Next, the second delay amount d2 selected in step 23 and 1 (for example, 1
One of the second delay amounts d2 (five fields before) and 30
The second delay amount d2 before the field is compared (step 24). The delay amount history data used in step 24 is the second delay amount d2 obtained in step 23 in the past.

If all the second delay amounts d2 match (YES in step 25), the target delay amount Pd is changed to the second delay amount selected in step 23 (Pd = d
2) (Step 26), and proceed to Step 30. Therefore, as shown in FIG. 5, when the three second delay amounts d2 (represented by d2-1, d2-2, and d2-3 in the order from the past) change and all the second delay amounts d2 match. , The target delay amount Pd is changed to the second delay amount (d2-3).

When all the second delay amounts d2 do not match (NO in step 25), all the second delay amounts d2 are larger than the current target delay amount Pd or all the second delay amounts d2.
It is determined whether 2 is smaller than the current target delay amount Pd or does not satisfy those conditions (step 27).

If all the second delay amounts d2 are larger than the current target delay amount Pd, the target delay amount Pd is incremented by 1 (Pd = Pd + 1) (step 28), and then step 30
Proceed to. For example, as shown in FIG. 6, three second delay amounts d2 (d2-1, d2-2, d2
When the second delay amount d2 is greater than the current target delay amount Pd, the target delay amount Pd is incremented by one.

When all the second delay amounts d2 are smaller than the current target delay amount Pd, the target delay amount Pd is decremented by one (Pd = Pd-1) (step 29), and then step 30
Proceed to. All the second delay amounts d2 are equal to the current target delay amount Pd.
If it is not larger and all the second delay amounts d2 are not smaller than the current target delay amount d, the process proceeds to step 30.

In step 30, it is determined whether or not the target delay amount Pd and the currently set delay amount (set delay amount d3) match. If the target delay amount Pd does not match the set delay amount d3, it is determined whether the current set delay amount d3 has already continued for four fields (step 31). If the current set delay amount d3 has already continued for 4 fields, the set delay amount d3 is changed by 1 in the direction approaching the target delay amount Pd (d3 = d
3 ± 1) (step 32). Then, step 7 in FIG.
Move to

If the target delay amount matches the current set delay amount in step 30 or if the current set delay amount has not already continued for four fields in step 31, the delay amount The process proceeds to step 7 in FIG. 3 without any change.

That is, in this example, the set delay amount d3 is 4
The control is performed so as to approach the target delay amount Pd in units of a field cycle and one field at a time.

After the power is turned on, step 21 is executed.
In the above, when the first delay amount d1 is calculated for the first time, the second delay amount d2, the target delay amount Pd, and the set delay amount d3 become equal to d1.

In the process of FIG. 4, in step 22, only the average value of the delay amounts for 10 fields from this time to the past 9 times is calculated, and this is set as the target delay amount.
The processing of 3, 24, 25, 26, 27, 28, 29 may be omitted.

In step 22, it is also possible to calculate only the average value of the delay amounts for the past nine fields from the present time for the ten fields, use this as the second delay amount, and omit the processing in step.

Further, the second delay amount obtained in step 23 is set as a target delay, and steps 24, 25, 26, 27,
Steps 28 and 29 may be omitted.

Further, the processing of steps 22 and 23 may be omitted. In this case, the first delay amount d1 obtained in step 21 is used as the second delay amount used in step 24.

[0094]

According to the present invention, the stereoscopic effect can be kept constant regardless of the type and size of the three-dimensional monitor.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a 2D / 3D conversion device.

FIG. 2 is a schematic diagram for explaining a method for preventing a stereoscopic effect from changing according to monitor conditions.

FIG. 3 is a flowchart showing an overall procedure of 2D / 3D conversion processing by a CPU.

FIG. 4 is a flowchart illustrating a detailed procedure of a delay amount calculation process in step 6 of FIG. 3;

FIG. 5 is a time chart showing how the target delay amount is changed when all three second delay amounts match.

FIG. 6 is a time chart showing how the target delay amount is changed when all three second delay amounts become larger than the current target delay amount.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Field memory 13 Video switching circuit 20 CPU 21 ROM 22 RAM 23 Operation / display part 24 Memory control circuit 41, 42 DIP switch

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Seiji Okada 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Sanyo Electric Co., Ltd. (72) Inventor Yukio Mori 2-5-2 Keihanhondori, Moriguchi-shi, Osaka No. 5 Sanyo Electric Co., Ltd.

Claims (3)

[Claims] 1. A two-dimensional image is converted into a three-dimensional image by generating a first image and a second image delayed from the first image using a field memory from the two-dimensional image. In the apparatus, a pixel number conversion value of a parallax amount for obtaining a predetermined parallax angle is determined in advance for each of various types of three-dimensional monitor conditions that can be used for displaying a three-dimensional image. Input means for allowing the operator to select a pixel number conversion value of the parallax amount according to the condition of the dimensional monitor, and a pixel number conversion value of the parallax amount selected by the input means and the first image. Means for determining a delay amount of the second image with respect to the first image based on the horizontal motion vector. An apparatus for converting a two-dimensional image into a three-dimensional image. 2. An image processing apparatus according to claim 1, wherein the input means converts a pixel number conversion value of the parallax amount according to a condition of the three-dimensional monitor used for displaying the three-dimensional image into an operator according to the size of the three-dimensional monitor. The apparatus for converting a two-dimensional image into a three-dimensional image according to claim 1, wherein the two-dimensional image is selected. 3. The three-dimensional monitor according to claim 3, wherein said input means converts a pixel number conversion value of a parallax amount according to a condition of the three-dimensional monitor used for displaying a three-dimensional image according to a type and a size of said three-dimensional monitor. The apparatus for converting a two-dimensional image into a three-dimensional image according to claim 1, which is to be selected by an operator.