JP3392627B2 - How to convert 2D video to 3D video - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for converting a two-dimensional image output from a VTR, a video camera or the like, or transmitted by CATV broadcasting, TV broadcasting or the like into a three-dimensional image.

[0002]

2. Description of the Related Art Most of the 3D image software used in the 3D image display system, which has been talked about recently, is 3D.
It was created specifically for the 3D video display system. Such three-dimensional video software is generally recorded by picking up a left-eye video and a right-eye video using two cameras. The left and right images recorded in the three-dimensional image software are superimposed and displayed on the display device almost at the same time. Then, the left-eye image and the right-eye image that are displayed in an overlapping manner are separately incident on the left and right eyes of the observer, so that the observer recognizes the three-dimensional image.

By the way, a lot of two-dimensional video software currently exists. Therefore, if a 3D image can be generated from these 2D image software, it is possible to save the trouble of recreating the 3D image software having the same contents as the existing 2D image software from the beginning.

Under these circumstances, a method for converting a two-dimensional image into a three-dimensional image has already been proposed. The following is a conventional method for converting a 2D image into a 3D image. That is, in the case of a two-dimensional image showing an object moving from left to right, the original two-dimensional image is the left-eye image, and the image several frames before the left-eye image is the right-eye image. Is the way. In this case, a parallax occurs between the left-eye image and the right-eye image, so that by displaying both images on the screen almost at the same time, the moving object is projected forward with respect to the background.

An image several frames before the image for the left eye can be obtained by storing the original two-dimensional image in the field memory and reading the image delayed by a predetermined number of fields. The conventional method as described above will be referred to as a field delay method.

[0006]

In the above conventional method, when the delay amount of one of the left-eye image and the right-eye image is constant, the parallax increases as the horizontal movement of the moving object increases. As a result, the stereoscopic effect changes and it becomes difficult to see a three-dimensional image.

Therefore, in order to obtain a stable three-dimensional effect, the applicant of the present invention, the faster the horizontal movement of the moving object,
We devised to reduce the delay amount of one of the left-eye image and the right-eye image to the other. In this way, a relatively new field is presented as a delayed image for a video with fast horizontal movement, and a relatively old field is presented as a delayed image for a video with slow horizontal movement.

In the method of determining the delay amount as described above, the delay amount is determined based only on the speed of the horizontal movement of the image. Therefore, even if there is an object having a large movement in the vertical direction, the delay amount is large if the movement of the video in the horizontal direction is small. In such a case, between the left-eye image and the right-eye image, the vertical distance of the object having large vertical movement increases, and the left-eye image and right-eye image of the object having large vertical movement. And will not merge.

It is an object of the present invention to provide a method for converting a two-dimensional image into a three-dimensional image, which can prevent the left-eye image and the right-eye image of an object moving in the vertical direction from not being fused. .

[0010]

In the method of converting a 2D image into a 3D image according to the present invention, a 2D image signal is converted into a main image signal and a sub image signal delayed from the main image signal. By generating, the two-dimensional image is converted into a three-dimensional image. The delay amount of the sub-picture signal with respect to the main picture signal is changed according to the speed of the horizontal movement of the main picture. The upper limit of the delay amount is determined based on the vertical component of the motion vector detected from the main video. And
The delay amount is determined so as to be equal to or less than the determined upper limit value.

The upper limit of the delay amount is determined as follows, for example. That is, the average absolute value of the vertical components of all or some of the motion vectors detected from each of the plurality of motion vector detection areas set in the video area of the main video is calculated and stored for each field. Then, in each field, the upper limit value of the delay amount is determined within a range in which the cumulative value of the average values from the present field to the past is less than the predetermined value.

A more specific method for calculating the upper limit value of the delay amount is, for example, the following method.

(1) First Method The average value of the absolute values of the vertical components of all or some of the motion vectors detected from each of the plurality of motion vector detection areas set in the video area of the main video is calculated for each field. Calculated and stored in the storage means (first step).

Of the average values for the predetermined number of past fields stored in the storage means, the cumulative number of fields indicating how many fields from the latest one are accumulated,
It is set to a predetermined maximum delay amount (second step).

Among the average values for the predetermined number of past fields stored in the storage means, the average values for the set cumulative fields are read out in order from the newest one, and the cumulative values are calculated (the first). 3 steps).

If the calculated cumulative value is less than the predetermined reference value, the currently set cumulative number of fields is determined as the delay amount upper limit value, and the calculated cumulative value is set to the predetermined reference value. If it is equal to or larger than the value, the currently set cumulative field number is decremented by 1 (4 steps).

When the cumulative field number is updated in the fourth step, the processes of steps 3 and 4 are repeatedly executed using the updated cumulative field number (fifth step).

(2) Second Method Of a plurality of motion vector detection areas set in the video area of the main video, all or some of the motion vectors detected respectively from the areas where it is determined that the subject is present are vertical. The average value of the absolute values of the components is calculated for each field, and the calculated average value is stored in the storage means as the first average value (first step).

The average value of the absolute values of the vertical components of all or some of the motion vectors respectively detected from the regions determined to have the background among the plurality of motion vector detection regions is calculated for each field and calculated. The calculated average value is stored in the storage means as the second average value (second step).

Of the first average value and the second average value for the predetermined number of past fields stored in the storage means, the cumulative field number indicating how many fields from the latest one to be accumulated is previously set. The maximum delay amount is set (third step).

From the first average values for the predetermined number of past fields stored in the storage means, the first average values for the set cumulative field numbers are read out in order from the newest one, and the cumulative values are calculated. Calculate a certain first cumulative value (fourth
Step).

From the second average value for the predetermined number of past fields stored in the storage means, the second average value for the set cumulative field number is read out in order from the newest one, and the cumulative value is calculated. Calculate a certain second cumulative value (fifth
Step).

When both the calculated first cumulative value and the second cumulative value are less than the predetermined reference value, the currently set cumulative field number is determined as the delay amount upper limit value and calculated. If at least one of the first cumulative value and the second cumulative value is greater than or equal to a predetermined reference value, the currently set cumulative field number is decremented by 1 (sixth step).

When the cumulative field number is updated in the sixth step, the processes of steps 4, 5 and 6 are repeated using the updated cumulative field number (seventh step).

(3) Third Method The absolute value of the vertical component of the motion vector detected from all or a part of the plurality of motion vector detection areas set in the video area of the main video is calculated for each field. It is determined whether the calculated absolute value is larger than a predetermined value (first step).

When the calculated absolute value is larger than the predetermined value, it is judged that the condition for judging whether or not an object having a large vertical motion exists in the main image is satisfied. The determination result is stored in the storage means in association with the motion vector detection area in which the motion vector, which is the absolute value calculation source, has been detected (second step).

Based on the determination result for the predetermined number of past fields stored in the storage means, the above condition is satisfied for a predetermined number of predetermined fields or more out of a predetermined number of past fields in each motion vector detection area. It is determined whether or not a motion vector detection area exists (third step).

In the motion vector detection areas, if there is a motion vector detection area satisfying the above condition for a predetermined number of fields or more out of a predetermined number of fields in the past, the delay amount upper limit value is determined to be 0. (4th step).

[0029]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the configuration of a 2D / 3D conversion device for converting a 2D image into a 3D image.

This 2D / 3D conversion device generates parallax by generating a left-eye image and a right-eye image by the field delay method, and shifts the phase of the generated left-eye image and right-eye image or both. The positional relationship between the subject and the reference screen surface is changed by applying.

A two-dimensional video signal a is input to the input terminal 1. The two-dimensional video signal a is sent to the motion vector detection circuit 16, the plurality of field memories 11 and the video switching circuit 13, respectively.

As is well known, the motion vector detection circuit 16 is for generating data for detecting a motion vector based on the representative point matching method.
The data generated by the motion vector detection circuit 16 is sent to the CPU 20.

The representative point matching method will be briefly described. As shown in FIG. 8, the video area 1 of each field
00, a plurality of motion vector detection areas E _{1 to} E ₁₂ are set. The motion vector detection areas E _{1 to} E ₁₂ have the same size. In addition, each motion vector detection area E ₁
As shown in FIG. 9, each of E to E ₁₂ has a plurality of small areas e.
It is divided into (e ₀ ~e _m). Then, as shown in FIG. 10, a plurality of sampling points S and one representative point R are set in each small area e.

Each sun in the small area e in the current field
The video signal level at the pulling point S and in the previous field
Difference from the video signal level of the representative point R of the corresponding small area e
(Correlation value at each sampling point) is
Detection area E₁~ E₁₂Required for each. And each movement
Vector detection area E₁~ E₁₂For each, motion vector inspection
With respect to the representative point R between all the small areas in the output area
The correlation values of sampling points with the same deviation are cumulatively added.
Be done. Therefore, each motion vector detection area E ₁~ E₁₂
For each, according to the number of sampling points in one small area e
The cumulative number of correlations is calculated.

In each of the motion vector detection areas E _{1 to} E ₁₂ , the deviation of the point having the smallest correlation cumulative value, that is, the deviation of the point having the highest correlation is the motion vector detection area E.
It is extracted as a motion vector of _{1 to} E ₁₂ (motion of the subject).

The field memory 11 is provided to delay and output the two-dimensional video signal a in units of fields, and a plurality of field memories are provided. Writing and reading of each field memory 11 are controlled by the memory control circuit 24.

The output b (delayed two-dimensional video signal) of the field memory 11 is sent to the video switching circuit 13 and the interpolation circuit 12, respectively. The interpolation circuit 12 generates an interpolation signal in the vertical direction with respect to the input signal b. The output c of the interpolation circuit 12 (vertical interpolation signal of the delayed two-dimensional video signal) is sent to the video switching circuit 13.

Therefore, the video switching circuit 13 receives the input two-dimensional video signal a and delayed two-dimensional video signal b.
And the vertically-interpolated signal c of the delayed two-dimensional video signal b is input. The video switching circuit 13 supplies one of the signals b and c (sub video signal) to the left image phase control circuit 14 and the right image phase control circuit 15.
The signal a (main video signal) is switched and output according to the moving direction of the subject. However, if the delay amount is 0,
Left image phase control circuit 14 and right image phase control circuit 15
The signal a is sent to both the and.

One of the signals b and c is selected based on whether the two-dimensional video signal a is an odd field or an even field. That is, the one corresponding to the field type (odd field or even field) of the two-dimensional video signal a is selected from the signals b and c. Switching of images by the image switching circuit 13 is performed by CP
Controlled by U20.

The phase control circuits 14 and 15 are provided to move the display position of the input image in the horizontal direction by shifting the phase of the input image signal. The phase shift amount and the shift direction are controlled by the memory control circuit 24. Left image phase control circuit 14
Is sent to the left image output terminal 2. The output of the right image phase control circuit 15 is sent to the right image output terminal 3.

The CPU 20 controls the memory control circuit 24 and the video switching circuit 13. The CPU 20 includes a ROM 21 that stores the program and the like and a RAM 22 that stores necessary data. Data necessary for motion vector detection is sent from the motion vector detection circuit 16 to the CPU 20. Further, the CPU 20 is connected with an operation / display unit 23 including various input means and a display.

The CPU 20 determines, based on the motion vector,
The number of delay fields (delay amount) by the field memory 11 is calculated. That is, in principle, the delay amount is determined so that the delay amount is small when the motion vector is large, and the delay amount is large when the motion vector is small.

The CPU 20 also controls the video switching circuit 13 based on the direction of the motion vector. That is, if the direction of the motion vector is from left to right, the input 2
The two-dimensional video signal a is supplied to the left-eye phase control circuit 14, and the delayed two-dimensional video signal b or c is supplied to the right-eye phase control circuit 15.
Send to. When the direction of the motion vector is from right to left, the input two-dimensional video signal a is supplied to the right-eye phase control circuit 14
Then, the delayed two-dimensional video signal b or c is sent to the left-eye phase control circuit 15.

In this 2D / 3D conversion apparatus, parallax is generated by generating the left-eye image and the right-eye image by the field delay method, and the generated left-eye image and right-eye image are phase-shifted to one or both. The positional relationship between the subject and the reference screen surface is changed by applying.

FIG. 2 shows a 2D / 3D conversion processing procedure by the CPU.

The 2D / 3D conversion processing by the CPU is performed every time the field switching timing of the input video signal a comes.

(1) In step 1, among the plurality of field memories 11, a memory (writing memory) in which the 2D video signal for the current field is to be written and a memory (readout in which the 2D video signal already stored is to be read (reading). Data indicating each memory) is output to the memory control circuit 24. In addition, each phase control circuit 14, 1
Data indicating the phase shift amount and the direction due to 5 is output to the memory control circuit 24. Further, a video switching control signal is output to the video switching circuit 13.

The read memory is determined based on the delay amount determined in the previous 2D / 3D conversion process.
Further, the phase shift amount and the direction by each of the phase control circuits 14 and 15 are determined based on the data already captured and stored in step 2 of the 2D / 3D conversion processing.

One of the delayed two-dimensional video signals b and c is selected by selecting the field type of the two-dimensional video signal b to be read from the field memory 11.
It is determined based on the field type of the 2D video signal a. Furthermore, the switching between the selected signal b or c and the signal a is determined based on the direction of the horizontal motion vector obtained in the previous 2D / 3D conversion process.
The switching direction between the selected signal b or c and the signal a is represented by the polarity of the delay amount.

(2) In step 2, the operation / display unit 23
Various input signals from are captured and stored. For various input signals, a signal for setting the phase shift amount and direction, an automatic / manual mode setting signal indicating whether the delay amount is automatically calculated (automatic mode) or manually (manual mode),
There are a delay amount magnification setting signal performed when the automatic mode is set, a delay amount setting signal performed when the manual mode is set, and the like.

(3) In step 3, the previous 2D / 3D
Only the reliable motion vector is extracted based on the reliability result of the motion vector for each motion vector detection area obtained in step 10 of the conversion process.

(4) In step 4, delay amount upper limit value calculation processing is performed based on the vertical component of the reliable motion vector extracted in step 3. The delay amount upper limit calculation processing is performed between the left-eye video and the right-eye video,
This is a process of calculating the upper limit value of the delay amount in order to prevent the vertical distance of an object moving in the vertical direction from becoming too large. For details on the delay amount upper limit calculation processing,
It will be described later.

(5) In step 5, only the reliable motion vector extracted in step 3 whose vertical component is smaller than a predetermined value is extracted.

(6) In step 6, the average value of the horizontal component (effective horizontal direction motion vector) of the reliable motion vector extracted in step 5 is calculated.

(7) In step 7, a delay amount calculation process is performed based on the average value of the effective horizontal direction motion vector calculated in step 6. Details of this delay amount calculation processing will be described later.

(8) In step 8, it is determined whether the mode is the automatic mode or the manual mode based on the data captured and stored in step 2.

(9) When it is determined in step 8 that the manual mode is set, the delay amount is fixed to the set value fetched in step 2 (step 9).

(10) If it is determined in step 8 that the automatic mode is set, the history data used in the delay amount calculation process in step 7 is updated (step 10).

(11) In step 11, the data required for motion vector detection is fetched from the motion vector detection circuit 16 and the motion vector for each motion vector detection area is calculated. Further, the reliability of the motion vector is determined for each motion vector detection area based on the average value, the minimum value, etc. of the correlation cumulative value for each motion vector detection area. Then, the calculated motion vector and the reliability determination result are stored in the RAM 22.

FIG. 11 shows the detailed procedure of the delay amount upper limit value calculation processing in step 4 of FIG. Here, it is assumed that six field memories are provided and the maximum number of delay fields is six.

First, the average absolute value of the vertical component (vertical vector) of the reliable motion vector extracted in step 3 of FIG. 2 is calculated and stored in the RAM 22 (step 41). The average value of the absolute values of the vertical direction vector calculated this time is y _0, and the average values of the absolute values of the vertical direction vector of the past 5 times already stored in the RAM 22 are y ₁ , y ₂ , y ₃ , y _4. , Y ₅ .

Hereinafter, the average value for the past 6 fields including the latest average value y ₀ stored in the RAM 22 is y.
It is represented by _n (n = 0, 1, 2, 3, 4, 5). Also, the average values y _n (n = 0, 1, 2, 3,
Among the 4 and 5), the variable i indicating the cumulative number of fields from the latest to the number of previous fields is referred to as the cumulative field number i.

Next, the cumulative field number i is set to 6 which is the maximum delay field number (delay amount maximum value) (step 42).

Next, the average value y for the past 6 fields
_{Of n} (n = 0, 1, 2, 3, 4, 5), the latest one to the i-th previous one are accumulated (step 43). That is, from the average values y _n of the past 6 fields stored in the RAM 22, the currently set average values of the cumulative number of fields are read in order from the newest, and the cumulative values are calculated. It That is, the cumulative value Z up to the i-field before the average value of the absolute values of the vertical direction vector is obtained based on the following formula 1.

[0066]

[Equation 1]

When the process proceeds from step 42 to step 43, since i = 6, the cumulative value Z is given by the following equation 2.

[0068]

[Equation 2]

Next, the cumulative value Z calculated in step 43
Is determined to be smaller than a predetermined reference value Zo (step 44). For example, 2 is set as the reference value Zo.

When the cumulative value Z calculated in step 43 is smaller than the predetermined reference value Zo (TES in step 44), in the period of the past i fields,
It is determined that the vertical movement of the image is small, and the delay amount upper limit value D _max is set to the currently set cumulative field number i (step 47). For example, if i = 6, the delay amount upper limit value D _max is set to 6.

When the cumulative value Z calculated in step 43 is equal to or larger than the predetermined reference value Zo (step 44)
NO), it is determined that the vertical movement of the image is large in the past i field period. In other words,
When the number of delay fields is i, the vertical distance of the vertically moving object increases between the left-eye image and the right-eye image, and the left-eye image and the right-eye image of the vertically moving object are increased. It is judged that the video for use will not merge.

In this case, i is decremented by 1 (i ← i-1) (step 45). Then, if the updated value of i is not 0 (NO in step 46),
Returning to step 43, the processing after step 43 is executed again.

For example, i before update is 6 and after i is updated
If i is 5, the absolute value of the vertical vector
Cumulative value Z (= y up to 5 fields before the average value)₀+ Y₁
+ Y ₂+ Y₃+ Y_Four) Is calculated (step 43).
When the calculated cumulative value Z is smaller than the reference value Zo,
(YES at step 44), the delay amount upper limit value D_{ma x}
Is set to 5 (step 47). Calculated cumulative value
When Z is equal to or greater than the reference value Zo, i is decremented by 1
After being processed (i ← i-1) (step 45), the step
Returning to step 43, the processing after step 43 is executed again.
It

In this way, the delay amount upper limit value D _max is determined. When i updated in step 45 is 0 (YES in step 46), the delay amount upper limit value D _max is set to 0.

FIG. 12 shows another example of the delay amount upper limit value calculation processing. Here, it is assumed that six field memories are provided and the maximum number of delay fields is six.
Further, in step 11 of FIG. 2, it is assumed that the area of the subject and the area of the background are discriminated in each motion vector detection area.

First, of the reliable motion vectors extracted in step 3 of FIG. 2, an average value of absolute values of vertical components of the motion vector of the object area (hereinafter referred to as a first average value) is calculated. It is stored in the RAM 22 (step 51). Let the first average value calculated this time be ya ₀ ,
The first average values of the past five times already stored in the RAM 22 are set as ya ₁ , ya ₂ , ya ₃ , ya ₄ , and ya ₅ .

Hereinafter, the first average value for the past 6 fields including the latest first average value ya ₀ stored in the RAM 22 is represented by ya _n (n = 0, 1, 2, 3, 4, 5). . Also, the first average value ya _n (n = 0,
Among the 1, 2, 3, 4, 5), the variable i indicating how many fields from the latest one to the previous one are to be accumulated is referred to as the accumulated field number i.

Further, of the reliable motion vectors extracted in step 3 of FIG. 2, the average value of absolute values of the vertical components of the motion vector of the background area (hereinafter referred to as the second average value) is calculated. It is stored in the RAM 22 (step 52). The second average value calculated this time is yb _0, and the second average values of the past five times already stored in the RAM 22 are yb ₁ , yb ₂ , yb ₃ , yb ₄ , and yb ₅ .

Hereinafter, the second average value for the past 6 fields including the latest second average value yb ₀ stored in the RAM 22 is represented by yb _n (n = 0, 1, 2, 3, 4, 5). . Also, the second average value yb _n (n = 0,
Among the 1, 2, 3, 4, 5), the variable i indicating how many fields from the latest one to the previous one are to be accumulated is referred to as the accumulated field number i.

Next, the cumulative field number i is set to 6 which is the maximum delay field number (step 53).

Next, the first average value y for the past 6 fields
Of a _{n (n} = 0,1,2,3,4,5), those before i field from the latest ones are accumulated. (Step 54). That is, the cumulative value Za (i.e., the first average value ya _n) of the first average value ya _{n up} to i fields before is calculated based on the following Equation 3.
The cumulative value Za) is obtained.

[0082]

[Equation 3]

When the process proceeds from step 53 to step 54, since i = 6, the first cumulative value Za is given by the following equation 4.

[0084]

[Equation 4]

The second average value y for the past 6 fields
Among b _n (n = 0, 1, 2, 3, 4, 5), the latest ones to the i-th field before are accumulated. (Step 55). That is, the cumulative value Zb (hereinafter, referred to as the second value) of the second average value yb _{n up} to i fields before is calculated based on the following Expression 5.
The cumulative value Zb) is obtained.

[0086]

[Equation 5]

When the process moves from step 53 to step 54 and step 55, since i = 6, the second accumulated value Z
b is given by the following expression 6.

[0088]

[Equation 6]

Next, it is determined whether the first cumulative value Za calculated in step 54 is smaller than a predetermined reference value Zo (step 56). As the reference value Zo,
For example, 2 is set.

First cumulative value Za calculated in step 54
Is smaller than the predetermined reference value Zo (YES in step 56), it is determined whether the second cumulative value Zb calculated in step 55 is smaller than the predetermined reference value Zo (step 57). ). For example, 2 is set as the reference value Zo.

Second accumulated value Zb calculated in step 55
Is smaller than a predetermined reference value Zo (step
Yes in step 57), in the past i-field period
Small vertical movement of both subject and background
And the delay amount upper limit value D _maxIs set to i (
Step 60). For example, if i = 6, the delay amount upper limit
Value D_maxIs set to 6.

When NO is obtained in either step 56 or 57, that is, the first cumulative value Za
Is greater than or equal to the reference value Zo or the second cumulative value Zb is greater than or equal to the reference value Zo, it is determined that the vertical movement of at least one of the subject and the background is large in the period of the past i fields. To be done. In this case,
i is decremented by 1 (i ← i-1) (step 58). Then, if the updated value of i is not 0 (NO in step 59), the process returns to step 54 and the processes of step 54 and thereafter are executed again.

For example, i before update is 6 and after i is updated
Of i is 5, the first average value ya_n5 fees
The first cumulative value Za (= ya)₀+ Ya₁+ Ya
₂+ Ya₃+ Ya_Four) Is calculated (step 54).
Also, the second average value yb_nSecond field up to 5 fields before
Product value Zb (= yb₀+ Yb₁+ Yb₂+ Yb₃+ Y
b _Four) Is calculated (step 55). And calculated
The first and second accumulated values Za and Zb are both set to the reference value Zo.
When it is smaller (YES in steps 56 and 57),
Delay amount upper limit D_maxIs set to 5 (step 6)
0). If the calculated first and second cumulative values Za and Zb are small
If at least one is equal to or greater than the reference value Zo, i is 1
After being incremented (i ← i-1), return to step 54.
Thus, the processing after step 54 is executed again.

In this way, the delay amount upper limit value D _max is determined. When i updated in step 58 is 0 (YES in step 59), the delay amount upper limit value D _max is set to 0.

FIG. 13 shows another example of the delay amount upper limit calculation processing. Here, it is assumed that six field memories are provided and the maximum number of delay fields is six.

Of all the motion vector detection areas e _{0 to} em, the motion vector detection area in which the reliable motion vector is extracted in step 3 of FIG. 2 is referred to as the reliable motion vector detection area. Absolute value | y _k | of the vertical component of the motion vector (hereinafter referred to as the absolute value of the vertical vector) extracted in the reliable motion vector detection area (where k is a number from 0 to m, (Indicating the detection area) is calculated and the RAM 22 is calculated.
(Step 61). The RAM 22 holds at least the absolute value of the vertical vector | y _k | calculated over the current to past five fields.

Next, whether a condition (hereinafter, referred to as a high-speed motion determination condition) for determining whether or not an object that moves at high speed in the vertical direction is present in each reliable motion vector detection area is satisfied. Whether or not it is inspected, the inspection result is RAM
22 (step 62). That is, the absolute value of the vertical direction vector | y _k | calculated for each reliable motion vector detection area is a predetermined value, for example, “5”.
It is determined whether or not it is larger than (the number of pixels). If the calculated absolute value | y _k | of the vertical direction vector is larger than a predetermined value, the motion vector detection area k is
It is determined that the conditions for high speed motion determination are satisfied.

The inspection result obtained for each reliable motion vector detection area k is used as data for determining whether or not an object moving at high speed in the vertical direction is present (hereinafter referred to as determination data). , Are stored in the RAM 22 in association with the motion vector detection area k.

The determination data for each motion vector detection area k calculated this time is set to W _k0 (where k is a number from 0 to m and indicates the motion vector detection area), and already stored in the RAM.
The past four judgment data stored in _No. 22 is W _k1 ,
_Let W _k2 , W _k3 , and W _k4 . Below, the last five fields worth of the determination data W _kn containing W _k0 latest determination data stored in RAM22 (n = 0,1,2,3,4)
It is represented by.

Next, based on the determination data W _kn for the past 5 fields, it is determined whether or not there is an object moving vertically at high speed in the video within the past 5 fields. That is, for each motion vector detection region k, it is determined whether or not the high-speed motion determination condition is satisfied for a predetermined number of fields or more in the past 5 fields (step 63). The predetermined number of fields is, for example, 2
Is set.

If there is a motion vector detection area k satisfying the high-speed motion determination condition for a predetermined number of fields or more in the past 5 fields, an object moving at high speed in the vertical direction exists in that area. It is determined that the possibility is high. Therefore, in each of the motion vector detection regions k, if there is at least one motion vector detection region satisfying the high-speed motion determination condition for a predetermined number of fields in the past five fields (YES in step 64). , Past 5
It is determined that the image in the field contains an object that moves at high speed in the vertical direction.

Of the past 5 fields, 2 or more fields,
The motion vector detection area that meets the conditions for high-speed motion determination is
If it does not exist (NO in step 64),
It is determined that there is no object moving vertically in the past five fields at high speed in the vertical direction.

When it is determined that there is an object moving vertically at high speed in the images in the past 5 fields (YES in step 64), the delay amount upper limit value D _max is set to 0. (Step 65). Then, the processing of this time ends.

If it is determined that there is no object moving vertically at high speed in the images in the past 5 fields (NO in step 64), the delay amount upper limit value D _max is not set, This processing ends.

FIG. 3 shows the detailed procedure of the delay amount calculation processing in step 7 of FIG.

First, based on the delay amount multiplication set value set and stored in step 2 above and the average value v (hereinafter referred to as motion vector average value) of the effective horizontal direction motion vector obtained in step 6 above. , The first delay amount d1
Is required (step 21).

FIG. 4 shows the relationship between the motion vector average value and the delay amount. The relationship as shown in FIG. 4 is stored in the ROM 21 as a delay amount table. Then, first, the delay amount corresponding to the motion vector average value is obtained from the delay amount table.

Even with the same three-dimensional video signal, the parallax differs depending on the conditions of the stereoscopic display device (monitor), that is, the type of monitor and the conditions for viewing the monitor. Therefore, regardless of the monitor conditions, in order to obtain the same stereoscopic effect or to meet the preference of the observer, the delay amount multiplying set value set and stored in the above step 2 is used as the delay amount table. The first delay amount d1 is obtained by integrating the delay amount obtained from the above.

In order to obtain a similar stereoscopic effect regardless of the monitor conditions, a plurality of types of delay amount tables are stored, and the operation / display unit 23 is used to meet the monitor conditions or the observer's preference. You may make it input the command for selecting a delay amount table.

The first delay amount may be obtained based on a predetermined relational expression instead of the delay amount table. How to obtain the relational expression in this case will be described with reference to FIG.

A suitable distance between the monitor surface S and the eyes 31 and 32 of the observer is defined as an appropriate viewing distance A [mm]. Further, the distance between the right image R and the left image L of the gaze object on the monitor surface S is defined as parallax B [mm]. Further, the inter-eye distance is C [mm]. The suitable viewing distance A is determined by the condition of the monitor.
The parallax B of the gazing object differs depending on the monitor conditions even if the three-dimensional video signals are the same.

The suitable visual distance A, the parallax B, and the inter-eye distance C determine the stereoscopic image position P of the gaze object. That is, the protruding amount D [mm] of the gaze object from the monitor surface S is determined by the appropriate viewing distance A, the parallax B, and the interocular distance C.

The parallax B for making the protruding amount of the gazing object from the monitor surface S to be a constant amount D is expressed by the following formula 7 regardless of the condition of the monitor.

[0114]

[Equation 7]

When the horizontal length of the monitor is H [mm], the number of pixels in the horizontal direction of the monitor is h [pixels], the motion vector average value is v [pixels / field], and the first delay amount is d1 [field]. The following relationship holds.

[0116]

[Equation 8]

Here, the pixel conversion amount of the parallax B (= (h /
If H) and B) are adjustment amounts X (data relating to monitor conditions or data according to the observer's preference) set by the operation / display unit 23, the first delay amount d1 is obtained by the following relational expression. To be

[0118]

[Equation 9]

When the first delay amount d1 is obtained in step 21, the past 9 times from the present time based on the delay amount history data.
The average value of the delay amount of 10 fields from the previous time to the past 9 times, the average value of the delay amount of 10 fields from the previous time to the past 9 times, and the average value of the delay amount of 10 fields from the previous time to the past 9 times Each is calculated (Step 2
2).

The delay amount history data used in step 22 is the first delay amount d1 obtained in step 21 in the past.

Next, if two or more of the three sets of average values have the same value, that value (a large number) is selected as the second delay amount d2, and if all are different, the intermediate value thereof. Is selected as the second delay amount d2 (step 23).

Next, the second delay amount d2 selected in step 23 and one of the second delay amounts 12 to 18 fields before
The delay amount d2 (for example, the second delay amount d2 15 fields before) is compared with the second delay amount d2 30 fields before (step 24). The delay amount history data used in step 24 is the second delay amount d2 obtained in step 23 in the past.

If all the second delay amounts d2 match (YES in step 25), the target delay amount Pd is changed to the second delay amount selected in step 23 (Pd = d
2) (Step 26) and the process proceeds to Step 30. Therefore, as shown in FIG. 6, when the three second delay amounts d2 (represented by d2-1, d2-2, and d2-3 in the order from the past) change, and all the second delay amounts d2 match. , The target delay amount Pd is changed to the second delay amount (d2-3).

If all the second delay amounts d2 do not match (NO in step 25), all the second delay amounts d2 are larger than the current target delay amount Pd or all the second delay amounts d2.
It is determined whether or not 2 is smaller than the current target delay amount Pd or the conditions are not met (step 27).

When all the second delay amounts d2 are larger than the current target delay amount Pd, the target delay amount Pd is incremented by 1 (Pd = Pd + 1) (step 28), and then step 30
Proceed to. For example, as shown in FIG. 7, three second delay amounts d2 (d2-1, d2-2, d2 in order from the past one)
-3) changes and all the second delay amounts d2 are larger than the current target delay amount Pd, the target delay amount Pd is incremented by +1.

When all the second delay amounts d2 are smaller than the current target delay amount Pd, the target delay amount Pd is decremented by 1 (Pd = Pd-1) (step 29), and then step 30
Proceed to. All the second delay amounts d2 are the current target delay amounts Pd
When it is not larger and all the second delay amounts d2 are not smaller than the current target delay amount d, the routine proceeds to step 30.

At step 30, the absolute value | Pd | of the target delay amount Pd calculated at step 26, 28 or 29.
Is less than or equal to the delay amount upper limit value D _max obtained in the delay amount upper limit value calculation process (step 4 in FIG. 2). The absolute value | Pd | of the target delay amount Pd is the delay amount upper limit value D
When it is less than or equal to _max (| Pd | ≦ D _max ), the _routine proceeds to step 32.

When the absolute value | Pd | of the target delay amount Pd is larger than the delay amount upper limit value D _max (| Pd |>
D _max ), the absolute value | Pd | of the target delay amount Pd is D _max
, And then proceeds to step 32. As a result, it is possible to prevent the left-eye image and the right-eye image of the object moving in the vertical direction from not being combined.

At step 32, it is judged if the target delay amount Pd and the currently set delay amount (set delay amount d3) match. If the target delay amount Pd and the set delay amount d3 do not match, it is determined whether or not the current set delay amount d3 has already continued for four fields (step 33). When the current set delay amount d3 has already continued for 4 fields, the set delay amount d3 is changed by 1 in the direction toward the target delay amount Pd (d3 = d).
3 ± 1) (step 34). Then, step 8 in FIG.
Move to.

If it is determined in step 32 that the target delay amount and the current set delay amount match, or if the current set delay amount has not continued for 4 fields in step 33, the delay amount is set. Without changing, the process proceeds to step 8 in FIG.

That is, in this example, the set delay amount d3 is 4
It is controlled so as to approach the target delay amount Pd for each field period and for each one field.

After the power is turned on, step 21
In the above, when the first delay amount d1 is calculated for the first time, the second delay amount d2, the target delay amount Pd, and the set delay amount d3 become equal to d1.

In the process of FIG. 3, in step 22, only the average value of the delay amounts for 10 fields from this time to the past 9 times is calculated, and this is set as the target delay amount, and step 2
The processing of 3, 24, 25, 26, 27, 28, 29 may be omitted.

Further, in step 22, only the average value of the delay amounts for 10 fields in the past 9 times from this time may be calculated and used as the second delay amount, and the process of step 23 may be omitted.

The second delay amount obtained in step 23 is set as the target delay amount, and steps 24, 25, 26, 2 are performed.
The processes of 7, 28 and 29 may be omitted.

The processing of steps 22 and 23 may be omitted. In this case, the first delay amount d1 obtained in step 21 is used as the second delay amount used in step 24.

[0137]

According to the present invention, it is possible to prevent the left-eye image and the right-eye image of an object moving in the vertical direction from not being fused.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a 2D / 3D conversion device.

FIG. 2 is a flowchart showing an overall procedure of 2D / 3D conversion processing by a CPU.

3 is a flowchart showing a detailed procedure of a delay amount calculation process in step 7 of FIG.

FIG. 4 is a graph showing a relationship between a motion vector average value and a first delay amount.

FIG. 5 is a schematic diagram for explaining how to derive a relational expression for obtaining a first delay amount from a motion vector average value.

FIG. 6 is a time chart showing how the target delay amount is changed when all the three second delay amounts match.

FIG. 7 is a time chart showing how the target delay amount is changed when all of the three second delay amounts become larger than the current target delay amount.

FIG. 8 is a schematic diagram showing a plurality of motion vector detection areas E _{1 to} E ₁₂ set in a video area of each field.

FIG. 9 is a schematic diagram showing a plurality of small areas e in a motion vector detection area.

FIG. 10 is a schematic diagram showing a plurality of sampling points S set in a small area e and one representative point R.

FIG. 11 is a flowchart showing a delay amount upper limit calculation processing procedure in step 4 of FIG.

FIG. 12 is a flowchart showing another example of the delay amount upper limit value calculation processing.

FIG. 13 is a flowchart showing still another example of the delay amount upper limit value calculation processing.

[Explanation of symbols]

11 field memory 12 Interpolation circuit 13 Video switching circuit 14, 15 Phase control circuit 20 CPU 21 ROM 22 RAM 23 Operation / display 24 Memory control circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshiya Iinuma 2-5-5 Keihan Hondori, Moriguchi City, Osaka Prefecture Sanyo Electric Co., Ltd. (72) Inventor Yukio Mori 2-5 Keihan Hondori, Moriguchi City, Osaka Prefecture No. 5 In Sanyo Electric Co., Ltd. (72) Inventor Akihiro Maenaka 2-5-5 Keihan Hon-dori, Moriguchi City, Osaka Prefecture Sanyo Electric Co., Ltd. (72) Seiji Okada 2-chome, Keihan Hon-dori, Moriguchi City, Osaka Prefecture No. 5 In Sanyo Electric Co., Ltd. (72) Inventor Hidekazu Uchida 2-5-5 Keihan Hondori, Moriguchi City, Osaka Prefecture Sanyo Denki Software Co., Ltd. (72) Inventor Kanji Ihara 2 Keihan Hondori, Moriguchi City, Osaka Prefecture 5-5 Sanyo Electric Software Co., Ltd. (56) Reference JP-A-7-264631 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 13/02 G02B 27 /twenty two

Claims (5)

(57) [Claims] 1. A method of converting a 2D video into a 3D video by generating a main video signal and a sub video signal delayed from the main video signal from the 2D video signal, In the method of converting a 2D image into a 3D image in which the delay amount of the sub image signal with respect to the main image signal is changed according to the speed of horizontal movement of the main image, a vertical component of a motion vector detected from the main image. A method of converting a two-dimensional image into a three-dimensional image, characterized in that the upper limit value of the delay amount is determined based on the above, and the delay amount is determined so as to be equal to or less than the determined upper limit value. 2. An average value of absolute values of vertical components of all or some of motion vectors detected from each of a plurality of motion vector detection areas set in a video area of a main video is calculated for each field. The upper limit value of the delay amount is determined within each field in which the cumulative value of the average values stored in each field from the present field to the past is less than a predetermined value. A method for converting the described two-dimensional image into a three-dimensional image. 3. An average value of absolute values of vertical components of all or some of motion vectors detected from each of a plurality of motion vector detection areas set in a video area of a main video is calculated for each field, and First stored in storage means
Step, of the average value of the past predetermined number of fields stored in the storage means, the cumulative number of fields indicating how many fields from the latest one are accumulated is set to a predetermined maximum delay value. Second step of setting, from the average values for the predetermined number of past fields stored in the storage means, the average values for the set cumulative fields are read in order from the newest, and the cumulative values are calculated. If the calculated cumulative value is less than a predetermined reference value, the currently set cumulative field number is determined as the delay amount upper limit value, and the calculated cumulative value is predetermined. If it is equal to or greater than the reference value, the cumulative field number is updated in the fourth step of decrementing the currently set cumulative field number by 1 and the fourth step. And a fifth step of repeatedly executing the processing of steps 3 and 4 by using the updated cumulative number of fields, the two-dimensional video according to claim 1 being a three-dimensional video. How to convert to. 4. An absolute value of a vertical component of all or some of the motion vectors detected from a region in which a subject is determined to be present among a plurality of motion vector detection regions set in the video area of the main image. A first step of calculating the average value of each of the fields and storing the calculated average value as a first average value in the storage means, each of the plurality of motion vector detection areas from an area determined to have a background. A second step of calculating an average value of absolute values of vertical components of all or part of the detected motion vector for each field, and storing the calculated average value as a second average value in the storage means; Of the first average value and the second average value for the predetermined number of past stored fields, a cumulative file indicating how many fields from the latest one are accumulated. The third step of setting the number of fields to a predetermined maximum delay amount, the first average value of the set cumulative field number from the first average value of the predetermined number of past fields stored in the storage means. A fourth step of reading the average values in order from the newest and calculating a first cumulative value, which is their cumulative value, setting from among the second average values for the predetermined number of past fields stored in the storage means. The fifth step of reading out the second average values for the accumulated number of accumulated fields in order from the newest one, and calculating the second accumulated value that is the accumulated value of them, of the calculated first accumulated value and the second accumulated value. If both are less than the predetermined reference value, the currently set cumulative field number is determined as the delay amount upper limit value, and at least one of the calculated first cumulative value and second cumulative value is predetermined. Was If it is equal to or more than the quasi-value, the currently set cumulative field number is decremented by 1, and when the cumulative field number is updated in the sixth step, the updated cumulative field number is used. The method for converting a two-dimensional image into a three-dimensional image according to claim 1, further comprising: a seventh step of repeating the processes of steps 4, 5 and 6. 5. The absolute value of the vertical component of the motion vector detected from all or a part of the plurality of motion vector detection areas set in the video area of the main video is calculated for each field and calculated. First step of determining whether or not the absolute value is larger than a predetermined value. If the calculated absolute value is larger than the predetermined value, whether or not there is an object with large vertical movement in the main image. A second step of determining that the condition for determining is satisfied, and storing the determination result in the storage means in association with the motion vector detection area in which the motion vector that is the calculation source of the absolute value is detected, Based on the determination result for the predetermined number of past fields stored in the storage means, the above condition is satisfied for a predetermined prescribed number of fields or more out of a predetermined number of past fields in each motion vector detection area. Plus a third step of determining whether the motion vector detection region is present which, as well as among the motion vector detection areas in the field of past predetermined number,
The fourth step of deciding the delay amount upper limit value to 0 when there is a motion vector detection area satisfying the above condition for a predetermined number of prescribed fields or more. A method for converting the described two-dimensional image into a three-dimensional image.