JP3454684B2 - Apparatus for converting 2D video to 3D video - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention
The present invention relates to a device for converting a three-dimensional image.

[0002]

2. Description of the Related Art As a method for converting a two-dimensional image into a three-dimensional image, a two-dimensional image signal is used by using a plurality of field memories and a main image signal and a sub-image delayed with respect to the main image signal in time. A signal is generated and one of the main video signal and the sub video signal is output as a left-eye video signal, and the other is output as a right-eye video signal.

The time delay of the sub video signal with respect to the main video signal (hereinafter referred to as the delay amount) is determined based on the speed of horizontal movement of the main video signal. That is, the larger the horizontal movement speed of the main image, the smaller the delay amount. Also,
Which of the main image and the sub-image is the left-eye image signal and which is the right-eye image signal is determined based on the horizontal movement direction (left or right) of the main image.

Since such a method can convert only a moving two-dimensional image into a three-dimensional image, this method will be referred to as a moving image 2D / 3D image converting method.

The applicant of the present invention has developed a method for converting a non-moving two-dimensional image into a three-dimensional image (hereinafter referred to as a still image 2D / 3).
D video conversion method) has already been developed and applied (Japanese Patent Application No. 8-208173, Japanese Patent Application No. 9-159949).
issue). However, these applications have not been published yet.

This 2D / 3D video conversion method for still images is
Based on the 2D input video signal, for each field,
For each of the plurality of parallax calculation areas set in the one-field screen, the image feature amount relating to the perspective of the video is extracted, and based on the image feature amount extracted for each parallax calculation region, the one-field screen is displayed. Parallax information for each predetermined unit area of the two-dimensional input video signal is generated, and the first video signal and the first video signal having a horizontal phase difference corresponding to the parallax information corresponding to the predetermined unit area from the signal in each predetermined unit area of the two-dimensional input video signal. Two video signals are generated respectively.

2D / 3 for a still image with reference to FIGS.
An example of the D image conversion method will be described.

FIG. 1 shows the overall structure of a 2D / 3D image conversion apparatus for converting a 2D image into a 3D image.

A luminance signal Y, a color difference signal RY and a color difference signal B-Y, which form a two-dimensional video signal, are converted into an AD conversion circuit 1.
(ADC) digital Y signal, R-
It is converted into a Y signal and a BY signal.

The Y signal is sent to the high frequency component integrating circuit 8 and the brightness contrast calculating circuit 9 and also to the first left image arbitrary pixel delay FIFO 11 and the first right image arbitrary pixel delay FIFO 21. The RY signal is sent to the RY component integrating circuit 31, and the second
To the left image arbitrary pixel delay FIFO 12 and the second right image arbitrary pixel delay FIFO 22. The BY signal is sent to the BY component integrating circuit 32, and the
Are sent to the left image arbitrary pixel delay FIFO 13 and the third right image arbitrary pixel delay FIFO 23.

The high-frequency component integrating circuit 8 calculates the integrated value of the high-frequency component for each field, as shown in FIG. 2, for each of the plurality of parallax calculation areas E1 to E12 preset in the one-field screen. . The brightness contrast calculation circuit 9 is configured to detect each parallax calculation area E1 for each field.
The brightness contrast for each of E12 to E12 is calculated. The RY component integrating circuit 31 calculates the integrated value of the RY components for each of the parallax calculation regions E1 to E12 for each field. The BY component integration circuit 32 calculates the integrated value of the BY component for each of the parallax calculation regions E1 to E12 for each field.

Integrated values of high-frequency components for the respective parallax calculation areas E1 to E12, and the respective parallax calculation areas E1 to E12.
Luminance contrast for each, each parallax calculation area E
The integrated value of the RY component for each of the parallax calculation areas E1 to E12 and the integrated value of the BY component for each of the parallax calculation areas E1 to E12 are used as image feature amounts regarding the perspective of the image for each of the parallax calculation areas E1 to E12.

Actually, a total of 60 parallax calculation regions of 6 rows and 10 columns are set in the 1-field screen as shown in FIG. 13, but for convenience of explanation, as shown in FIG. In addition, it is assumed that a total of 12 parallax calculation areas E1 to E12 of 3 rows and 4 columns are set in one field screen.

Based on the information sent from the high frequency component integrating circuit 8, the brightness contrast calculating circuit 9, the RY component integrating circuit 31, and the BY component integrating circuit 32, the CPU 3 calculates the parallax calculation areas E1 to E12. Disparity information for the. In this example, parallax information is generated such that an object on the front side such as a subject has a smaller parallax amount, and an object on the back side such as a background has a larger parallax amount. Details of the method of generating the parallax information will be described later.

The parallax information for each parallax calculation area E1 to E12 calculated by the CPU 3 is sent to the parallax control circuit 4. The parallax control circuit 4 uses the parallax calculation areas E1 to E.
Based on the parallax information for 12, the parallax information for each pixel position in each field is generated. Then, based on the obtained parallax information for each pixel position, each FIFO 11 to
13, 21 to 23 to video signals (Y signal, RY signal,
Each FI is arranged such that the read address when reading the (B-Y signal) is deviated between the left video arbitrary pixel delay FIFOs 11 to 13 and the right video arbitrary pixel delay FIFOs 21 to 23.
The read addresses of the FOs 11 to 13 and 21 to 23 are controlled. Therefore, the left image arbitrary pixel delay FIFO 11
The horizontal phase of the left video signal read from each of the image data sets 13 to 13 and the horizontal phase of the right video signal read from each of the right image arbitrary pixel delay FIFOs 21 to 23 become different.

Left image arbitrary pixel delay FIFOs 11 to 13
Left video signal (YL signal, (RY) L read from
Signal, (BY) L signal) is a DA conversion circuit (DAC)
After being converted into an analog signal by 5, it is sent to a stereoscopic display device (not shown). Arbitrary pixel delay FIFO for right image
The right video signal (YR signal,
The (RY) R signal and the (BY) R signal) are converted into analog signals by the DA conversion circuit (DAC) 6 and then,
It is sent to a stereoscopic display device (not shown).

Since the horizontal phase of the left video signal is different from the horizontal phase of the right video signal, parallax occurs between the left video and the right video. As a result, the left image is observed only with the left eye,
If the right image is observed only by the right eye, a stereoscopic image in which the subject is in the front position with respect to the background is obtained.

FIG. 3 shows the configuration of the RY component integrating circuit 31.

In FIG. 2, the parallax calculation areas E1 to E are shown.
The number of horizontal pixels of 12 is m, and each of the parallax calculation areas E1 to E
The number of 12 vertical pixels is n, and the first parallax calculation region E1
The horizontal position (HAD) with the upper left coordinates of (a, b)
And the vertical position (VAD) is represented.

The RY component integrating circuit 31 includes a timing signal generating circuit 201, an adding circuit 202, an RY component integrating register group 203, and a selecting circuit (SEL) 204. The RY component integration register group 203 includes first to first corresponding parallax calculation areas E1 to E12, respectively.
Two RY component integration registers 211 to 222 are provided.

To the timing signal generation circuit 201, the horizontal synchronizing signal Hsync and the vertical synchronizing signal Vsync of the input video signal and the clock signal CLK for detecting the horizontal address of each horizontal period are input.

The timing signal generating circuit 201 outputs first to twelfth enable signals EN1 to EN12, a reset signal RST and an output timing signal DOUT based on the horizontal synchronizing signal Hsync, the vertical synchronizing signal Vsync and the clock signal CLK. .

The enable signals EN1 to EN12 respectively correspond to the parallax calculation areas E1 to E12, are normally at the L level, and are H when the horizontal and vertical positions of the input video signal are within the corresponding areas. It becomes a level. The first to twelfth enable signals EN1 to EN12 are input as write signals to the first to twelfth luminance integration registers 211 to 222, respectively. The first to twelfth enable signals EN1 to EN12 are also sent to the selection circuit 204. The selection circuit 204 selects and outputs the input data corresponding to the H level enable signal.

The reset signal RST is output at the effective video start timing of each field in the input video signal and sent to each RY component integration register 211-222. When the reset signal RST is input to each of the RY component integration registers 211 to 222, the content thereof is set to 0.

As shown in FIG. 2, the output timing signal DOUT becomes the H level for a certain period from the time when the vertical position of the input video signal exceeds the vertical position of the lower end of the parallax calculation area E12 at the bottom. Output timing signal DOUT
Is sent to the CPU 3.

A reset signal is output at the effective image start timing of the input image signal, and the contents of the respective RY component integration registers 211 to 222 are set to zero. When the horizontal and vertical positions of the input video signal are within the first parallax calculation area E1, the first enable signal EN1 is at the H level, and therefore is held in the first RY component integration register 211. The RY value is added to the addition circuit 202 via the selection circuit 204.
And the RY signal in the input video signal is input to the adding circuit 202.

Therefore, the first brightness integrating register 21
The RY value held at 1 and the RY signal in the input video signal are added by the adder circuit 202, and the addition result is stored in the first RY component integration register 211. That is, when the horizontal and vertical positions of the input video signal are within the first parallax calculation area E1, the RY values of the pixels within the first parallax calculation area E1 are integrated, and the integration result is the 1 is stored in the R-Y component integration register 211.

In this way, the parallax calculation areas E1 to E
The RY component integrated value for each 12 is stored in the corresponding RY component integration registers 211 to 222. Then, when the output timing signal DOUT becomes H level, each R-Y
The RY component integrated value for each parallax calculation region E1 to E12 accumulated in the component integrating registers 211 to 222 is C
It is sent to PU3 via a data bus (DATA-BUS).

The configuration of the BY component integrating circuit 32 is similar to that of the RY component integrating circuit 31 of FIG.

FIG. 4 shows the configuration of the high frequency component integrating circuit 8.

The high frequency component integrating circuit 8 includes a timing signal generating circuit 231, a high pass filter (HPF) 232,
An absolute value conversion circuit 233, a slice processing circuit 234, an addition circuit 235, a high frequency component integration register group 236 and a selection circuit 237 are provided. The high frequency component integration register group 236 includes the first to twelfth high frequency component integration registers 241 to 241 respectively corresponding to the parallax calculation areas E1 to E12.
252 is provided.

The input signal and the output signal of the timing signal generation circuit 231 are the same as the timing signal generation circuit 20 of FIG.
1 is the same as the input signal and the output signal.

As the high pass filter 232, for example, as shown in FIG. 5, five D flip-flops 26 are provided.
1 to 265, a high-pass filter having tap coefficients of -1, 0, 2, 0, and -1, which includes a bit shift circuit 266 for obtaining an output twice the input value, an adder 267, and a subtracter 268, is used. To be

As the slice processing circuit 234,
A circuit having an input / output characteristic as shown in FIG. 6 is used. The output is set to 0 for the input of 0 to Ia in order to prevent noise from being extracted as a high frequency component.

Therefore, the high frequency component of the Y signal in the input video signal is extracted by the high pass filter 232, the absolute value thereof is obtained by the absolute value conversion circuit 233, and the noise is removed from the absolute value of the high frequency component by the slice processing circuit 234. It

A reset signal is output at the effective video start timing in the input video signal, and the contents of the respective high frequency component integration registers 241 to 252 are set to zero. When the horizontal and vertical positions of the input video signal are within the first parallax calculation area E1, the first enable signal EN1 is at the H level, so the high frequency component held in the first high frequency component integration register 241. Via the selection circuit 237
5, and the high frequency component of the Y signal in the input video signal (the output of the slice processing circuit 234) is input to the adding circuit 235.

Therefore, the high frequency component held in the first high frequency component integration register 241 and the high frequency component of the Y signal in the input video signal are added by the addition circuit 235, and the addition result is the first high frequency component integration. It is stored in the register 241. That is, when the horizontal and vertical positions of the input video signal are in the first parallax calculation area E1, the high frequency components of the pixels in the first parallax calculation area E1 are integrated, and the integration result is the first It is stored in the high frequency component integration register 241.

In this way, the parallax calculation areas E1 to E
The integrated value of the high frequency component for each 12 is stored in the corresponding high frequency component integration registers 241 to 252. Then, when the output timing signal DOUT becomes H level, the integrated value of the high frequency components for each of the parallax calculation regions E1 to E12 accumulated in the respective high frequency component integration registers 241-252 is sent to the CPU 3 via the data bus. .

FIG. 7 shows another example of the high frequency component integrating circuit 8.

The high frequency component integrating circuit 8 includes a timing signal generating circuit 238, a high pass filter 232, a peak detecting circuit 239, an adding circuit 235, a high frequency component integrating register group 236 and a selecting circuit 237.

The timing signal generation circuit 238 is almost the same as the timing signal generation circuit 201 in FIG. 3, but as shown in FIG. 2, the horizontal position of the input video signal is immediately before the parallax calculation areas E1, E5, E9. 3 is different from the timing signal generation circuit 201 in FIG. 3 in that a trigger pulse (region boundary signal RST1) is output when the horizontal position and the last horizontal position of each parallax calculation region E1 to E12 are reached. . The area boundary signal RST1 is supplied to the peak detection circuit 23.
Sent to 9.

The high frequency component of the Y signal extracted by the high pass filter 232 is sent to the peak detection circuit 239. The peak detection circuit 239 uses the parallax calculation areas E1 to E1.
The maximum value of the high frequency component is detected for each horizontal line in E12. As the peak detection circuit 239, as shown in FIG. 8, a circuit including a comparison circuit 271, a maximum value register 272 and a gate 273 is used. FIG. 9 shows the horizontal synchronizing signal Hsync of the input video signal and the area boundary signal RS.
The outputs of T1, the gate 273, etc. are shown.

The maximum value register 272 has a high frequency component of the Y signal extracted by the high pass filter 232, a region boundary signal RST1, and a judgment result signal L of the comparison circuit 271.
a and the clock signal CLK are input. Comparison circuit 2
Reference numeral 71 compares the output of the maximum value register 272 with the high frequency component of the Y signal in the input video signal, and when the high frequency component of the Y signal is larger than the output of the maximum value register 272,
The determination result signal La is set to H level.

When the region boundary signal RST1 becomes H level, the content of the maximum value register 272 is set to 0. If the determination result signal La from the comparison circuit 271 is at H level when the region boundary signal RST1 is at L level,
The high frequency component of the Y signal is stored in the maximum value register 272. That is, the content of the maximum value register 272 is updated. Therefore, in the maximum value register 272, the parallax calculation regions E1 to E12 corresponding to the horizontal and vertical positions of the input video signal are stored in the maximum value register 272 every period when the region boundary signal RST1 is at the L level.
The maximum value of the high frequency components of the Y signal for each pixel on one horizontal line is stored.

The gate 273 outputs the output value of the maximum value register 272 when the area boundary signal RST1 becomes H level, and outputs 0 when the area boundary signal RST1 is L level. That is, from the gate circuit 273, each time the area boundary signal RST1 becomes H level, the predetermined parallax calculation areas E1 to E1 stored in the maximum value register 272 are stored.
The maximum value of the high frequency components of the Y signal for one horizontal line in 2 is output. Therefore, in each of the high frequency component integration registers 241-252 (see FIG. 7), the integrated value of the maximum values of the high frequency components of the Y signal for each horizontal line in the corresponding parallax calculation area is accumulated.

FIG. 10 shows the configuration of the brightness contrast calculation circuit 9.

The brightness contrast calculation circuit 9 includes a timing signal generation circuit 301 and a brightness contrast detection circuit group 302. Brightness contrast detection circuit group 3
Reference numeral 02 denotes the first to twelfth luminance contrast detection circuits 311 to 312 corresponding to the parallax calculation areas E1 to E12, respectively.
22 is provided.

The input signal and the output signal of the timing signal generation circuit 301 are the same as the timing signal generation circuit 20 of FIG.
1 is the same as the input signal and the output signal.

Brightness contrast detection circuits 311 to 32
As shown in FIG. 11, the second circuit 2 includes a first comparison circuit 331, a maximum value register 332, a second comparison circuit 333, a minimum value register 334, and a subtractor 335.

In the maximum value register 332, the Y signal in the input video signal and the enable signals EN (N = 1, N = 1, for the areas E1 to E12 corresponding to the luminance contrast detection circuit) are stored.
2 ... 12), reset signal RST, first comparison circuit 33
1 output from the determination signal Lb and the clock signal CL
K is inputting. The first comparison circuit 331 compares the output value of the maximum value register 332 with the Y signal of the input video signal, and when the Y signal of the input video signal is larger than the output value of the maximum value register 332, the determination signal Lb is set to H. To level.

When the reset signal RST goes high,
The content of the maximum value register 332 is set to 0. The enable signal EN of the areas E1 to E12 corresponding to the brightness contrast detection circuit is at H level and the determination signal Lb is at H level.
At the level, the Y signal is stored in the maximum value register 332. That is, the contents of the maximum value register 332 are updated. Therefore, immediately before the output timing signal DOUT is output, the maximum value register 332 stores the parallax calculation area E corresponding to the luminance contrast detection circuit.
The maximum value of the brightness values of each pixel in 1 to E12 is accumulated.

In the minimum value register 334, the Y signal in the input video signal and the enable signals EN (N = 1, N = 1,
2 ... 12), reset signal RST, second comparison circuit 33
The determination signal Lc and the clock signal CL output from 3
K is inputting. The second comparison circuit 333 compares the output value of the minimum value register 334 with the Y signal of the input video signal, and when the Y signal of the input video signal is smaller than the output value of the minimum value register 334, sets the determination signal Lc to H level. To level.

When the reset signal RST goes high,
A predetermined maximum value is set in the minimum value register 334. When the enable signal EN of the areas E1 to E12 corresponding to the luminance contrast detection circuit is at H level and the determination signal Lc is at H level, the Y signal is stored in the minimum value register 334. That is, the minimum value register 3
The contents of 34 are updated. Therefore, immediately before the output timing signal DOUT is output, the minimum value register 334 stores the minimum value of the brightness values of the pixels in the parallax calculation areas E1 to E12 corresponding to the brightness contrast detection circuit. It

As a result, at the time when the output timing signal DOUT is output, the output of the subtractor 335 is the maximum value and the minimum value of the luminance values of the pixels in the corresponding parallax calculation areas E1 to E12. The value corresponds to the difference (luminance contrast). Then, the output timing signal DOUT
Is output, the output of the subtractor 335 (luminance contrast) is sent to the CPU 3.

FIG. 12 shows a procedure of parallax information generation processing for each parallax calculation area performed by the CPU 3.

In the parallax information generation processing for each divided area, grouping processing (step 1), spatial separation processing (step 2), singular point processing (step 3), inter-group coupling processing (step 4), and group-by-group processing are performed. Depth information generation processing (step 5), depth information correction processing for all areas (step 6), depth information correction processing for group boundaries (step 7), depth information correction processing for inside groups (step 8), and disparity information calculation processing ( Step 9) is performed.

The parallax information generation process will be described by taking as an example the 60 parallax calculation regions actually set for one field. FIG. 13 shows 60 parallax calculation areas F1 to F60 that are actually set for one field.

(1) Description of grouping process

The grouping process of step 1 is the first process carried out for the purpose of grouping all the regions forming one image for each object included in the image.

There are two methods for grouping as described below.

(1-1) First Method First, the high frequency integrated value obtained for each of the parallax calculation areas F1 to F60 is normalized to a value within a predetermined range (for example, 0 to 20). Then, a distribution (histogram) of the number of parallax calculation regions belonging to each normalized value of the high frequency integrated value is generated. Figure 1
4 shows an example of the generated histogram. And
Parallax calculation areas included in a mountain between valleys of the histogram are set as one group. Luminance contrast may be used instead of the high frequency integrated value. Figure 15
Indicates the result of grouping the parallax calculation regions F1 to F60 in this way. In FIG.
The numbers G1 to G4 indicate group numbers.

(1-2) Second Method The integrated value of the RY components obtained for each of the parallax calculation regions F1 to F60 is normalized to a value in the range of 0 to 20. And R
Generate a distribution (histogram) of the number of parallax calculation regions belonging to each normalized value of the Y component integrated value. Based on this histogram, the boundary value between the groups is obtained from the normalized value of the R-Y component integrated value.

The integrated value of the BY component obtained for each of the parallax calculation areas F1 to F60 is normalized to a value in the range of 0 to 10. Then, a distribution (histogram) of the number of parallax calculation regions belonging to each normalized value of the BY component integrated value is generated. Based on this histogram, the boundary value between the groups is obtained from the normalized value of the BY component integrated value.

Then, as shown in FIG. 16, the two types of boundary values thus obtained are used to group all parallax calculation regions. 17 and 18 show the results of grouping the parallax calculation regions F1 to F60 in this way. 17 and 18, G
1 to G5 indicate group numbers.

In this example, it is assumed that the grouping process is performed by the second method.

(2) Description of space separation processing

In the spatial separation processing of step 2, among the parallax calculation areas belonging to the same group by the processing of step 1, spatially adjacent parallax calculation areas are made into one group. That is, even in the parallax calculation areas that belong to the same group by the process of step 1, the parallax calculation areas that are spatially separated by another group are separated into different groups.

Specifically, as shown in FIG. 19, in step 1, the parallax calculation areas that are determined to belong to group 3 (G3) are group 31 (G31) and group 32.
(G32) and group 33 (G33) are separated into three groups.

(3) Description of singular point processing

In this singular point processing, when there is a group composed of only one parallax calculation area, does this one parallax calculation area correspond to an object different from other adjacent groups? , It is determined whether or not it corresponds to the same object as another adjacent object.

For example, as shown in FIG. 20, it is assumed that a certain group is composed of only one parallax calculation area A. The two parallax calculation areas in the upward direction of the parallax calculation area A are selected from the one closest to the parallax calculation area A to U.
1 and U2. The two parallax calculation areas in the lower direction of the parallax calculation area A are selected from the one closest to the parallax calculation area A to D1,
D2. The two parallax calculation areas to the left of the parallax calculation area A are L1 and L2 in order from the one closest to the parallax calculation area A.
And The two parallax calculation areas to the right of the parallax calculation area A are set to R1 and R2 from the one that is closer to the parallax calculation area A.

In this case, the color distance between the region A and the regions U1, D1, L1, and R1 which are one outside the region A is one outside the region A in each of the up, down, left, and right directions centering on the region A. Areas U1, D1, L1, R1 and more
If it is larger than the color distances to the outer regions U2, D2, L2, and R2, it is determined that the group formed of only the region A forms one group independently. Otherwise, the area A is determined to belong to the surrounding groups. That is, the grouping is corrected.

The definition of the color distance will be described. The BY component integrated value and the RY component integrated value with respect to a certain parallax calculation region Fa are represented by Fa (BY) and Fa (RY), respectively, and the BY component integrated value with respect to a certain parallax calculation region Fb.
The R-Y component integrated values are Fb (B-Y) and Fb (R
-Y), the color distance di between the area Fa and the area Fb
st is defined by the following mathematical formula 1.

[0074]

[Equation 1]

For example, if the (BY component integrated value, RY component integrated value) of the region A in FIG. 20 is (-4, 5), the region U1
((B-Y component integrated value, RY component integrated value)) is (-5,
4), it is assumed that the (BY component integrated value, RY component integrated value) of the region U2 is (−7, 2). Area A and area U1
And the color distance dist between the areas U1 and U2 is "4".

If the group 31 (G31) in FIG. 19 is composed of only one parallax calculation area and it is determined by the singular point processing as described above that the group 31 (G31) belongs to the group 1 (G1), it is shown in FIG. The grouping is modified as follows.

(4) Description of intergroup connection processing

In the inter-group coupling process of step 4, first, for each group, the average value of the R-Y component integrated values and the average value of the BY component integrated values of the parallax calculation regions forming the group are calculated. It

Next, the color distance between two adjacent groups is calculated. That is, two adjacent groups are Ga and Gb. If the group Ga is composed of n parallax calculation regions a1, a2, ... An, the average value of the BY component integrated values of the group Ga * Ga
Average value of (BY) and RY component integrated values * Ga (R
-Y) is calculated by the following Equation 2.

[0080]

[Equation 2]

If the group Gb is composed of m pieces of parallax calculation areas b1, b2, ... Bm, the mean value * Gb (BY) and R- of the BY component integrated values of the group Gb. The average value * Gb (RY) of the Y component integrated value is obtained by the following mathematical formula 3.

[0082]

[Equation 3]

The color distance dist between the group Ga and the group Gb is defined by the following mathematical formula 4.

[0084]

[Equation 4]

Then, it is judged whether or not the color distance between two adjacent groups is smaller than the threshold value. When the color distance is smaller than the threshold value, these two groups are combined. That is, these two groups are combined into one group.

(5) Description of depth information generation processing for each group

In the depth information generation processing for each group in step 5, first, the integrated value of the high frequency components obtained for each of the parallax calculation regions F1 to F60 is normalized to a value in the range of 0 to 10. The brightness contrast obtained for each of the parallax calculation regions F1 to F60 is normalized to a value in the range of 0 to 10.

Then, the normalized value of the obtained integrated value of the high frequency components, the normalized value of the luminance contrast, and the background weight component given in advance for each of the parallax calculation regions F1 to F60 are obtained as shown in FIG. Based on this, depth information for each group is generated.

A method of generating depth information for any one group will be described. First, the number n of parallax calculation areas belonging to the group is calculated. Further, the sum Σa of the normalized value a of the integrated value of the high frequency components for the parallax calculation region belonging to the group is calculated. In addition, the sum Σb of the normalization values b of the brightness contrast for the parallax calculation area belonging to the group is calculated. Also,
The sum Σc of the background weight components c for the parallax calculation area belonging to the group is calculated.

Then, based on the following equation 5, depth information H for the group is generated.

[0091]

[Equation 5]

In Equation 5, K1, K2 and K3 are coefficients, for example, K1 = 3/8, K2 = 1/8,
K3 is set to 4/8.

(6) Description of depth information correction processing for all areas

In the depth information correction processing for each group in step 5, first, the average value of the depth information is calculated for each row of the parallax calculation areas F1 to F60. If the depth information for each of the parallax calculation regions F1 to F60 has a value as shown in FIG. 23, for example, the average value of the depth information for each of the first to sixth rows is 1.2, 3. .6, 6.0, 7.
It becomes 2, 4.0 and 1.2.

Next, of each line of the parallax calculation region, a region in which many objects at the front position are reflected is extracted. That is,
The row with the largest average depth information is extracted. Figure 2
In the example of No. 3, the area of the fourth row is extracted.

Next, regarding each area of the row below the extracted row, each area of the row below the extracted row is prevented so that the depth information does not suddenly become smaller than the area immediately above. Depth information is adjusted. In particular,
For a region where the depth information of each region in the row below the extracted line is 3 or more smaller than the region immediately above, the depth information of that region is set to a value that is 2 smaller than the depth information of the region immediately above. Can be changed.

In the example of FIG. 23, as shown in FIG. 24, first of all, among the areas F41 to F50 in the fifth row, the depth information is smaller than the depth information of the area immediately above by three or more areas F42 to F49. , The depth information is corrected.
Thereafter, of the regions F51 to F60 in the sixth row, the depth information is corrected for the regions F53 to F58 whose depth information is smaller than the depth information of the region immediately above (corrected depth information) by 3 or more. To be done.

That is, when the relationship of the depth information with respect to the height of the screen at an arbitrary horizontal position is as shown by the curve U1 in FIG. 25, the relationship between the depth information and the height of the screen is corrected by the depth correction. However, in Fig. 25, the curve U
It is corrected so that the relationship shown in FIG.

In this way, of each line in the parallax calculation area,
The depth information in the lower region than the region in which many objects at the front position are reflected is corrected for the following reason.

In general, an object existing in front is often reflected on the lower side of the screen. In addition, the object displayed on the lower side of the screen is often an image with little change such as the ground. An image with little change such as the ground has a low high-frequency component, and therefore has a small depth information value even though it is in front. Therefore, by depth correction, the depth information for an image in front of which the high-frequency component is low is made large enough not to become larger than the value of the depth information of the region immediately above.

(7) Description of depth information correction processing for group boundaries

At the boundary between two adjacent groups, grouping may not be performed correctly. Further, when the depth estimation value for each group greatly differs at the boundary between two adjacent groups, image distortion becomes remarkable.

Therefore, in the depth information correction processing for the group boundary in step 7, first, for each boundary portion between two adjacent groups, the depth information of the parallax calculation area of one group and the parallax calculation area of the other group are first described. It is determined whether or not the difference between the depth information and the depth information is greater than or equal to a predetermined value. If the difference in depth information between the two is greater than or equal to a predetermined value, the depth information with the smaller depth information (that is, the one positioned rearward) is set so that the difference in depth information between the two is smaller than the predetermined value. Increase the depth information for the parallax calculation area.

(8) Description of depth information correction processing for the inside of the group

By the correction processing in steps 6 and 7, the depth information varies depending on the regions even within the same group. When this difference becomes large, the image distortion becomes remarkable. Therefore, in the depth information correction process for the inside of the group in step 8, the depth estimated value within the group is smoothed for each group.

That is, as shown in FIG. 26, in the same group, the attention area is A, the depth information for the attention area is HA, and the four adjacent areas are U, D, L, and
R, depth information for them is HU, HD, HL, H
When R is set, the depth estimation value HA for the attention area A is corrected by the following Expression 6.

[0107]

[Equation 6]

Each parallax calculation region F obtained in this way
The depth information for each of 1 to F60 is normalized again within the range of 0 to 10.

(9) Description of parallax information calculation processing

In the parallax information calculation processing in step 9, the depth information for each parallax calculation area F1 to F60 is calculated for each area F1.
~ Converted into parallax information for each F60.

That is, the depth information is converted into the parallax information for each of the areas F1 to F60 based on the relationship between the depth information and the parallax information set in advance. The relationship between depth information and disparity information is shown in FIG.
As shown by 2, the relationship is inversely proportional.

In FIG. 27, the relationship between the depth information and the parallax information indicated by the straight line S1 is used when it is desired to obtain a stereoscopic image having a relatively strong stereoscopic effect. The relationship between the depth information and the parallax information indicated by the straight line S2 is used when it is desired to obtain a stereoscopic image with a relatively weak stereoscopic effect. The relationship between the depth information and the parallax information is shown by a straight line S1 and a straight line S2.
It is possible to adjust the stereoscopic effect by adjusting between and.

The parallax information for each parallax calculation region thus obtained is sent to the parallax control circuit 4 (see FIG. 1).

FIG. 28 mainly shows the configuration of the parallax control circuit and the arbitrary pixel delay FIFO shown in FIG. In the following, the parallax calculation area is set to E1 as shown in FIG.
~ E12 will be described.

FIG. 28 shows the arbitrary pixel delay FIFOs 11 to 11.
13, 21 to 23, the left video arbitrary pixel delay FIFO 11 and the right video arbitrary pixel delay FI with respect to the Y signal
Only FO21 is shown, but other arbitrary pixel delays FI
Since the FOs 12, 13, 22, and 23 have the same configuration and the same control is performed, other arbitrary pixel delay FIFOs are also provided.
The description of the configurations and control methods of 12, 13, 22, and 23 will be omitted.

By the way, the parallax information calculated by the CPU 3 is the parallax information for the center position of each of the parallax calculation areas E1 to E12. The parallax control circuit 4 obtains the parallax information for each pixel position on the one-field screen based on the parallax information for the center positions of the parallax calculation regions E1 to E12. Then, in order to generate a left image and a right image having parallax according to the parallax information for the pixel position from the two-dimensional image signal for each pixel position, an arbitrary left image for the left image is generated based on the parallax information for each pixel position. Pixel delay F
IFO 11 to 13 and arbitrary pixel delay FIFO for right image
The read addresses 21 to 23 are controlled.

The parallax information for each pixel position on the one-field screen is the timing signal generation circuit 51, the parallax interpolation coefficient generation circuit 52, the parallax information storage means 60, and the parallax selection circuit 8.
0, the first to fourth multipliers 81 to 84, and the adder circuit 85.

The horizontal synchronizing signal Hsync and the vertical synchronizing signal Vsync of the input video signal are input to the timing signal generating circuit 51. The clock signal CLK for detecting the horizontal address of each horizontal period is also input to the timing signal generation circuit 51.

The timing signal generating circuit 51, based on the horizontal synchronizing signal Hsync, the vertical synchronizing signal Vsync and the clock signal CLK, outputs a horizontal address signal HAD indicating the absolute horizontal position of the input video signal and an absolute vertical position of the input video signal. And a relative horizontal position signal HPOS indicating a relative horizontal position of the input video signal.
And a relative vertical position signal VPOS representing the relative vertical position of the input video signal is generated and output.

The relative horizontal position and relative vertical position of the input video signal will be described.

As shown in FIG. 29, the parallax calculation areas E1 to E12 in FIG. 2 are set as follows. The entire screen is divided into 20 areas of 4 rows and 5 columns (hereinafter referred to as the first divided areas), as shown by the dotted lines in FIG. Then, there are three quadrilateral regions having four vertices at the center of the first divided region at the upper left end, the center of the first divided region at the upper right end, the center of the first divided region at the lower left end, and the center of the first divided region at the lower right end. Line 4
It is divided into 12 regions (hereinafter referred to as second divided regions) in a row, and each second divided region is set as parallax calculation regions E1 to E12.

The number of pixels in the horizontal direction of the first divided area and the second divided area is represented by m, and the number of pixels in the vertical direction of the first divided area and the second divided area is represented by n. The relative horizontal position of the input video signal is represented by 0 to (m-1) where the left end of each first divided area is 0 and the right end is m.
The relative vertical position of the input video signal is represented by 0 to (n-1), where the upper end of each first divided area is 0 and the lower end is n.

Relative horizontal position signal HPO of input video signal
The S and the relative vertical position VPOS are sent to the parallax interpolation coefficient generating circuit 52. The parallax interpolation coefficient generating circuit 52 calculates the first parallax interpolation coefficient KUL, based on the relative horizontal position signal HPOS, the relative vertical position VPOS, and the following Expression 7.
The second parallax interpolation coefficient KUR, the third parallax interpolation coefficient KDL, and the fourth parallax interpolation coefficient KDR are generated and output.

[0124]

[Equation 7]

FIG. 3 shows the basic idea of the method of generating the parallax information for each pixel position on the 1-field screen.
It will be described using 0. It is assumed that the horizontal / vertical position (hereinafter referred to as the attention position) represented by the horizontal address signal HAD and the vertical address signal VAD is Pxy in FIG. A case where parallax information for the attention position Pxy is obtained will be described.

(1) First, among the parallax information for each of the parallax calculation regions E1 to E12 calculated by the CPU 3, the four vertices of the first divided region including the target position Pxy,
In this example, the parallax information for the parallax calculation areas E1, E2, E5, E6 centered on PE1, PE2, PE5, PE6 is extracted as UL, UR, DL, DR, respectively. That is, of the four vertices of the first divided region including the attention position Pxy, the parallax information of the region E1 centered on the upper left vertex is the first parallax information UL, and the parallax information of the region E2 centered on the upper right vertex is set. Is extracted as the second disparity information UR, the disparity information of the area E5 centered on the lower left apex is extracted as the third disparity information DL, and the disparity information of the area E6 centered on the lower right apex is extracted as the fourth disparity information DR. It

However, as in the case where the first divided area including the attention position is the first divided area at the upper left end, only one of the four vertices of the first divided area including the attention position has parallax. When it corresponds to the center of the detection region, the parallax information of the parallax calculation region is the first to fourth parallax information U.
It is extracted as L, UR, DL, DR.

Further, as in the case where the first divided area including the attention position is the first divided area on the right side of the first divided area at the upper left end, the four vertices of the first divided area including the attention position. In the case where only the lower two vertices of the parallax calculation region correspond to the center of the parallax calculation region, the parallax information UL corresponding to the upper two vertices of the four vertices of the first divided region including the attention position, As the UR, the parallax information of the parallax calculation region centered on the lower vertex is extracted.

In addition, as in the case where the first divided area including the attention position is the first divided area below and adjacent to the first divided area at the upper left end, the four vertices of the first divided area including the attention position If only the two vertices on the right side correspond to the center of the parallax calculation region, the parallax information UL, DL corresponding to the two vertices on the left side among the four vertices of the first divided region including the attention position As, the parallax information of the parallax calculation region centered on the right vertex is extracted.

Further, as in the case where the first divided area including the attention position is the first divided area adjacent to the left of the first divided area at the lower right corner, the four vertices of the first divided area including the attention position. If only the upper two vertices of the four vertices correspond to the center of the parallax calculation region, the parallax information DL corresponding to the two lower vertices of the four vertices of the first divided region including the attention position, As DR, the parallax information of the parallax calculation region centered on the upper vertex is extracted.

Further, as in the case where the first divided area including the attention position is the first divided area above and adjacent to the first divided area at the lower right corner, the four vertices of the first divided area including the attention position are included. If only the two vertices on the left side correspond to the center of the parallax calculation region, the parallax information UR, DR corresponding to the two vertices on the right side among the four vertices of the first divided region including the attention position. As, the parallax information of the parallax calculation region centered on the left vertex is extracted.

(2) Next, the first to fourth parallax interpolation coefficients K
UL, KUR, KDL and KDR are required.

The first parallax interpolation coefficient KUL is the position P of interest.
The distance ΔX from the attention position Pxy to the right side of the first divided area e with respect to the horizontal width m of the first divided area e including xy.
Ratio with R {(m-HPOS) / m} and first divided area e
Of the distance ΔYD from the target position Pxy to the lower side of the first divided area e with respect to the vertical width n of {{n-VPO.
S) / n}. That is, the first
The parallax interpolation coefficient KUL becomes larger as the distance between the upper left apex PE1 of the first divided area e including the target position Pxy and the target position Pxy is smaller.

The second parallax interpolation coefficient KUR is the target position P
The distance ΔX from the position of interest Pxy to the left side of the first divided region e with respect to the horizontal width m of the first divided region e including xy.
The ratio (HPOS / m} of L to the vertical width n of the first divided area e and the distance ΔYD from the position of interest Pxy to the lower side of the first divided area e {(n-VPOS) / n}.
It is calculated by the product of That is, the second parallax interpolation coefficient KUR increases as the distance between the attention position Pxy and the upper right apex PE2 of the first divided region e including the attention position Pxy increases.

The third parallax interpolation coefficient KDL is the target position P
The distance ΔX from the attention position Pxy to the right side of the first divided area e with respect to the horizontal width m of the first divided area e including xy.
Ratio with R {(m-HPOS) / m} and first divided area e
Of the distance ΔYU from the position of interest Pxy to the upper side of the first divided region e with respect to the vertical width n of the (VPOS / n)
It is calculated by the product of That is, the third parallax interpolation coefficient KDL increases as the distance between the lower left apex PE5 of the first divided region e including the target position Pxy and the target position Pxy decreases.

The fourth parallax interpolation coefficient KDR is the position P of interest.
The distance ΔX from the position of interest Pxy to the left side of the first divided region e with respect to the horizontal width m of the first divided region e including xy.
By the product of the ratio with L (HPOS / m) and the ratio (VPOS / n) with respect to the vertical width n of the first divided region e and the distance ΔYU from the target position Pxy to the upper side of the first divided region e. Desired. That is, the fourth parallax interpolation coefficient KD
R increases as the distance between the lower right apex PE6 of the first divided area e including the target position Pxy and the target position Pxy decreases.

(3) First to fourth extracted in (1) above
Of the parallax information UL, UR, DL, and DR of the first to fourth parallax interpolation coefficients KUL calculated in (2) above, respectively.
Each of KUR, KDL, and KDR is multiplied. Then, the obtained four multiplication values are added to generate parallax information for the target position Pxy.

The parallax information storage means 60 stores areas E1 to E1.
The first to twelfth parallax registers 61 to 72 are provided corresponding to the No. 2 and No. 2, respectively. The first to twelfth parallax registers 61 to 72 store parallax information for the respective areas E1 to E12 generated by the CPU 3.

A parallax selection circuit 80 is provided at the subsequent stage of the parallax information storage means 60. In the parallax selection circuit 80,
Parallax information is sent from each of the parallax registers 61 to 72. Further, the horizontal address signal HAD and the vertical address signal VAD are sent from the timing signal generation circuit 51 to the parallax selection circuit 80.

According to the rule shown in FIG. 31A, the parallax selection circuit 80 has an area corresponding to the horizontal address signal HAD and the vertical address signal VAD (the first area including the target position in the example of FIG. 30). The parallax information for the parallax calculation area centered on the upper left apex of the
Select as L and output. Furthermore, the parallax selection circuit 80
According to the rule shown in FIG. 31B, the area corresponding to the horizontal address signal HAD and the vertical address signal VAD (in the example of FIG. 30, the upper right corner of the first area including the target position is centered). The parallax information for the parallax calculation region) is selected and output as the second parallax information UR.

Furthermore, the parallax selection circuit 80 is shown in FIG.
According to the rule shown in (c), a region corresponding to the horizontal address signal HAD and the vertical address signal VAD (in the example of FIG. 30, a parallax calculation region centered on the lower left apex of the first region including the attention position). The parallax information for is selected and output as the third parallax information DL. Further, the parallax selection circuit 80 follows the rule shown in FIG. 31 (d), and the area corresponding to the horizontal address signal HAD and the vertical address signal VAD (in the example of FIG. 30, the first area including the target position). Parallax calculation area centered on the lower right vertex)
The parallax information for is selected and output as the fourth parallax information DR. In FIG. 31, for example, the symbols “to” represented by a to b such as 0 to m are used as symbols indicating a to a but less than b.

The first selected by the parallax selection circuit 80
The parallax information UL, the second parallax information UR, the third parallax information DL, and the fourth parallax information DR are input to the first, second, third, and fourth multipliers 81, 82, 83, 84, respectively.

First, second, third and fourth multiplier 8
The first parallax interpolation coefficient KUL, the second parallax interpolation coefficient KUR, the third parallax interpolation coefficient KDL, and the fourth parallax interpolation coefficient KDR from the parallax interpolation coefficient generation circuit 52 are also input to 1, 82, 83, and 84, respectively. There is.

The first multiplier 81 multiplies the first parallax information UL by the first parallax interpolation coefficient KUL. The second multiplier 82 is
The second parallax information UR is multiplied by the second parallax interpolation coefficient KUR. The third multiplier 83 multiplies the third parallax information DL by the third parallax interpolation coefficient KDL. The fourth multiplier 84 multiplies the fourth parallax information DR by the fourth parallax interpolation coefficient KDR.

The outputs of the multipliers 81, 82, 83 and 84 are added by the adder circuit 85. Thereby, the parallax information PR for the position of interest is obtained.

Each arbitrary pixel delay FIFO 11, 21 has a value of 1
In order to perform horizontal phase control in units smaller than pixels,
Two line memories 11a, 11b, 21a,
21b is provided. Each arbitrary pixel delay FIFO 11, 2
Two line memories 11a, 11b, 21a, 2 in 1
The Y signal and the clock signal CLK are input to 1b, respectively.

The horizontal address signal HAD output from the timing signal generation circuit 51 is also input to the standard address generation circuit 90. Standard address generation circuit 90
Is a standard write address WAD and a standard read address R for the two line memories 11a, 11b, 21a, 21b in each arbitrary pixel delay FIFO 11, 21.
Generate and output AD. Further, the standard address generation circuit 90 uses the sync signal Csyn added to the left video signal and the right video signal obtained by the 2D / 3D video conversion device.
Also output c. The horizontal synchronizing signal represented by the synchronizing signal Csync is the horizontal synchronizing signal Hs of the input video signal.
The signal is delayed by a predetermined number of clocks from ync.

The standard read address RAD is for enabling the horizontal phase of the video signal input to each arbitrary pixel delay FIFO 11, 21 to be advanced or delayed with respect to the reference horizontal phase defined by the standard read address. In addition, the standard write address WAD is delayed by a predetermined number of clocks. The standard write address WAD output from the standard address generation circuit 90 is the two line memories 11 in each arbitrary pixel delay FIFO 11, 21.
A write control signal indicating a write address is input to a, 11b, 21a, and 21b.

The standard read address RAD output from the standard address generation circuit 90 is input to the adder 91 and the subtractor 92, respectively. Adder 91 and subtractor 9
2 also receives the parallax information PR of the position of interest output from the adder circuit 85.

In the adder 91, the standard read address R
The parallax information PR is added to AD. As a result, the left video read address PRL is obtained.

The integer part PRL1 of the left video read address PRL is stored in the first line memory 11a in the left video arbitrary pixel delay FIFO 11 as a read address RADL1.
Enter as. Therefore, the first line memory 11
The Y signal is read from the address corresponding to the address RADL1 of a. The read Y signal is input to the first left video multiplier 101.

The address value obtained by adding 1 to the integer part PRL1 of the left video read address PRL is input to the second line memory 11b in the left video arbitrary pixel delay FIFO 11 as the read address RADL2. Therefore, the Y signal is read from the address corresponding to the address RADL2 of the second line memory 11b. The read Y signal is input to the second left video multiplier 102.

Since the read address RADL1 for the first line memory 11a and the read address RADL2 for the second line memory 11b differ by one, the Y signal read from the first line memory 11a, The horizontal position of the Y signal read from the second line memory 11b is shifted by 1.

The fractional part PRL2 of the left video read address PRL is input to the second left video multiplier 102 as the second left video interpolation coefficient. A value obtained by subtracting the fractional part PRL2 of the left video read address PRL from 1 (1-PR
L2) is input to the first left video multiplier 101 as a first left video interpolation coefficient.

Therefore, the first left video multiplier 101
Then, the Y signal read from the first line memory 11a is multiplied by the first left video interpolation coefficient (1-PRL2). In the second left video multiplier 102, the Y signal read from the second line memory 11b is multiplied by the second left video interpolation coefficient PRL2. Then, each multiplier 101,
The Y signal obtained by 102 is added by the adder 103, and then output as the left video Y signal YL-OUT.

As a result, the standard read address RAD
A left image Y signal whose horizontal phase amount is delayed by an amount corresponding to the parallax information for the position of interest with respect to the reference horizontal phase defined by

In the subtractor 92, the standard read address R
The parallax information PR is subtracted from AD. As a result, the right video read address PRR is obtained.

The integer part PRR1 of the right video read address PRR is stored in the first line memory 21a in the right video arbitrary pixel delay FIFO 21 as the read address RADR1.
Enter as. Therefore, the first line memory 21
The Y signal is read from the address corresponding to the address RADR1 of a. The read Y signal is input to the first right video multiplier 111.

The address value obtained by adding 1 to the integer part PRR1 of the right video read address PRR is input to the second line memory 21b in the right video arbitrary pixel delay FIFO 21 as the read address RADR2. Therefore, the Y signal is read from the address corresponding to the address RADR2 of the second line memory 21b. The read Y signal is input to the second right video multiplier 112.

Since the read address RADR1 for the first line memory 21a and the read address RADR2 for the second line memory 21b differ by one, the Y signal read from the first line memory 21a, The horizontal position of the Y signal read from the second line memory 21b is shifted by 1.

The fractional part PRR2 of the right video read address PRR is input to the second right video multiplier 112 as the second right video interpolation coefficient. A value obtained by subtracting the fractional part PRR2 of the right video read address PRR from 1 (1-PR
R2) is input to the first right video multiplier 111 as a first right video interpolation coefficient.

Therefore, the first right video multiplier 111
Then, the Y signal read from the first line memory 21a is multiplied by the first right video interpolation coefficient (1-PRR2). In the second right video multiplier 112, the Y signal read from the second line memory 21b is multiplied by the second right video interpolation coefficient PRR2. Then, each multiplier 111,
The Y signal obtained by 112 is added by the adder 113, and then output as the right image Y signal YR-OUT.

As a result, the standard read address RAD
A right image Y signal in which the horizontal phase amount is advanced by an amount corresponding to the parallax information with respect to the position of interest with respect to the reference horizontal phase defined by

In FIG. 32, the parallax information for the position of interest is 0.
In the case of, the signal of each part is shown.

When the parallax information is 0, the left video read address PRL output from the adder 91 and the right video read address PRR output from the subtractor 92.
Are both addresses that are equal to the standard read address RAD and consist of only an integer part without a decimal part.

Therefore, the left image arbitrary pixel delay FIF
Read address RADL1 for the first line memory 11a in O11 and right image arbitrary pixel delay FIFO
The read address RADR1 for the first line memory 21a in 21 becomes an address equal to the standard read address RAD.

Further, the left image arbitrary pixel delay FIFO 11
The read address RADL2 for the second line memory 11b in the inside and the read address RADR2 for the second line memory 21b in the right video arbitrary pixel delay FIFO 21 are one larger than the standard read address RAD.

The first left video interpolation coefficient (1-PRL)
2) and the first right video interpolation coefficient (1-PRR2) is 1
Therefore, the second left video interpolation coefficient PRL2 and the second right video interpolation coefficient PRR2 are zero.

As a result, the left image arbitrary pixel delay FIFO
Standard address RA of the first line memory 11a in 11
The Y signal read from the address corresponding to D is output from the adder 103 as the left image Y signal YL-OUT and corresponds to the standard address RAD of the first line memory 21a in the right image arbitrary pixel delay FIFO 21. The Y signal read from the address to be output is output from the adder 113 as the Y signal YR-OUT for the right image. That is, two Y signals having the same amount of phase shift in the horizontal direction, that is, two Y signals having no parallax are output as the left image Y signal and the right image Y signal.

FIG. 33 shows each address value when the standard write address WAD for a certain target position is 20, the standard read address RAD for the target position is 10, and the parallax information for the target position is 1.2. Shows a specific example of. FIG. 34 shows the signals of the respective parts at that time.

In this case, the left video read address PRL output from the adder 91 is 11.2.
The integer part PRL1 becomes 11, and the decimal part PRL2
Is 0.2.

Therefore, the left image arbitrary pixel delay FIF
The read address RADL1 for the first line memory 11a in O11 is 11, and the read address RADL2 for the second line memory 11b is 12. Also, the first left image interpolation coefficient KL1 {= (1-PR
L2)} becomes 0.8, and the second left image interpolation coefficient KL2
(= PRL2) is 0.2.

Therefore, the left-picture arbitrary pixel delay FIF
The Y signal (Y ₁₁ ) is read from the address 11 of the first line memory 11a in the O11, and the signal (0 that is 0.8 times the read Y signal (Y ₁₁ ) is read from the first multiplier 101. ．
8 * Y ₁₁ ) is output.

On the other hand, the left image arbitrary pixel delay FIFO 11
Second Y signal from the address 12 of the line memory 11b of the inner (Y ₁₂₎ is read, 0.2 times of the signals from the second multiplier 102 read Y signal (Y ₁₂₎ (0. 2 * Y
₁₂ ) is output. Then, from the adder 103, 0.
Left image Y signal YL corresponding to 8 * Y ₁₁ + 0.2 * Y ₁₂
-OUT is output. That is, read address 1
The Y signal corresponding to 1.2 is the Y signal YL-OU for the left image.
It is output as T.

The right video read address PRR output from the subtractor 92 is 8.8, and its integer part PRR
1 becomes 8 and its fractional part PRR2 becomes 0.8.

Therefore, the right pixel arbitrary pixel delay FIF
The read address RADR1 for the first line memory 21a in O21 becomes 8 and the second line memory 2
The read address RADR2 for 1b is 9.
In addition, the first right image interpolation coefficient KR1 {= (1-PRR
2)} becomes 0.2, and the second right video interpolation coefficient KR2
(= PRR2) is 0.8.

Therefore, the right pixel arbitrary pixel delay FIF
The Y signal (Y ₈ ) is read from the address 8 of the first line memory 21a in the O21, and the signal (0) that is 0.2 times the read Y signal (Y ₈ ) is read from the first multiplier 111. .2
* Y ₈ ) is output.

On the other hand, the right image arbitrary pixel delay FIFO 21
Y signal from the address 9 of the second line memory 21b of the inner (Y ₉₎ is read, 0.8 times the signal from the second multiplier 112 read Y signal (Y ₉₎ (0. 8 *
Y ₉ ) is output. Then, from the adder 113,
0.2 * Y ₈ + 0.8 * Y signal for the right image corresponding to Y ₉ YR-OUT is output. That is, the Y signal corresponding to the read address 8.8 is the Y signal YR-OU for the right image.
It is output as T.

As a result, a left image and a right image having a parallax of 11.2-8.8 = 2.4, that is, a parallax twice as large as the parallax information 1.2 are obtained.

[0180]

2D / for still images described above
In the 3D image conversion method, the amount of parallax between regions in one screen may be large, and the amount of parallax between fields may be large. There is a problem of being difficult.

An object of the present invention is to provide a device for converting a two-dimensional image into a three-dimensional image, which can obtain a three-dimensional image with less image distortion.

[0182]

Means for Solving the Problems The first two according to the present invention
An apparatus for converting a three-dimensional image into a three-dimensional image extracts image feature amounts related to the perspective of the image for each of a plurality of parallax calculation regions set in a one-field screen based on the two-dimensional input image, and extracts the image feature amount. Based on the image feature amount, depth information generating means for generating depth information for each parallax calculation area, depth range for a predetermined target phase amount and depth information for each predetermined unit area generated by the depth information generating means Based on the
Target phase amount calculating means for calculating a target amount of phase Zui predetermined unit regions, for each of the predetermined unit regions, the actual phase of the previous
So as to approach gradually the target phase quantity calculated by said target phase amount calculating means based, the actual phase amount calculating means for determining an actual amount of phase current, and a signal of each predetermined unit area of the two-dimensional input image, the It is characterized by further comprising phase control means for respectively generating a first video signal and a second video signal having a horizontal phase difference corresponding to a current actual phase amount corresponding to a predetermined unit area.

A device for converting a second two-dimensional image into a three-dimensional image according to the present invention is based on a two-dimensional input image.
Depth information that extracts the image feature amount related to the perspective of the video for each of the plurality of parallax calculation regions set in the field screen, and generates depth information for each parallax calculation region based on the extracted image feature amount When the maximum value of the difference in the depth information between the adjacent areas is larger than the reference value, the dynamic range of the target phase amount is corrected so that the preset dynamic range of the target phase amount becomes smaller. Based on the correction unit, the dynamic range of the target phase amount obtained by the correction unit, and the depth information for each predetermined unit region based on each parallax calculation region generated by the depth information generation unit, for each parallax calculation region target phase amount calculating means for calculating a target phase amount, the
For each predetermined unit area, the target based on the previous actual phase amount
Gradually approaches the target phase amount calculated by the phase amount calculation means
As described above, from the actual phase amount calculating means for obtaining the actual phase amount this time, and the signal in each predetermined unit area of the two-dimensional input image,
It is characterized by further comprising phase control means for respectively generating a first video signal and a second video signal having a horizontal phase difference corresponding to the current actual phase amount corresponding to the predetermined unit area.

A third apparatus for converting a two-dimensional image into a three-dimensional image according to the present invention generates a reference main image and a sub-image delayed in time with respect to the main image, based on the two-dimensional input image. A video conversion unit that outputs one of the main video and the sub-video as a left-eye video and the other as a right-eye video, and a video for determining whether the two-dimensional input video is a video or a still video. / Still image ratio calculating means for calculating the still image ratio based on the determination results of the still image determining means and the moving image / still image determining means, and 1 based on the two-dimensional input image
Depth information that extracts the image feature amount related to the perspective of the video for each of the plurality of parallax calculation regions set in the field screen, and generates depth information for each parallax calculation region based on the extracted image feature amount generating means, by multiplying the still image ratio on the dynamic range of a preset target phase amount, correcting means for correcting the dynamic range of the target phase amount, the dynamic range of the target phase amount obtained by the correction means, the depth Target phase amount calculation means for calculating a target phase amount for each parallax calculation area based on depth information for each predetermined unit area based on each parallax calculation area generated by the information generation means, the predetermined
For each unit area, the target phase based on the previous actual phase amount
Gradually approach the target phase amount calculated by the amount calculation means
In addition, the actual phase amount calculating means for obtaining the actual phase amount at this time, and the signal in each predetermined unit area of the right-eye image and the left-eye image output from the image converting means are obtained by the actual phase amount calculating means In addition, a phase control unit that performs horizontal phase control according to the actual phase amount corresponding to the predetermined unit area is provided.

A fourth apparatus for converting a two-dimensional image into a three-dimensional image according to the present invention generates a reference main image and a sub-image delayed with respect to the main image based on the two-dimensional input image. A video conversion unit that outputs one of the main video and the sub-video as a left-eye video and the other as a right-eye video, and a video for determining whether the two-dimensional input video is a video or a still video. / Still image ratio calculating means for calculating the still image ratio based on the determination results of the still image determining means and the moving image / still image determining means, and 1 based on the two-dimensional input image
Depth information that extracts the image feature amount related to the perspective of the video for each of the plurality of parallax calculation regions set in the field screen, and generates depth information for each parallax calculation region based on the extracted image feature amount When the maximum value of the difference in the depth information between the adjacent areas is larger than the reference value, the dynamic range of the target phase amount is corrected so that the preset dynamic range of the target phase amount becomes smaller. The first correction unit and the second correction unit and the second correction unit correct the dynamic range of the target phase amount by multiplying the dynamic range of the target phase amount obtained by the first correction unit by the still image ratio. and the dynamic range of the target phase amount, <br/> place based on the parallax calculation regions generated by the depth information generating unit constant unit per area Based on the depth information, the target phase amount calculating means for calculating a target phase of each parallax calculation region,
For each of the predetermined unit areas, based on the previous actual phase amount,
Gradually approaches the target phase amount calculated by the target phase amount calculation means.
As described below, the actual phase amount calculating means for obtaining the actual phase amount this time, and the signal in each predetermined unit area of the right eye image and the left eye image output from the image converting means Phase control means for performing horizontal phase control according to the obtained actual phase amount corresponding to the predetermined unit area is provided.

As the still picture ratio calculating means, for example, when the moving picture / still picture judging means judges that the two-dimensional input video is a moving picture, 0 is set as the still picture ratio, and the two-dimensional input picture is a still picture. If it is determined that the ratio is 100, a unit having means for setting 100 as the still image ratio and a means for smoothing the set still image ratio between fields are used.

As the actual phase amount calculating means, for example, the amount of motion of the image in each parallax calculation region is calculated based on the two-dimensional input image, and the actual phase of the region is larger as the amount of motion of the image is larger. The actual phase amount of this time for each parallax calculation region is calculated so that the amount approaches the target phase amount of that region faster, and one field is calculated based on the obtained actual phase amount of each parallax calculation region. The one that finds the actual phase amount this time for each predetermined unit area in the screen is used.

[0188]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 35 shows the structure of a 2D / 3D video conversion device.

The 2D / 3D video converter is an integrated circuit (L
SI) 510, a plurality of field memories 521 to 524 connected to the integrated circuit 510, and a CPU 530.

The integrated circuit 510 includes a motion vector detection circuit 511, various in-screen data calculation circuits 512, a memory control section 513, a right-eye image selection circuit 514, a left-eye image selection circuit 515, a right-eye parallax generation circuit 516, a left-eye image generation circuit. A parallax generation circuit 517, a parallax control circuit 518, a CPU interface 519 and the like are provided.

The motion vector detection circuit 511 detects a motion vector for each field based on the representative point matching method. The motion vector detected by the motion vector detection circuit 511 is sent to the CPU 530 via the CPU interface 519.

The in-screen data calculation circuit 512 is shown in FIG.
The high-frequency component integrating circuit 8, the brightness contrast calculating circuit 9, the RY component integrating circuit 31, and the BY component integrating circuit 32 are included, and a plurality of parallaxes set in one field screen for each field. The image feature amount relating to the perspective of the image is calculated for each of the calculation regions.

The high-frequency component integrating circuit 8 calculates the integrated value of the high-frequency components for each of the plurality of parallax calculation areas E1 to E12 preset in the one-field screen, as shown in FIG. 2, for each field. . The brightness contrast calculation circuit 9 is configured to detect each parallax calculation area E1 for each field.
The brightness contrast for each of E12 to E12 is calculated. The RY component integrating circuit 31 calculates the integrated value of the RY components for each of the parallax calculation regions E1 to E12 for each field. The BY component integration circuit 32 calculates the integrated value of the BY component for each of the parallax calculation regions E1 to E12 for each field. The image feature amount relating to the perspective of the image calculated by the various in-screen data calculation circuit 512 is sent to the CPU 530 via the CPU interface 519.

FIG. 36 shows the processing executed by the CPU 530.

The CPU 530 is the motion vector detecting circuit 5
Based on the horizontal component of the motion vector detected by 11, the delay amount used in the moving image 2D / 3D image conversion method is calculated for each field (step 11). At this time, the CPU 530 determines, based on the direction (right or left) of the horizontal component of the motion vector, a main image that serves as a reference in the moving image 2D / 3D image conversion method and a sub image that is temporally delayed with respect to the main image. Which of these is the image for the left eye,
Which one is to be the image for the right eye is determined, and the result of this determination is reflected as the sign (positive or negative) of the delay amount.

As a method of determining the delay amount, for example, the algorithm shown in FIG. 3 of Japanese Patent Laid-Open No. 8-149513 can be used. It should be noted that, in order to reduce the variation in the delay amount between fields, the method disclosed in Japanese Patent Laid-Open No. 8-149
The smoothing process may be performed on the delay amount obtained by using the algorithm shown in FIG. The delay amount calculated by the CPU 530 is sent to the memory control unit 513.

Further, the CPU 530 calculates depth information for each field for each parallax calculation region in each field based on the image feature amount relating to the perspective of the image calculated by the various in-screen data calculation circuit 512. (Step 12). As a method of calculating the depth information, the algorithm of steps 1 to 8 in FIG. 12 can be used.

Further, based on the horizontal component of the motion vector detected by the motion vector detection circuit 511, the CPU 530, for each field, shows that the current field is a moving image (moving image) or the current field is moving. Determine if there is no video (still image) (Step 1
3). If it is determined to be a moving image, the still image ratio is set to 0 (step 14). When it is determined that the image is a still image, the still image ratio is set to 100 (step 15). Then, in order to reduce the fluctuation of the still image ratio between fields, the still image ratio smoothing process is performed (step 16). As a smoothing process of this still image ratio
Japanese Patent Application No.
It can be realized by the technique described in 9-253878.

The still image ratio smoothing process is actually executed by software, but for convenience of description, it will be described using the block diagram of FIG.

That is, when the still image ratio is set in step 14 or 15, the multiplier 6 is added to the set still image ratio.
The coefficient K is multiplied by 01. As this coefficient,
For example, it is set to a value such as 1/4, 1/8, 1/16. The multiplication result is sent to the adder 602.

The adder 602 is supplied to the adder 602 by the delay memory 603 and the multiplier 604 that multiplies the coefficient (1-K), and the addition result of the adder 602 in the previous field is (1- The value multiplied by K) is sent. Therefore, in the adder 602, the still image ratio set this time is multiplied by the coefficient K and the delay memory 60
3 and a value obtained by adding (1-K) to the addition result of the adder 602 in the immediately preceding field, which is supplied to the adder 602 by the multiplier 604 that multiplies the coefficient (1-K). It

The result of this addition is stored in the delay memory 603. The addition result stored in the delay memory 603 is output as the still image ratio after the smoothing process in the next field.

According to such a still image ratio smoothing process, even if the input image is switched between a moving image and a still image,
The still image ratio after the smoothing process gradually changes between fields.

Further, the CPU 530 performs a phase amount calculation process for calculating a phase amount for each parallax calculation region based on the depth information for each parallax calculation region in the one-field screen obtained by the depth information calculation process. (Step 1
7).

FIG. 37 shows the phase amount calculation processing procedure.

Here, the depth information is 0 (back) to 10
It is assumed that it is generated within the range of 0 (front). Figure 1
1 obtained by the algorithm of steps 1 to 8 of 2
The following method can be considered as a method of converting the depth information for each parallax calculation region in the field screen into the target phase amount.

As for the phase amount, the limit value of the target phase amount when the image is projected in front of the tube surface is front [pixels], and the limit value of the target phase amount when the image is retracted behind the tube surface is re.
If ar [pixels], then the dynamic range of the phase is fron
t-rear [pixel]. The value of front [pixel] is, for example, 10 [pixel], and the value of rear [pixel] is, for example, −10 [pixel]. front [pixel]
The values set by the user by the volume are used as the values of and rear [pixels].

[0209] The depth information is depth, and the target phase amount is tp
Assuming hase, the target phase amount tphase is expressed by the following formula 8.

[0210]

[Equation 8]

From the above equation 8, it can be seen that the target phase amount increases as the dynamic range (front-rear) increases. Obviously, the larger the target phase amount, the larger the phase difference between the adjacent parallax calculation regions. As the phase difference between adjacent regions increases, the image distortion increases between those regions.

Therefore, in the phase amount calculation processing, distortion correction processing for correcting such distortion is performed (step 21).

In the distortion correction process, if the maximum value of the phase difference between adjacent regions is equal to or larger than the predetermined value PHASETH, the dynamic range of the target phase amount is changed in order to reduce the phase difference.

[0214] Actually, when the maximum value dvmax of the depth information differences between adjacent areas is equal to or greater than the predetermined value DEPTHTH, the dynamic range dvrange (= front-rear) of the target phase amount set by the user. Can be changed.

The predetermined value DEPTHTH is calculated by the following expression 9.

[0216]

[Equation 9]

If the maximum value dvmax of the differences in depth information between adjacent areas is greater than or equal to the predetermined value DEPTHTH, the dynamic range dvra of the target phase amount is calculated by the following expression 10.
nge can be changed.

[0218]

[Equation 10]

In Expression 10, dvrange 'is the dynamic range after being corrected by the distortion correction. Also, front 'and rear' are the target phase amount limit values when projecting the image in front of the tube surface and the target phase when retracting the image behind the tube surface after being corrected by distortion correction, respectively. This is the limit value of the quantity and is defined by the following formula 11.

[0220]

[Equation 11]

In the above example, the predetermined value DEPTHTH is a fixed value, but as shown in FIG. 39, it may be a variable value that decreases as the dynamic range dvrange set by the user increases. This will set the dynamic range dvra set by the user.
It is possible to provide an optimal stereoscopic effect for nge. Figure 3
In Fig. 9, DEPTHTH is saturated when the dynamic range dvrange set by the user is small because the dynamic range dvra set by the user is large.
This is because when nge is small, image distortion is difficult to be recognized even if the difference in depth information between adjacent regions is large.

When the distortion correction process is performed, a correction process based on the still image ratio obtained in step 16 (hereinafter referred to as a ratio correction process) is then performed (step 22).

If the still image ratio obtained in step 16 above is rate, the dv obtained by the distortion correction process is calculated.
Range ', front', and rear 'are converted into dvrange'',front'', and rear''by the following formula 12.

[0224]

[Equation 12]

It should be noted that instead of DEPTHTH in the equations 10 and 11 used for the distortion correction in the step 21, the DEPT considering the still image ratio rate represented by the following equation 13 is used.
HTH '' may be used.

[0226]

[Equation 13]

When the distortion correction process and the ratio correction process are completed, the target phase amount calculation process is performed (step 2).
3). That is, using the dynamic range dvrange ″ (= front ″ −rear ″) obtained after the ratio correction processing, each depth information depth is converted into the target phase amount tphase by the following Expression 14.

[0228]

[Equation 14]

[0229] The target phase amount tphase thus obtained
Is directly affected by the depth information generation process, the distortion correction process, and the ratio correction process, so that its value is very likely to change. In order to alleviate this fluctuation, phase fluctuation suppression processing is performed (step 24).

In the phase fluctuation suppression processing, the target phase amount tphase for each parallax calculation region and the actual phase amount (actual phase amount) phase for each parallax calculation region are compared, and if there is a difference, In addition, the actual phase amount is added / subtracted by a predetermined phase amount variation amount VT (> 0). That is, the predetermined value VT (>
The actual phase amount is brought closer to the target phase amount by 0) minutes.

The contents of the phase fluctuation suppression processing are as shown in the following mathematical formula 15.
It is represented by the conditional expression.

[0232]

[Equation 15]

The predetermined value VT is preferably about 1/32 [pixel], for example. When the predetermined value VT = 1/32 [pixel], in order to change the phase amount by 1 [pixel],
It will take 32 fields.

In the above example, the predetermined value VT used for the phase fluctuation suppression processing is a fixed value, but as shown in FIG. 40, VT is maximized when the still image ratio rate is about 50%. 41 may be variable, or as shown in FIG. 41, VT may be variable such that VT increases as the absolute value of the difference between the target phase amount tphase and the actual phase amount phase increases.

40 and 41, BASE is the lower limit value of VT and MAX is the upper limit value of VT. The upper limit of VT is specified because if the VT is made too large, the observer will perceive that the image is distorted according to the fluctuation of the actual phase amount. When VT is set to 1 [pixel] or more,
An observer often recognizes that the image is distorted.

Further, VT may be changed for each parallax calculation region in accordance with the movement of the image in each parallax calculation region. That is, the amount of movement of the image is detected for each parallax calculation region, and as shown in FIG. 42, VT is increased in the region where the amount of movement of the image is large. The reason why the VT is made larger in the region where the amount of motion of the image is larger is that it is difficult to recognize the distorting process of the image in the moving image as compared with the image without moving.

Further, as shown in FIG. 42, according to the method of increasing the VT in the region where the amount of movement of the image is large, it is possible to set the actual phase amount to 0 earlier in the moving image region where the deterioration of the image quality is likely to occur. Become. This is due to the phase control
This means that resolution deterioration that occurs when the image is distorted is eliminated more quickly in the moving image area. Generally, the higher the resolution, the greater the three-dimensional effect, so that the three-dimensional effect in the moving image area can be improved.

In normal times, the actual phase amount phase for each parallax calculation area obtained by the phase fluctuation suppression processing is sent to the parallax control circuit 518.

However, when there is a scene change, the depth information, the target phase amount tphase, the actual phase amount phase and VT are controlled as follows by the scene change detection and control processing (step 25).

[0240] That is, when a scene change is detected,
Depth information, target phase amount tphase, and actual phase amount phase
Is forcibly reset to zero. That is, depth information,
The target phase amount tphase and the actual phase amount phase are set to zero. As a method for detecting a scene change, for example, the method disclosed in Japanese Patent Laid-Open No. 8-149514 can be used.

In a system in which the time required from the scene change to the determination of the target phase amount is less than several fields, the target phase amount initially determined immediately after the scene change is set as the actual phase amount.

In a system in which the time required from the scene change to the determination of the target phase amount is several fields or more, the actual phase amount gradually approaches the target phase amount according to the predetermined value VT. However, when the value of VT used for the phase fluctuation suppression process is small, it takes time until a stereoscopic effect is obtained after the scene change.
The VT used for the phase fluctuation suppression processing of is set to a value larger than the VT under normal conditions.

The memory control unit 513 shown in FIG.
Based on the delay amount sent from U530, the field memory to write the input video and the field memory to read the video signal are determined. The memory control unit 513 also outputs a selection control signal to the right-eye video selection circuit 514 and the left-eye video selection circuit 515 based on the sign of the delay amount.

Here, for convenience of explanation, it is assumed that the input video itself is the main video and the video read from the field memory is the sub-video delayed with respect to the main video. The main image is sent to the first input terminal of the right-eye image selection circuit 514 and the first input terminal of the left-eye image selection circuit 515. The sub-picture is the second picture of the right-eye picture selection circuit 514.
It is sent to the input terminal and the second input terminal of the left-eye image selection circuit 515.

Each of the selection circuits 514 and 515 selects and outputs one of the video signals input to the first and second input terminals based on the selection control signal from the memory control unit 513. As a result, both selection circuits 514, 51
One of the 5 outputs the main video and the other outputs the sub video.

The parallax control circuit 518 shown in FIG.
Actual phase amount ph for each parallax calculation region obtained from U530
The actual phase amount for each pixel is generated based on ase. Figure 3
The parallax control circuit 518 shown in FIG.
28, and more specifically, a circuit 7
This is a circuit corresponding to the part indicated by 01. However, in FIG.
1 is different from the parallax control circuit 4 of FIG. 1 in that the actual phase amount phase for each parallax calculation region obtained by the above-described phase amount calculation processing (see FIG. 38) is input to the parallax control circuit 518 of FIG. There is. Except for this point, the operation of the parallax control circuit 518 shown in FIG. 35 is the same as the operation of the circuit corresponding to the portion indicated by reference numeral 701 in FIG. 28, and therefore description thereof will be omitted.

Also, the right-eye parallax generation circuit 5 shown in FIG.
16 and the parallax generation circuit 517 for the left eye are 2D / for moving images.
For each pixel of the right-eye video signal and the left-eye video signal obtained by the 3D video conversion method, the parallax control circuit 51 is provided.
The horizontal phase control is performed according to the actual phase amount corresponding to the pixel obtained in step 8.

The right-eye parallax generation circuit 516 shown in FIG.
28 corresponds to the circuit denoted by reference numeral 702, and the left-eye parallax generation circuit 517 shown in FIG. 35 is denoted by reference numeral 703 in FIG.
Corresponds to the circuit indicated by. However, the video signal input to the parallax generation circuit 516 for the right eye shown in FIG.
In FIG. 28, reference numeral 702 denotes a right-eye video signal and a left-eye video signal obtained by the 3D video conversion method.
It is different from the video signal input to the circuit indicated by 703. Except for that point, the operation of the right-eye parallax generation circuit 516 shown in FIG. 35 is the same as the operation of the circuit shown by reference numeral 702 in FIG. 28, and the operation of the left-eye parallax generation circuit 517 shown in FIG. Since the operation is the same as that of the circuit denoted by reference numeral 703 in FIG. 28, detailed description thereof will be omitted.

In the above embodiment, 2D / for moving images is always
The right-eye image and the left-eye image are generated by the 3D image conversion method. Then, with respect to the right-eye video and left-eye video generated by the moving-picture 2D / 3D video conversion method, the actual pixel-by-pixel realization generated by the same idea as the still-picture 2D / 3D video conversion method. Phase control is performed based on the phase amount.

In this embodiment, the distortion correction processing (see FIG.
In step 21 of 7), the size of the target phase amount is determined such that image distortion does not occur between the parallax calculation regions in one screen. In addition, the ratio correction process (see FIG.
The target phase amount is determined to be small when the still image ratio is small by the step 22). Furthermore, the actual phase amount is determined by the phase fluctuation suppression process (see step 24 in FIG. 37) so that image distortion does not occur between fields. Therefore, the finally obtained three-dimensional image becomes easy to see. Note that the distortion correction process (see FIG.
Step 21 of 7) may be omitted.

In the above embodiment, the phase control based on the actual phase amount is performed on the right-eye image and the left-eye image generated by the moving image 2D / 3D image conversion method. 2D / 3 by 3D video conversion method
Instead of performing the D conversion processing, only the phase control based on the actual phase amount may be performed on the two-dimensional input signal. In this case, the ratio correction process (see step 22 in FIG. 37)
Is omitted. In this case, the distortion correction process (see FIG.
Step 21 of 7) may be omitted.

[0252]

According to the present invention, the image distortion is small.
A three-dimensional image is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a 2D / 3D video conversion device.

FIG. 2 is a schematic diagram showing a parallax calculation region.

FIG. 3 is a block diagram showing a configuration of an RY component integrating circuit.

FIG. 4 is a block diagram showing a configuration of a high frequency component integrating circuit.

5 is a circuit diagram showing a specific example of a high pass filter 232 of FIG.

6 is a graph showing the input / output characteristics of the slice processing circuit 234 of FIG.

FIG. 7 is a block diagram showing another example of a high frequency component integrating circuit.

8 is a circuit diagram showing a specific example of the peak detection circuit 239 of FIG.

FIG. 9 is a time chart showing signals of respective parts of the peak detection circuit 239.

FIG. 10 is a block diagram showing a configuration of a brightness contrast calculation circuit.

11 is a circuit diagram showing a configuration of the brightness contrast detection circuit of FIG.

[Fig. 12] Fig. 12 is a flowchart illustrating a procedure for generating disparity information by a CPU.

FIG. 13 is a schematic diagram showing a parallax calculation area that is actually set.

FIG. 14 is a histogram showing the number of parallax calculation areas for each normalized value of a high frequency component integrated value.

15 is a schematic diagram showing a grouping result obtained based on the histogram of FIG.

FIG. 16 shows the normalized value of the R-Y component integrated value on the vertical axis,
6 is a graph showing the distribution of parallax calculation regions with the normalized value of the BY component integrated value as the horizontal axis.

17 is a schematic diagram showing a grouping result obtained based on the graph of FIG.

FIG. 18 is a schematic diagram showing a grouping result obtained based on the graph of FIG.

FIG. 19 is a schematic diagram showing a grouping result corrected by the space separation processing.

FIG. 20 is a schematic diagram for explaining singular point processing.

FIG. 21 is a schematic diagram showing a grouping result corrected by singularity processing.

[Fig. 22] Fig. 22 is a schematic diagram illustrating a background weight component preset for each parallax calculation region.

[Fig. 23] Fig. 23 is a schematic diagram illustrating an example of depth information of each parallax calculation region before depth correction.

[Fig. 24] Fig. 24 is a schematic diagram illustrating depth information of each parallax calculation region after depth correction.

FIG. 25 is a graph showing a relationship between depth information with respect to a screen height position before depth correction and a relationship with depth information with respect to a screen height position after depth correction.

FIG. 26 is a schematic diagram for explaining depth information correction processing for the inside of a group.

FIG. 27 is a graph showing the relationship between depth information and disparity information.

FIG. 28 is a block diagram mainly showing configurations of a parallax control circuit and an arbitrary pixel delay FIFO.

FIG. 29 is a schematic diagram showing a relative horizontal position, a relative vertical position, and the like.

FIG. 30 is an explanatory diagram illustrating a method of generating disparity information for a pixel of interest.

[Fig. 31] Fig. 31 is a diagram illustrating a selection rule by a parallax selection circuit.

[Fig. 32] Fig. 32 is a time chart illustrating signals of respective units when parallax information is 0.

FIG. 33 is a block diagram in which each address value when the disparity information is 1.2 is added to the disparity control circuit.

[Fig. 34] Fig. 34 is a time chart illustrating signals of respective units when the disparity information is 1.2.

FIG. 35 is a block diagram showing a configuration of a 2D / 3D video conversion device which is an embodiment of the present invention.

FIG. 36 is a flowchart showing processing executed by CPU 530.

FIG. 37 is a flowchart showing a phase amount calculation processing procedure.

[Fig. 38] Fig. 38 is a block diagram for describing the smoothing processing of the still image ratio.

[FIG. 39] DEPTHTH used for distortion correction processing according to the dynamic range dvrange set by the user
Dynamic range dvrange and DE when changing
It is a graph which shows the relationship with PTHTH.

FIG. 40 is a still image ratio rate and VT when VT used for phase fluctuation suppression processing is changed according to the still image ratio rate.
It is a graph which shows the relationship with.

FIG. 41 is an absolute difference between the target phase amount tphase and the actual phase amount phase when VT used for phase fluctuation suppression processing is changed according to the absolute value of the difference between the target phase amount tphase and the actual phase amount phase. It is a graph which shows the relationship between a value and VT.

[Fig. 42] Fig. 42 is a graph illustrating the relationship between the motion amount of the parallax calculation region and VT when the VT used for the phase variation suppression process is changed for each parallax calculation region according to the motion amount of each parallax calculation region. .

[Explanation of symbols]

511 motion vector detection circuit 512 Various screen data calculation circuits 513 memory control unit 514, 515 selection circuit 516 and 517 parallax generation circuit 518 Parallax control circuit 519 CPU interface 521 to 524 field memory 530 CPU

Claims (6)

(57) [Claims] 1. An image feature amount relating to perspective of an image is extracted for each of a plurality of parallax calculation regions set in a one-field screen based on a two-dimensional input image, and based on the extracted image feature amount. A depth information generating unit that generates depth information for each parallax calculation region, each parallax based on a dynamic range of a preset target phase amount and depth information for each predetermined unit region generated by the depth information generating unit Place based on calculation area
Target phase amount calculating means for calculating a target phase of the constant unit area, for each of the predetermined unit area, based on said actual phase of the previous
Gradually as closer to the target phase amount calculated by the target phase amount calculating means, the actual phase amount calculating means for determining an actual amount of phase current, and a signal of each predetermined unit area of the two-dimensional input image, the predetermined unit A device for converting a two-dimensional image into a three-dimensional image, comprising phase control means for respectively generating a first image signal and a second image signal having a horizontal phase difference corresponding to an actual phase amount of this time corresponding to a region. . 2. An image feature amount relating to the perspective of the image is extracted for each of a plurality of parallax calculation regions set in one field screen based on the two-dimensional input image, and based on the extracted image feature amount. A depth information generation unit that generates depth information for each parallax calculation region, and a dynamic range of a preset target phase amount when a maximum value of differences in depth information between adjacent regions is larger than a reference value. each parallax produced by the correction means for correcting the dynamic range of the target phase amount to be smaller, the dynamic range of the target phase amount obtained by the correction means, the depth information generating unit
Based on the depth information for each predetermined unit area based on the calculation area, a target phase amount calculation means for calculating the target phase amount for each parallax calculation area, for each predetermined unit area, based on the previous actual phase amount
Gradually approaches the target phase amount calculated by the target phase amount calculation means.
As described above, the actual phase amount calculating means for calculating the actual phase amount of this time, and the horizontal phase difference corresponding to the actual phase amount of this time corresponding to the predetermined unit area from the signal in each predetermined unit area of the two-dimensional input image. An apparatus for converting a two-dimensional image into a three-dimensional image, comprising: phase control means for respectively generating a first image signal and a second image signal having the above. 3. A main image serving as a reference and a sub-image delayed in time with respect to the main image are generated based on a two-dimensional input image, and one of the main image and the sub-image is output as a left-eye image. Then, the image conversion means for outputting the other as the image for the right eye, the moving picture / still picture judging means for judging whether the two-dimensional input picture is a moving picture or a still picture, based on the judgment result of the moving picture / still picture judging means. A still image ratio calculating means for calculating a still image ratio, and based on the two-dimensional input image, extracts image feature amounts relating to the perspective of the image for each of the plurality of parallax calculation regions set in the one-field screen. , Depth information generating means for generating depth information for each parallax calculation region based on the extracted image feature amount, by multiplying a still image ratio by a dynamic range of a preset target phase amount, Correcting means for correcting the dynamic range of the phase amount, and the dynamic range of the target phase amount obtained by the correction means, the parallax generated by the depth information generating unit
Based on the depth information for each predetermined unit area based on the calculation area, a target phase amount calculation means for calculating the target phase amount for each parallax calculation area, for each predetermined unit area, based on the previous actual phase amount
Gradually approaches the target phase amount calculated by the target phase amount calculation means.
As described below, the actual phase amount calculating means for obtaining the actual phase amount this time, and the signal in each predetermined unit area of the right eye image and the left eye image output from the image converting means An apparatus for converting a two-dimensional image into a three-dimensional image, comprising phase control means for performing horizontal phase control according to the obtained actual phase amount corresponding to the predetermined unit area. 4. A main image serving as a reference and a sub-image delayed in time from the main image are generated based on a two-dimensional input image, and one of the main image and the sub-image is output as a left-eye image. Then, the image conversion means for outputting the other as the image for the right eye, the moving picture / still picture judging means for judging whether the two-dimensional input picture is a moving picture or a still picture, based on the judgment result of the moving picture / still picture judging means. A still image ratio calculating means for calculating a still image ratio, and based on the two-dimensional input image, extracts image feature amounts relating to the perspective of the image for each of the plurality of parallax calculation regions set in the one-field screen. , Depth information generating means for generating depth information for each parallax calculation region based on the extracted image feature amount, if the maximum value of the differences in depth information between adjacent regions is larger than a reference value, Set The first correction unit that corrects the dynamic range of the target phase amount so that the dynamic range of the target phase amount becomes smaller, and the dynamic range of the target phase amount obtained by the first correction unit is multiplied by the still image ratio, second correction means for correcting the dynamic range of the target phase amount, the dynamic range of the target phase amount obtained by the second correction means, each generated by the depth information generating unit
Based on depth information for each predetermined unit area based on the parallax calculation area, a target phase amount calculation means for calculating the target phase amount for each parallax calculation area, for each of the predetermined unit areas, based on the previous actual phase amount The above
Gradually approaches the target phase amount calculated by the target phase amount calculation means.
As described below, the actual phase amount calculating means for obtaining the actual phase amount this time, and the signal in each predetermined unit area of the right eye image and the left eye image output from the image converting means An apparatus for converting a two-dimensional image into a three-dimensional image, comprising phase control means for performing horizontal phase control according to the obtained actual phase amount corresponding to the predetermined unit area. 5. The still image ratio calculating means sets 0 as the still image ratio when the moving image / still image determining means determines that the two-dimensional input image is a moving image, and the two-dimensional input image is a still image. The two-dimensional image according to any one of claims 3 and 4, further comprising: means for setting 100 as a still image ratio when it is determined, and means for smoothing the set still image ratio between fields. A device that converts 3D images. 6. The actual phase amount calculation means calculates the amount of motion of the image in each parallax calculation region based on the two-dimensional input image, and the greater the amount of motion of the image, the greater the amount of actual phase of that region. The actual phase amount of this time for each parallax calculation region is calculated so as to approach the target phase amount of that region earlier, and based on the obtained actual phase amount of each parallax calculation region, the 1-field screen The apparatus for converting a two-dimensional image into a three-dimensional image according to any one of claims 1, 2, 3, 4 and 5, wherein the present actual phase amount is calculated for each of the predetermined unit areas.