JPH10191397A - Intention adaptive device for converting two-dimensional video into three-dimensional video - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an intention adaptive device for converting a two-dimensional video into three-dimensional video which emphasizes a stereoscopic effect of a specific feeling among persons and objects within photographic video in accordance with a person's intention. SOLUTION: A sensitive word is inputted, significant ranges which correspond to the sensitive word are separately selected for plural elements (high frequency component, luminance, luminance contrast, saturation, etc.) which device image feature quantity about the perspective of video, parallax calculating areas which become stereoscopic effect emphasizing objects are selected by an intention adaptive correction operating part 301 from the values of elements which are selected among the image feature quantity in each of plural parallax calculation areas that are set in one field screen, and a parallax modulating part 302 corrects the parallax information so that the stereoscopic effect of the selected parallax calculating areas may be emphasized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for converting a two-dimensional image into a three-dimensional image, and more particularly, to an intention-adaptive type adapted to obtain a three-dimensional image reflecting the intention of a photographer or an editor. The present invention relates to an apparatus for converting a two-dimensional image into a three-dimensional image.

[0002]

2. Description of the Related Art As a technique for converting a two-dimensional image into a three-dimensional image, the applicant of the present application uses image features (luminance, high-frequency components, luminance contrast, saturation, etc.) relating to the perspective of the image in each area within the screen. The parallax information for each region is calculated, and a signal corresponding to each region in the two-dimensional input video signal, a right-eye video signal and a left-eye video signal having a horizontal phase difference corresponding to the parallax information of the region. We have proposed a technique to generate 3D images by generating 3D images. With this technology, stereoscopic viewing of still images is difficult with a three-dimensional video conversion technology that simply delays a two-dimensional video by a frame and uses the original video as a video for one eye and the delayed video as a video for the other eye. Becomes possible.

[0003]

By the way, a two-dimensional image is usually obtained by a photographer by photographing a desired scenery with a video camera or the like. For example, a person is in front of a red rose. Even if the photographer looks at the standing landscape and wants to take a gorgeous landscape, and shoots with consciousness of the "rose", the "rose" is not in focus. A two-dimensional image cannot be obtained, and as a result, a three-dimensional image as intended may not be obtained. Also, even if a “rose” was taken as intended and a two-dimensional image was obtained with a gorgeous feeling, the three-dimensional effect of the “rose” was reduced by the three-dimensional image processing, and the “glossiness” was lost. May be

[0004] In view of the above circumstances, the present invention converts an intention-adaptive two-dimensional image into a three-dimensional image that can enhance the stereoscopic effect of a specific sense among persons and objects in a photographed image according to the intention of a person. It is an object of the present invention to provide a device for converting the data into a file.

[0005]

According to the present invention, there is provided an apparatus for converting an intention-adaptive two-dimensional image into a three-dimensional image according to the present invention. A feature amount extracting means for extracting an image feature amount related to perspective of a video for each of a plurality of parallax calculation regions set in one field screen; and an image feature amount extracted for each parallax calculation region. A disparity information generating unit that generates disparity information for each predetermined unit area in one field screen based on the disparity information corresponding to the predetermined unit area from a signal in each predetermined unit area of the two-dimensional input video signal. And phase control means for respectively generating a first video signal and a second video signal having a horizontal phase difference.
An apparatus for converting a three-dimensional image into a three-dimensional image, comprising: means for inputting a sensitivity word; and importance setting for selecting an important range corresponding to the sensitivity word for each of a plurality of elements for determining the image feature quantity. Means for selecting a parallax calculation area to be subjected to stereoscopic effect emphasis from the values of the selected elements among the image feature amounts of the respective parallax calculation areas; and a stereoscopic effect of the selected parallax calculation area. And a disparity information correcting unit that corrects the disparity information so that is emphasized.

With the above arrangement, the stereoscopic effect of a specific sensation of a person or an object in the imaging screen is emphasized in accordance with the input sensibility word, and according to the intention of the photographer or editor. 3D images can be obtained.

[0007]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an overall configuration of a 2D / 3D video converter for converting a 2D video into a 3D video.

A luminance signal Y, a color difference signal RY and a color difference signal BY constituting a two-dimensional video signal are supplied to an AD conversion circuit 1.
(ADC), the digital Y signal, R-
It is converted into a Y signal and a BY signal.

The Y signal is sent to a luminance integrating circuit 7, a high-frequency component integrating circuit 8, and a luminance contrast calculating circuit, and a first left image arbitrary pixel delay FIFO 11 and a first
To the right image arbitrary pixel delay FIFO 21. R-
The Y signal is sent to the saturation integration circuit 10 and sent to the second left image arbitrary pixel delay FIFO 12 and the second right image arbitrary pixel delay FIFO 22. The BY signal is
The signal is sent to the saturation accumulating circuit 10 and sent to the third left image arbitrary pixel delay FIFO 13 and the third right image arbitrary pixel delay FIFO 23.

The luminance integrating circuit 7 is provided for each field,
As shown in FIG. 2, an integrated luminance value is calculated for each of a plurality of parallax calculation areas E <b> 1 to E <b> 12 set in advance in one field screen. High frequency component integration circuit 8
Are the parallax calculation areas E1 to E12 for each field.
The integrated value of the high frequency component for each is calculated. The luminance contrast calculation circuit 9 calculates the luminance contrast for each of the parallax calculation areas E1 to E12 for each field. The luminance integrating circuit 10 calculates an integrated value of saturation for each of the parallax calculation areas E1 to E12 for each field.

The luminance integrated value for each of the parallax calculation areas E1 to E12, the integrated value of the high frequency component for each of the parallax calculation areas E1 to E12, and the respective parallax calculation areas E1 to E1
The integrated values of the luminance contrast for each of the two and the saturation for each of the parallax calculation regions E1 to E12 are image feature amounts relating to the perspective of the video for each of the parallax calculation regions E1 to E12.

Note that a total of 60 parallax calculation areas of 6 rows and 10 columns are actually set in the one-field screen as shown in FIG. 5 and the like, but for convenience of explanation, they are shown in FIG. As described above, it is assumed that a total of twelve parallax calculation areas E1 to E12 in three rows and four columns are set in one field screen.

The CPU 3 generates parallax information for each of the parallax calculation areas E1 to E12 based on the information sent from the brightness integration circuit 7, the high-frequency component integration circuit 8, the brightness contrast calculation circuit 9 and the saturation integration circuit 10. I do. In this example, the parallax information is generated such that the front-side object such as the subject has a smaller parallax amount, and the back-side object such as the background has a larger parallax amount. The details of the method of generating the parallax information will be described later.

The luminance integrating circuit 7 includes a plurality of registers provided corresponding to each parallax calculation area.
When distributing and storing the digitized luminance data supplied from 1 to a register in charge of a parallax calculation area to which the luminance data belongs based on a horizontal synchronization signal, a vertical synchronization signal, and a clock, The newly input luminance data is added to the stored luminance data. Then, for each field period, the luminance integration data for each parallax calculation area stored in each register is provided to the CPU 3 via a bus (not shown).

The high-frequency component integrating circuit 8 and the luminance contrast calculating circuit 9 have substantially the same configuration as the above-mentioned luminance integrating circuit 7, and receive digitized luminance data supplied from the ADC 1 to perform processing. Then, the integrated value of the high-frequency component and the integrated value of the luminance contrast are calculated and given to the CPU 3.

The saturation accumulating circuit 10 has the same configuration, but inputs the (RY) value and the (BY) value from the ADC 1 and performs a calculation based on the following equation to obtain the saturation. An arithmetic unit for calculating data (SAI) is provided.

[0018]

(Equation 1)

The saturation data of the arithmetic means is based on a horizontal synchronizing signal, a vertical synchronizing signal, and a clock, and the (RY) and (BY) values which are the basis of the saturation data belong to the arithmetic unit. When the parallax calculation area is sorted and stored in the register in charge, the newly input chroma data is added to the already stored chroma data. Then, the CPU 3 is provided with the saturation integration data for each parallax calculation area stored in each register for each field period.

The disparity information for each of the disparity calculation areas E1 to E12 calculated by the CPU 3 is sent to the disparity control circuit 4. The parallax control circuit 4 controls the parallax calculation areas E1 to E
Based on the disparity information for No. 12, disparity information for each pixel position in each field is generated. Then, based on the obtained parallax information for each pixel position, each of the FIFOs 11 to
13, 21 to 23, video signals (Y signal, RY signal,
Each of the FIs is adjusted so that the read address when reading the (BY signal) is shifted between the arbitrary pixel delay FIFOs 11 to 13 for the left image and the arbitrary pixel delay FIFOs 21 to 23 for the right image.
The read addresses of the FOs 11 to 13 and 21 to 23 are controlled. Therefore, the left image arbitrary pixel delay FIFOs 11 to 1
3 is different from the horizontal phase of the right video signal read from the right video arbitrary pixel delay FIFOs 21 to 23.

Arbitrary pixel delay FIFO 11-13 for left image
From the left video signal (YL signal, (RY) L
Signal, (BY) L signal) is a DA conversion circuit (DAC)
After being converted into an analog signal by 5, it is sent to a three-dimensional display device (not shown). Arbitrary pixel delay FIFO for right image
Right video signal (YR signal,
The (RY) R signal and the (BY) R signal are converted into analog signals by a DA conversion circuit (DAC) 6,
It is sent to a three-dimensional display device (not shown).

Since the horizontal phase of the left video signal is different from the horizontal phase of the right video signal, a parallax occurs between the left video and the right video. As a result, the left image is observed only with the left eye,
When the right image is observed only with the right eye, a three-dimensional image in which the subject is located in front of the background is obtained.

In FIG. 2, each of the parallax calculation areas E1 to E
12, the number of pixels in the horizontal direction is m, and each of the parallax calculation areas E1 to E
12, the number of pixels in the vertical direction is n, and the first parallax calculation area E1
The horizontal position (HAD), where the coordinates of the upper left of
And the vertical position (VAD).

FIG. 3 shows a method of calculating the amount of parallax performed by the CPU 3.

The first normalizing means 401 normalizes the integrated value of the high frequency component for each of the parallax calculation areas E1 to E12 to a value in the range of 0 to 10. The second normalization unit 402 sets the luminance contrast of each of the parallax calculation areas E1 to E12 to 0.
Normalize to a value in the range of -10. Third normalizing means 40
3 normalizes the integrated luminance value for each of the parallax calculation areas E1 to E12 to a value in the range of 0 to 10. Fourth normalizing means 4
A step 04 normalizes the saturation integration value for each of the parallax calculation areas E1 to E12 to a value in a range of 0 to 10.

Each of the normalized parallax calculation areas E1 to E12
The integrated value of the high-frequency component for each is multiplied by the coefficient K1 by the multiplying unit 405 and then sent to the adding unit 409. The normalized luminance contrast for each of the parallax calculation areas E1 to E12 is added to the coefficient K2 by the multiplication unit 406.
Are accumulated and sent to the adding means 409. After the coefficient K3 is multiplied by the multiplication means 407 to the normalized luminance integrated value for each of the parallax calculation areas E1 to E12,
It is sent to the adding means 409. The normalized saturation integrated value for each of the parallax calculation areas E1 to E12 is multiplied by a multiplication unit 408.
After the coefficient K4 is integrated by the above, it is sent to the adding means 409.

As specific examples of the coefficients K1, K2, K3, and K4, K1 = 0.6, K2 = 0.2, K3 = 0.1, K3
4 = 0.1. Also, K1 = 0.75, K2
= 0.25, K3 = 0.0, K4 = 0.0.

By controlling the set values of these coefficients K1 to K4, any one or any combination selected from the integrated value of the high-frequency component, the luminance contrast, the luminance integrated value and the chroma integrated value can be determined. , Can be used as an image feature amount related to the perspective of a video.

Therefore, it is also possible to use only the integrated value of the high frequency component as the image feature amount relating to the perspective of the video.
It is also possible to use only the luminance contrast as the image feature amount relating to the perspective of the video. The integrated value of the high-frequency component and the luminance contrast may be used as the image feature amount relating to the perspective of the video. An integrated value of a high-frequency component, a luminance contrast, and a luminance integrated value can also be used as the image feature amount relating to the perspective of a video. The integrated value of the high-frequency component, the luminance contrast, and the integrated value of the saturation can also be used as the image feature amount related to the perspective of the video. The integrated value of the high-frequency component, the luminance contrast, the integrated luminance value, and the integrated chroma value can also be used as the image feature amount relating to the perspective of the video.

In the adding means 409, each of the multiplying means 405
The values for each of the parallax calculation areas E1 to E12 obtained by 408 are added. The value for each of the parallax calculation areas E1 to E12 obtained by the adding unit 409 is normalized by the fifth normalizing unit 410 to a value in the range of 0 to 10 (hereinafter, referred to as depth information). FIG.
9 shows the relationship between the output value of No. 9 and the depth information obtained by the fifth normalization means 410. Each parallax calculation area E1
The depth information for each of E1 to E12 is the disparity calculation area E1 to E1
This is information on the distance between two images. Each of the parallax calculation areas E1 to E obtained by the fifth normalization unit 410
The depth information for each 12 is sent to the depth correction unit 411.

In a general image, a subject exists in the front and a background exists in the rear. Also, since there are many images that are in focus with respect to the subject, it is considered that the closer the object is, the higher the high-frequency component, contrast, brightness, and saturation are. Therefore, in the present embodiment, it is assumed that an object having a higher frequency is integrated in a region with a higher integrated value, luminance contrast, integrated value of luminance, and integrated value of saturation.

Therefore, it is possible to determine that the region where the depth information obtained by the adding means 409 is larger is the region in which the object existing ahead is captured. When the stereoscopic position of the region in which the object present in the forefront is captured is set as the tube position of the stereoscopic display device, the depth information obtained by the adding unit 409 is inversely proportional to the depth from the tube position.

Hereinafter, the depth correction processing by the depth correction means 411 will be described.

It is easier to understand the depth correction process by taking the actually set parallax calculation area as an example. Therefore, the 60 parallax calculation areas actually set for one field are taken as an example. The depth correction processing by the depth correction means 411 will be described. FIG. 5 shows 60 parallax calculation areas F1 to F60 actually set for one field.

First, an average value of depth information is calculated for each row of the parallax calculation areas F1 to F60. When the depth information for each of the parallax calculation areas F1 to F60 has a value as shown in FIG. 6, the average value of the depth information for each of the first to sixth rows is 1.2, 3.6, 6.0, 7.2, 4.0,
1.2.

Next, from each row of the parallax calculation area, an area where many objects at the near position are photographed is extracted. That is,
The row having the largest average value of the depth information is extracted. FIG.
In the example, the area of the fourth row is extracted.

Next, with respect to each area of the row below the extracted row, each area of the row below the extracted row is set so that the depth information does not suddenly decrease with respect to the area immediately above. Is adjusted. In particular,
For an area in which the depth information of each area in the row below the extracted row is smaller than the area immediately above by 3 or more, the depth information of the area is set to a value smaller by 2 than the depth information of the area immediately above. Can be changed.

In the example of FIG. 6, first, as shown in FIG.
Among the regions F41 to F50 in the fifth row, the region F whose depth information is smaller than the depth information of the region immediately above by three or more.
The depth information is corrected for 42 to F49. After that, of the areas F51 to F60 in the sixth row, the depth information is the depth information of the area immediately above (the corrected depth information).
The depth information is corrected for the regions F53 to F58 that are three or more smaller than.

That is, when the relationship of the depth information with respect to the screen height at an arbitrary horizontal position is as shown by the curve U1 in FIG. 8, the relationship of the depth information with respect to the screen height is determined by the depth correction. Is corrected so as to have a relationship as shown by a curve U2 in FIG.

As described above, in each row of the parallax calculation area,
The reason why the depth information in the area below the area where many objects at the near position are captured is corrected is as follows.

In general, an object existing in front is often photographed on the lower side of the screen. Also, the object shown on the lower side of the screen is often an image with little change such as the ground. An image with little change, such as the ground, has a low high-frequency component, so that the value of the depth information is small even though it is ahead. Therefore, depth information is increased by depth correction so that the depth information of an image located in front of the object and having a low high-frequency component is not larger than the value of the depth information of the area immediately above the image.

The depth information for each area (actually, F1 to F60, but E1 to E12 for convenience of explanation) whose depth information has been corrected by the depth correction means 411 is as follows.
The renormalization means 412 normalizes the data within the range of 0 to 10. The depth information for each of the areas E1 to E12 obtained by the re-normalization means 412 is
13 is converted into parallax information for each of the regions E1 to E12.

The parallax information generating means 413 converts the depth information into parallax information for each of the regions E1 to E12 based on the relationship between the predetermined depth information and the parallax information. The relationship between the depth information and the disparity information is inversely proportional, as shown by the straight line S1 or S2 in FIG.

In FIG. 9, the relationship between the depth information indicated by the straight line S1 and the parallax information is used when it is desired to obtain a stereoscopic image having a relatively strong stereoscopic effect. The relationship between the depth information indicated by the straight line S2 and the parallax information is used when it is desired to obtain a stereoscopic image having a relatively weak stereoscopic effect. By adjusting the relationship between the depth information and the parallax information based on the relationship between the straight line S1 and the straight line S2, it is possible to adjust the stereoscopic effect.

Each of the regions E1 to E1 thus obtained
The parallax information for each of the two is sent to the parallax control circuit 4 (see FIG. 1) after the intention adaptive correction described later is performed. Note that the depth correction by the depth correction unit 411 may be omitted.

Next, a description will be given of constituent parts for enhancing the stereoscopic effect of a specific part of the subject based on the sensitivity word. Here, the sensibility word is a word indicating the intention of the photographer or the editor, such as “glossy”, “spacious”, and “bright”, for example, as shown in FIG. 10 on the screen of the display device. As described above, an image is displayed for selecting a sensitivity word and its influence degree, and the influence degree of each sensitivity word can be selected by an input knob or the like provided on the display device.
When the degree of influence of "glossy" is set high, what kind of meaning it has in relation to the person or object in the captured video can be considered in various ways, for example, FIG.
In a shot image such as 0, it can be interpreted that the intention is to emphasize "flowers" around the person and not to emphasize "airplanes".

Since "flowers" often have primary colors, it is considered that the saturation of the portion (parallax calculation area) where "flowers" appear is high. Therefore, among several factors (luminance, high-frequency component, luminance contrast, and saturation) that determine the image feature amount, attention is particularly paid to “saturation” in correspondence with the sensitivity word “glossy”. From the information on the saturation of the parallax calculation area,
Is selected, and the three-dimensional effect of the region including the selected parallax calculation region is emphasized, the three-dimensional effect of the sense of "flower" is emphasized, and the intention of the photographer is reflected. It will be.

The intention adaptation processing unit 300 (see FIG. 1) is based on a sentiment instruction (input of each degree of influence of a sentiment word). Each important range is selected for the degree, and this important range information is given to the CPU 3. Here, when the sensitivity word “glossy” has a high degree of influence, important range information (corresponding to no important range) such as “0 to 10” is given for luminance, high-frequency components, and luminance contrast. For the degree, see "7-1
It is conceivable to give important range information such as "0".
It is a matter of course that other important elements besides the saturation may be given their respective important ranges. The output of the important range information based on the sentiment instruction may be performed using, for example, a look-up table in which the degree of influence of each sentiment word is associated with the important range of each element (intention parameter). it can.

Intention adaptive correction value calculation section 301 in CPU 3
(See FIG. 3), the important range information from the intention adaptation processing unit 300, the integrated value information for each normalized parallax calculation area from the normalization circuits 401 to 404, and the re-normalization unit 41.
2 receives the renormalized depth information for each parallax calculation area. The intention adaptive correction value calculation unit 301
Assuming that important range information such as “7 to 10” has been received for the saturation, the value is 7 to 1 based on the normalized integrated value of the saturation values from the normalization circuit 404 (see FIG. 11D).
A parallax calculation area of 0 is selected. FIG.
In the case of (a) and (d), the second column, the seventh column, and the ninth column of the second row from the bottom, which is an area corresponding to the area where “flower” exists in the captured video And the parallax calculation area in the tenth column is selected.

Then, the depth information of the selected parallax calculation area is extracted from the depth information of each parallax calculation area (see FIG. 11B) (see FIG. 11E). From this depth information, the parallax information of the selected parallax calculation area is obtained. If the disparity information of the disparity calculation area is as shown in FIG. 11C, “9” in the second column of the second row, “6” in the seventh column, “12” in the ninth column, And “12” in the tenth column is obtained as the disparity information of the selected disparity calculation area.

By the above processing, the parallax information of the area where the “flower” exists in the photographed image of FIG.
It can be seen that they exist with the width (range). The width (range) of the parallax information is a value related to the strength of the stereoscopic effect, and the stereoscopic effect is emphasized as the width increases. Therefore, the intention adaptive correction value calculation unit 301 determines that the width of the parallax information “6-1”.
"2" is widened, for example, "4-18", and "1"
“−6” and “12 to 30” generate correction information that narrows. If an example of this correction information is shown in a graph,
As shown in FIG.

The parallax modulator 302 (see FIG. 3) receives the above correction information and corrects the parallax information (see FIG. 11C) generated by the parallax information generator 413 according to the correction information. When the correction information has the content described above, the parallax information is corrected as shown in FIG. In addition, since the width (range) of the portion where the parallax information is “6 to 12” other than the region where “flower” is present is also expanded, it is possible to avoid that only “flower” unnaturally pops out. Then, based on the corrected disparity information, as described later, according to the disparity information after the intention adaptive correction corresponding to the predetermined unit area, from a signal in each predetermined unit area of the two-dimensional input video signal. A first video signal and a second video signal having a horizontal phase difference are respectively generated.

FIG. 12 is an explanatory diagram showing the flow of correction processing when a high-frequency component is selected from a plurality of elements that determine an image feature amount and "7 to 10" is given as important range information. It is. As for the high-frequency component, a normalized value of the integrated value of the high-frequency component in each parallax calculation area as shown in FIG. Then, a parallax calculation area having a value of “7 to 10” is selected as a normalized value. Further, the depth information of the selected parallax calculation area is extracted from the depth information of each parallax calculation area (see FIG. 3B) (see FIG. 3E), and the depth information is selected from the depth information. The parallax information of the obtained parallax calculation area is obtained. This disparity information is “0
１２12 ”. Then, correction information is generated such that “0 to 12” is widened, for example, to “0 to 18”, and “12 to 30” is narrowed, for example. A graph of an example of the correction information is as shown in FIG. Then, the disparity information (output from the re-normalization circuit 412) in FIG. 11C is corrected by the disparity modulation unit 302 as shown in FIG.

FIG. 13 is an explanatory diagram showing a case where correction information is synthesized. In the example of this figure, correction information is generated by combining the correction information shown by the graph of FIG. 11G and the correction information shown by the graph of FIG. Then, the intention adaptive correction is performed on the parallax information of each parallax calculation area based on the combined correction information. For example, with respect to a certain sensitivity word, an important range is selected for each of the luminance, high-frequency component, luminance contrast, and saturation, which are elements that determine the image feature amount. In such a case, correction information (correction curve) is generated for each element, so that these are integrated to generate one correction information, and parallax information is generated based on the integrated correction information. It will be corrected.

FIG. 14 mainly shows the configuration of the parallax control circuit 4 and the arbitrary pixel delay FIFO of FIG.

FIG. 14 shows an arbitrary pixel delay FIFO 11 to 11.
13, 21 to 23, left image arbitrary pixel delay FIFO 11 and right image arbitrary pixel delay FI for Y signal
Although only FO21 is shown, other arbitrary pixel delays FI
Since the FOs 12, 13, 22, and 23 have the same configuration and perform the same control, the other arbitrary pixel delay FIFO1
A description of the configurations and control methods of 2, 13, 22, and 23 will be omitted.

The disparity information generated by the CPU 3 is disparity information for the center position of each of the disparity calculation areas E1 to E12. The parallax control circuit 4 obtains parallax information for each pixel position on a one-field screen based on the parallax information for the center position of each of the parallax calculation areas E1 to E12. Then, in order to generate a left image and a right image having parallax corresponding to the parallax information for the pixel position from the two-dimensional video signal for each pixel position, an arbitrary left video image is generated based on the parallax information for each pixel position. Pixel delay F
IFO 11-13 and right pixel arbitrary pixel delay FIFO
The read addresses 21 to 23 are controlled.

The parallax information for each pixel position on the one-field screen is obtained from a timing signal generation circuit 51, a parallax interpolation coefficient generation circuit 52, a parallax information storage means 60, and a parallax selection circuit 8.
0, and is generated by the first to fourth multipliers 81 to 84 and the addition circuit 85.

The horizontal synchronizing signal Hsync and the vertical synchronizing signal Vsync of the input video signal are input to the timing signal generating circuit 51. Further, a clock signal CLK for detecting a horizontal address in each horizontal period is also input to the timing signal generation circuit 51.

The timing signal generating circuit 51 generates a horizontal address signal HAD indicating an absolute horizontal position of the input video signal, an absolute vertical position of the input video signal based on the horizontal synchronizing signal Hsync, the vertical synchronizing signal Vsync, and the clock signal CLK. And a relative horizontal position signal HPOS representing the relative horizontal position of the input video signal.
And a relative vertical position signal VPOS indicating the relative vertical position of the input video signal is generated and output.

The relative horizontal position and the relative vertical position of the input video signal will be described.

As shown in FIG. 15, the parallax calculation position areas E1 to E12 in FIG. 2 are set as follows. The entire screen is divided into 20 regions of 4 rows and 5 columns (hereinafter, referred to as a first divided region) as indicated by a dotted line in FIG.
Then, three rows of quadrangular areas having four vertices at the center of the first divided area at the upper left corner, the center of the first divided area at the upper right corner, the center of the first divided area at the lower left corner, and the center of the first area at the lower right corner are provided. 4
The column is divided into 12 regions (hereinafter, referred to as second divided regions), and each of the second divided regions is set as parallax calculation regions E1 to E12.

The number of pixels in the horizontal direction of the first divided region and the second divided region is represented by m, and the number of pixels in the first divided region and the second divided region in the vertical direction is represented by n. The relative horizontal position of the input video signal is represented by 0 to (m-1), with the left end of each first divided area being 0 and the right end being m. The relative vertical position of the input video signal is represented by 0 to (n-1), with the upper end of each first divided area being 0 and the lower end being n.

The relative horizontal position signal HPO of the input video signal
S and the relative vertical position VPOS are sent to the parallax interpolation coefficient generation circuit 52. The parallax interpolation coefficient generation circuit 52 generates a first parallax interpolation coefficient KUL, a second parallax interpolation coefficient KUR, a third parallax interpolation coefficient KDL, and a third parallax interpolation coefficient KUL based on the relative horizontal position signal HPOS, the relative vertical position VPOS, and the following equation. A four-parallax interpolation coefficient KDR is generated and output.

[0065]

KUL = (m-HPOS) / m * (n-VPOS) / n KUR = HPOS / m * (n-VPOS) / n KDL = (m-HPOS) / m * VPOS / n KDL = HPOS ) / M * VPOS / n

FIG. 1 shows the basic concept of a method of generating parallax information for each pixel position on a one-field screen.
6 will be described. It is assumed that the horizontal / vertical position (hereinafter, referred to as a target position) represented by the horizontal address signal HAD and the vertical address signal VAD is Pxy in FIG. A case where parallax information for the target position Pxy is obtained will be described.

(1) First, from among the parallax information for each of the parallax calculation areas E1 to E12 calculated by the CPU 3, four vertices of the first divided area including the target position Pxy,
In this example, disparity information for the disparity calculation areas E1, E2, E5, and E6 centered on PE1, PE2, PE5, and PE6 is extracted as UL, UR, DL, and DR, respectively. That is, of the four vertices of the first divided region including the target position Pxy, the disparity information of the region E1 centered on the upper left vertex is the first disparity information UL, and the disparity information of the region E2 centered on the upper right vertex is UL. Is extracted as the second disparity information UR, the disparity information of the region E5 centered on the lower left vertex is extracted as the third disparity information DL, and the disparity information of the region E6 centered on the lower right vertex is extracted as the fourth disparity information DR. You.

However, only one vertex of the four vertices of the first divided region including the target position is disparity, as in the case where the first divided region including the target position is the first upper left divided region. If the parallax information corresponds to the center of the detection area, the parallax information of the parallax calculation area is the first to fourth parallax information U
Extracted as L, UR, DL, DR.

Also, as in the case where the first divided region including the target position is the first divided region to the right of the first divided region at the upper left end, four vertices of the first divided region including the target position are used. If only the lower two vertices of the first divided area correspond to the center of the parallax calculation area, the parallax information UL corresponding to the upper two vertices of the four vertices of the first divided area including the target position, As the UR, parallax information of a parallax calculation area centered on the lower vertex is extracted.

Also, as in the case where the first divided region including the target position is the first divided region below the first divided region at the upper left end, the four vertices of the first divided region including the target position are used. Among the four vertices of the first divided area including the position of interest, only the two vertices on the right correspond to the center of the parallax calculation area. , The parallax information of the parallax calculation area centered on the right vertex is extracted.

Also, as in the case where the first divided region including the target position is the first divided region to the left of the first lower right divided region, the four vertices of the first divided region including the target position are used. If only the upper two vertices of the first divisional area correspond to the center of the parallax calculation area, the parallax information DL corresponding to the lower two vertices of the four vertices of the first divided area including the target position, As DR, parallax information of a parallax calculation area centered on the upper vertex is extracted.

Further, as in the case where the first divided region including the target position is the first divided region above the first divided region at the lower right, four vertices of the first divided region including the target position are used. Among the four vertices of the first divided area including the target position, only the two vertexes on the left side of the parallax calculation area correspond to the center of the parallax calculation area. , The parallax information of the parallax calculation area centered on the left vertex is extracted.

(2) Next, the first to fourth parallax interpolation coefficients K
UL, KUR, KDL and KDR are required.

The first parallax interpolation coefficient KUL is determined by the target position P
xy, the distance ΔX from the target position Pxy to the right side of the first divided area e with respect to the horizontal width m of the first divided area e including xy
R and the ratio {(m-HPOS) / m} to the first divided region e
の (n−VPO) of the vertical width n to the distance ΔYD from the target position Pxy to the lower side of the first divided region e.
S) / n}. That is, the first
Is larger as the distance between the upper left vertex PE1 of the first divided area e including the target position Pxy and the target position Pxy is smaller.

The second parallax interpolation coefficient KUR is calculated based on the target position P
xy, the distance ΔX from the target position Pxy to the left side of the first divided region e with respect to the horizontal width m of the first divided region e
L (HPOS / m) and the ratio {(n−VPOS) / n} of the distance ΔYD from the target position Pxy to the lower side of the first divided region e with respect to the vertical width n of the first divided region e.
And the product of That is, the second parallax interpolation coefficient KUR increases as the distance between the upper right vertex PE2 of the first divided region e including the target position Pxy and the target position Pxy decreases.

The third parallax interpolation coefficient KDL is calculated based on the target position P
xy, the distance ΔX from the target position Pxy to the right side of the first divided area e with respect to the horizontal width m of the first divided area e including xy
R and the ratio {(m-HPOS) / m} to the first divided region e
(VPOS / n) of the distance ΔYU from the target position Pxy to the upper side of the first divided area e with respect to the vertical width n of
And the product of That is, the third parallax interpolation coefficient KDL increases as the distance between the lower left vertex PE5 of the first divided region e including the target position Pxy and the target position Pxy decreases.

The fourth parallax interpolation coefficient KDR is calculated based on the target position P
xy, the distance ΔX from the target position Pxy to the left side of the first divided region e with respect to the horizontal width m of the first divided region e
L and the ratio (VPOS / n) of the vertical width n of the first divided area e to the distance ΔYU from the target position Pxy to the upper side of the first divided area e. Desired. That is, the fourth parallax interpolation coefficient KD
R increases as the distance between the lower right vertex PE6 of the first divided area e including the target position Pxy and the target position Pxy decreases.

(3) The first to fourth disparity interpolation coefficients KU calculated in (2) are respectively added to the first to fourth disparity information UL, UR, DL, DR extracted in (1).
L, KUR, KDL, and KDR are respectively multiplied. Then, by adding the obtained four multiplication values,
Parallax information for the target position Pxy is generated.

The parallax information storage means 60 stores the areas E1 to E1.
2 are provided with first to twelfth parallax registers 61 to 72 provided correspondingly to the two. The first to twelfth disparity registers 61 to 72 store disparity information for each of the regions E1 to E12 generated by the CPU 3.

At the subsequent stage of the parallax information storage means 60, a parallax selection circuit 80 is provided. In the parallax selection circuit 80,
Parallax information is sent from each of the parallax registers 61 to 72. Further, the horizontal address signal HAD and the vertical address signal VAD are sent from the timing signal generation circuit 51 to the parallax selection circuit 80.

In accordance with the rule shown in FIG. 17A, the parallax selecting circuit 80 determines the area corresponding to the horizontal address signal HAD and the vertical address signal VAD (in the example of FIG. 16, the first area including the signal of interest). The disparity information for the disparity calculation area centered on the upper left vertex is selected and output as first disparity information UL. Further, the parallax selection circuit 80
According to the rule shown in FIG. 17B, a region corresponding to the horizontal address signal HAD and the vertical address signal VAD (in the example of FIG. 16, a parallax calculation region centered on the upper right vertex of the first region including the target position) ),
It is selected and output as the second parallax information UR.

Further, the parallax selecting circuit 80 is provided in FIG.
According to the rule shown in (c), a region corresponding to the horizontal address signal HAD and the vertical address signal VAD (a parallax calculation region centered on the lower left vertex of the first region including the target position in the example of FIG. 16). Parallax information is
Select and output as parallax information DL. Further, the parallax selecting circuit 80 applies the horizontal address signal HAD and the vertical address signal V in accordance with the rule shown in FIG.
The disparity information for the region corresponding to AD (in the example of FIG. 16, the disparity calculation region centered on the lower right vertex of the first region including the target position) is selected and output as the fourth disparity information DR. In FIG. 17, for example, a to
The symbol "~" represented by b is used as a symbol meaning a or more and less than b.

The first selected by the parallax selection circuit 80
The disparity information UL, the second disparity information UR, the third disparity information DL, and the fourth disparity information DR are input to first, second, third, and fourth multipliers 81, 82, 83, 84, respectively.

First, second, third and fourth multipliers 8
1, 82, 83, and 84 also receive the first parallax interpolation coefficient KUL, the second parallax interpolation coefficient KUR, the third parallax interpolation coefficient KDL, and the fourth parallax interpolation coefficient KDR from the parallax interpolation coefficient generation circuit 52, respectively. I have.

The first multiplier 81 multiplies the first disparity information UL by a first disparity interpolation coefficient KUL. The second multiplier 82
The second parallax information UR is multiplied by a second parallax interpolation coefficient KUR. The third multiplier 83 multiplies the third disparity information DL by a third disparity interpolation coefficient KDL. The fourth multiplier 84 multiplies the fourth disparity information DR by a fourth disparity interpolation coefficient KDR.

The outputs of the multipliers 81, 82, 83 and 84 are added by an adder 85. Thereby, the disparity information PR for the target position is obtained.

Each of the arbitrary pixel delay FIFOs 11 and 21 is 1
In order to perform horizontal phase control in units smaller than pixels, two line memories 11a, 11b, 21a,
1b. Each arbitrary pixel delay FIFO11, 21
Of the two line memories 11a, 11b, 21a, 21
In b, the Y signal is input and the clock signal CLK is input.

The horizontal address signal HAD output from the timing signal generation circuit 51 is also input to the standard address generation circuit 90. Standard address generation circuit 90
Is a standard write address WAD and a standard read address R for the two line memories 11a, 11b, 21a, 21b in each of the arbitrary pixel delay FIFOs 11, 21.
Generate and output AD. Further, the standard address generation circuit 90 also outputs a synchronization signal Csync added to the left video signal and the right video signal obtained by the 2D / 3D converter. The horizontal synchronization signal represented by the synchronization signal Csync is a horizontal synchronization signal Hsync of the input video signal.
The signal is delayed by a predetermined number of clocks from c.

The standard read address RAD is used to advance or delay the horizontal phase of the video signal input to each of the arbitrary pixel delay FIFOs 11 and 21 with respect to the reference horizontal phase defined by the standard read address. In addition, it is delayed by a predetermined number of clocks from the standard write address WAD. The standard write address WAD output from the standard address generation circuit 90 is stored in the two line memories 11 in each of the arbitrary pixel delay FIFOs 11 and 21.
a, 11b, 21a, and 21b are input as write control signals indicating write addresses.

The standard read address RAD output from the standard address generation circuit 90 is input to an adder 91 and a subtractor 92, respectively. Adder 91 and subtractor 9
2, the parallax information PR of the target position output from the adding circuit 85 is also input.

In the adder 91, the standard read address R
The disparity information PR is added to AD. As a result, the left video read address PRL is obtained.

The integer part PRL1 of the left video read address PRL is stored in the first line memory 11a in the left video arbitrary pixel delay FIFO 11 in the read address RADL1.
Enter as Therefore, the first line memory 11
The Y signal is read from the address corresponding to the address RADL1 of a. The read Y signal is input to the first left video multiplier 101.

The address value obtained by adding 1 to the integer part PRL1 of the left video read address PRL is input to the second line memory 11b in the left video arbitrary pixel delay FIFO 11 as the read address RADL2. Therefore, the Y signal is read from the address corresponding to the address RADL2 of the second line memory 11b. The read Y signal is input to the second left video multiplier 102.

The read address RADL1 for the first line memory 11a is different from the read address RADL2 for the second line memory 11b by one, so that the Y signal read from the first line memory 11a The Y signal read from the second line memory 11b is a signal whose horizontal position is shifted by one.

The decimal part PRL2 of the left video read address PRL is input to the second left video multiplier 102 as a second left video interpolation coefficient. A value obtained by subtracting the decimal part PRL2 of the read address PRL for the left image from 1 (1-PR
L2) is input to the first left video multiplier 101 as a first left video interpolation coefficient.

Therefore, the first left video multiplier 101
Then, the Y signal read from the first line memory 11a is multiplied by a first left video interpolation coefficient (1-PRL2). The second left video multiplier 102 multiplies the Y signal read from the second line memory 11b by the second left video interpolation coefficient PRL2. Then, each multiplier 10
The Y signals obtained by 1 and 102 are added by an adder 103, and then output as a left video Y signal YL-OUT.

Thus, the standard read address RAD
With respect to the reference horizontal phase defined by the above, a left image Y signal whose horizontal phase amount is delayed by an amount corresponding to the parallax information for the target position is obtained.

In the subtractor 92, the standard read address R
The disparity information PR is subtracted from AD. As a result, the right video read address PRR is obtained.

The integer part PRR1 of the read address PRR for the right image is inputted as the read address RADR1 to the first line memory 21a in the arbitrary pixel delay FIFO 21 for the right image. Therefore, the first line memory 21a
Is read from the address corresponding to the address RADR1. The read Y signal is input to the first right video multiplier 111.

The address value obtained by adding 1 to the integer part PRR1 of the right video read address PRR is input to the second line memory 21b in the right video arbitrary pixel delay FIFO 21 as the read address RADR2. Therefore, the Y signal is read from the address corresponding to the address RADR2 of the second line memory 21b. The read Y signal is input to the second right video multiplier 112.

The read address RADR1 for the first line memory 21a is different from the read address RADR2 for the second line memory 21b by one, so that the Y signal read from the first line memory 21a is The Y signal read from the second line memory 21b is a signal whose horizontal position is shifted by one.

The fractional part PRR2 of the read address PRR for the right image is input to the second right image multiplier 112 as a second right image interpolation coefficient. A value obtained by subtracting the decimal part PRR2 of the read address PRR for the right image from 1 (1-PR
R2) is input to the first right image multiplier 111 as a first right image interpolation coefficient.

Therefore, the first right video multiplier 111
Then, the Y signal read from the first line memory 21a is multiplied by a first right video interpolation coefficient (1-PRR2). In the second right video multiplier 112, the Y signal read from the second line memory 21b is multiplied by a second right video interpolation coefficient PPR2. Then, each multiplier 111,
The Y signal obtained by 112 is added by the adder 113 and then output as a right video Y signal YR-OUT.

Thus, the standard read address RAD
With respect to the reference horizontal phase defined by the above, a Y signal for right video is obtained in which the horizontal phase amount is advanced by an amount corresponding to the parallax information for the target position.

FIG. 18 shows that the parallax information for the target position is 0.
3 shows the signals of the respective units.

If the disparity information is 0, the left video read address PRL output from the adder 91 and the right video read address PPR output from the subtractor 92 are output.
Is an address consisting of only an integer part without a decimal part equal to the standard read address RAD.

Therefore, an arbitrary pixel delay FIF for left video
Read address RADL1 for the first line memory 11a in O11 and right image arbitrary pixel delay FIFO
The read address RADR1 for the first line memory 21a in the memory 21 is equal to the standard read address RAD.

Further, a left image arbitrary pixel delay FIFO 11
The read address RADL2 for the second line memory 11b in the first and the read address RADR2 for the second line memory 21b in the right image arbitrary pixel delay FIFO 21 have a value that is one greater than the standard read address RAD.

The first left image interpolation coefficient (1-PRL)
2) and the first right video interpolation coefficient (1-PRR2) are 1
And the second left video interpolation coefficient PRL2 and the second right video interpolation coefficient PRR2 become zero.

As a result, the left image arbitrary pixel delay FIFO
11, the standard address RA of the first line memory 11a
The Y signal read from the address corresponding to D is output from the adder 103 as the left video Y signal YL-OUT, and corresponds to the standard address RAD of the first line memory 21a in the right video arbitrary pixel delay FIFO 21. The Y signal read from the address to be output is output from the adder 113 as a right video Y signal YR-OUT. That is, two Y signals having the same horizontal phase shift amount, that is, two Y signals having no parallax.
The two Y signals are output as a left video Y signal and a right video Y signal.

FIG. 19 shows each address value when the standard write address WAD for a certain target position is 20, the standard read address RAD for the target position is 10, and the disparity information for the target position is 1.2. Is shown. FIG. 20 shows the signals of the respective units at that time.

In this case, the left video read address PRL output from the adder 91 is 11.2,
Its integer part PRL1 becomes 11, and its decimal part PRL2
Is 0.2.

Therefore, the left image arbitrary pixel delay FIF
The read address RADL1 for the first line memory 11a in O11 is 11, and the read address RADL2 for the second line memory 11b is 12. Further, the first left video interpolation coefficient KL1 ｛= (1-PR
L2)｝ is 0.8, and the second left image interpolation coefficient KL2
(= PRL2) is 0.2.

Accordingly, the left image arbitrary pixel delay FIF
The Y signal (Y ₁₁ ) is read from the address 11 of the first line memory 11 a in O 11, and a signal (0) that is 0.8 times the read Y signal (Y ₁₁ ) is read from the first multiplier 101. .
8 * Y ₁₁ ) is output.

On the other hand, a left image arbitrary pixel delay FIFO 11
, The Y signal (Y ₁₂ ) is read from the address 12 of the second line memory 11 b, and the signal (0...) Which is 0.2 times the Y signal (Y ₁₂ ) read from the second multiplier 102. 2 * Y
₁₂ ) is output. Then, from the adder 103, 0.
Y signal YL for left image corresponding to 8 * Y ₁₁ + 0.2 * Y ₁₂
−OUT is output. That is, read address 1
The Y signal corresponding to 1.2 is a left video Y signal YL-OU.
Output as T.

The right video read address PRR output from the subtractor 92 is 8.8, and its integer part PRR
1 becomes 8, and the decimal part PRR2 becomes 0.8.

Therefore, the right picture arbitrary pixel delay FIF
The read address RADR1 for the first line memory 21a in O21 becomes 8, and the second line memory 2a
The read address RADR2 for 1b is 9.
Also, the first right video interpolation coefficient KR1 ｛= (1−PRR)
2) と な り is 0.2, and the second right video interpolation coefficient KR2
(= PRR2) is 0.8.

Therefore, the arbitrary pixel delay FIF for the right image
Y signal (Y ₈₎ is read from the first line memory 21a of the address 8 in O21, 0.2 times the signal of the Y signal read out from the first multiplier 111 (Y ₈₎ (0 .2
* Y ₈ ) is output.

On the other hand, a right image arbitrary pixel delay FIFO 21
, The Y signal (Y ₉ ) is read from the address 9 of the second line memory 21 b, and the signal (0...) 0.8 times the read Y signal (Y ₉ ) from the second multiplier 112. 8 *
Y ₉ ) is output. Then, from the adder 113,
A right video Y signal YR-OUT corresponding to 0.2 * Y ₈ + 0.8 * Y ₉ is output. In other words, the Y signal corresponding to the read address 8.8 is the right video Y signal YR-OU.
Output as T.

As a result, a left image and a right image having a disparity of 11.2−8.8 = 2.4, that is, a disparity twice as large as the disparity information 1.2 are obtained.

[0121]

As described above, according to the present invention, the three-dimensional effect of a specific sensation of a person or an object in an imaging screen is emphasized in accordance with an input sensibility word. Alternatively, it is possible to obtain a three-dimensional image according to the editor's intention.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an apparatus for converting an intention-adaptive two-dimensional image into a three-dimensional image according to an embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating a parallax calculation area.

FIG. 3 is a block diagram illustrating details of a CPU in FIG. 1;

FIG. 4 is a graph showing an input / output relationship of a normalization unit 410 of FIG.

FIG. 5 is a schematic diagram showing a parallax calculation area actually set.

FIG. 6 is a schematic diagram illustrating an example of depth information of each parallax calculation area before depth correction.

FIG. 7 is a schematic diagram illustrating depth information of each parallax calculation area after depth correction.

FIG. 8 is a graph showing a relationship between a height position of a screen before depth correction and depth information and a relationship between a height position of a screen after depth correction and depth information.

FIG. 9 is a graph showing a relationship between depth information and disparity information.

FIG. 10 is an explanatory diagram regarding selection of a sensitivity word and its influence degree.

FIG. 11 is an explanatory diagram showing intention adaptation correction of disparity information based on saturation.

FIG. 12 is an explanatory diagram illustrating intention adaptive correction of disparity information based on a high-frequency component.

13 is an explanatory diagram when performing intentional adaptive correction using a correction curve obtained by integrating the correction curve of FIG. 11G and the correction curve of FIG. 12G.

FIG. 14 is a block diagram mainly showing a configuration of a parallax control circuit and an arbitrary pixel delay FIFO.

FIG. 15 is a schematic diagram showing a relative horizontal position, a relative vertical position, and the like.

FIG. 16 is an explanatory diagram for describing a method of generating disparity information for a target pixel.

FIG. 17 is a diagram illustrating a selection rule by a parallax selection circuit.

FIG. 18 is a time chart illustrating signals of respective units when disparity information is 0.

FIG. 19 is a block diagram in which each address value when the disparity information is 1.2 is added to the disparity control circuit.

FIG. 20 is a time chart illustrating signals of respective units when the disparity information is 1.2.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 AD converter 3 CPU 4 Parallax control circuit 5, 6 DA converter 7 Luminance integration circuit 8 High frequency component integration circuit 9 Luminance contrast calculation circuit 10 Saturation integration circuit 300 Intention adaptation processing unit 302 Intention adaptation correction value calculation unit 303 Parallax modulation Department

Claims (1)

[Claims] 1. A feature amount for extracting an image feature amount related to perspective of a video for each of a plurality of parallax calculation regions set in one field screen for each field based on a two-dimensional input video signal. Extracting means, disparity information generating means for generating disparity information for each predetermined unit area in one field screen based on the image feature amount extracted for each disparity calculation area, and each predetermined unit of the two-dimensional input video signal A phase control unit for generating a first video signal and a second video signal having a horizontal phase difference corresponding to the parallax information corresponding to the predetermined unit region from the signals in the region. A device for converting into a three-dimensional image, comprising: means for inputting a sensitivity word; and importance for selecting an important range corresponding to the sensitivity word for each of a plurality of elements for determining the image feature quantity Determining means, area selecting means for selecting a parallax calculation area to be subjected to stereoscopic effect enhancement from the values of the selected elements among the image feature amounts of the respective parallax calculation areas, A device for converting an intention-adaptive two-dimensional image into a three-dimensional image, comprising: a disparity information correcting unit that corrects the disparity information so that a sense is enhanced.