JP5780214B2 - Depth information generation device, depth information generation method, depth information generation program, pseudo stereoscopic image generation device - Google Patents

Description

  The present invention relates to a depth information generation device that generates depth information from a non-stereo image that is not given depth information explicitly as in a normal image and that is not given depth information like a stereo image. The present invention relates to a depth information generation method, a depth information generation program, and a pseudo stereoscopic image generation apparatus that generates a pseudo stereoscopic image using depth information.

  In recent years, with the widespread use of stereoscopic images, non-stereo images that are not given depth information either explicitly or implicitly have been converted into pseudo stereoscopic images. Various pseudo-stereoscopic image generation apparatuses for converting a non-stereoscopic image into a pseudo-stereoscopic image have been proposed.

  As an example, Patent Literature 1 describes a pseudo stereoscopic image generation device that generates depth estimation data using a plurality of types of basic depth models indicating depth values for each of a plurality of types of basic scene structures.

JP 2005-151534 A

  In the depth information generation device included in the pseudo-stereoscopic image generation device described in Patent Literature 1, it is impossible to generate depth information that can provide a sufficient stereoscopic effect with an image having a subject of interest such as a person in the center of the screen. There was a problem.

  In view of such a problem, the present invention provides a depth information generation device, a depth information generation method, a depth information, and a depth information generation method capable of generating depth information that provides a sufficient stereoscopic effect for an image having a subject of interest in the center of the screen. An object is to provide an information generation program. It is another object of the present invention to provide a pseudo-stereoscopic image generation apparatus that can obtain a sufficient stereoscopic effect for an image having a subject of interest at the center of the screen.

  In order to solve the above-described problems of the conventional technology, the present invention uses a basic depth model for determining the parallax value of the entire screen and generates parallax data indicating the parallax value of each pixel in the screen. The generation unit (114 to 117) and a center for calculating central part distribution ratio data indicating the degree to which the target subject included in the non-stereoscopic image signal is concentrated and distributed in the first central part excluding the peripheral part of the screen The amplitude of the non-stereoscopic image signal in the second central portion that is the same as or different from the first central portion of the screen is increased according to the size of the partial distribution ratio calculating unit (84) and the central portion distribution ratio data. An amplitude correction unit (86, 87) that outputs a corrected non-stereoscopic image signal, parallax data generated by the parallax data generation unit, and a corrected non-stereoscopic image signal output from the amplitude correction unit Add Te, provides depth information generating device (1), characterized in that it comprises an adder (119) for generating a depth estimation data for each pixel within the screen.

  In the depth information generation apparatus, the disparity data generation unit combines disparity data indicating disparity values of the entire screen of each of the plurality of basic depth models based on the characteristics of the non-stereoscopic image signal (117). ).

  In the depth information generating apparatus, an upper high-frequency component evaluation unit (112) that evaluates the amount of a high-frequency component included in a predetermined region on the screen and generates an upper high-frequency component evaluation value, and a lower portion on the screen A lower high-frequency component evaluation unit (113) that generates a lower high-frequency component evaluation value by evaluating an amount of the high-frequency component included in the predetermined region, and the combining unit evaluates the upper high-frequency component evaluation It is preferable to combine disparity data indicating disparity values of the entire screen of each of the plurality of basic depth models according to the value and the lower high-frequency component evaluation value.

  In the depth information generating apparatus, the central portion distribution ratio calculating unit indicates high frequency component frequency data indicating the frequency of high frequency components included in the peripheral portion, and indicates the frequency of high frequency components included in the first central portion. It is preferable to calculate the central portion distribution ratio data using high frequency component frequency data.

  In the depth information generating apparatus, the non-stereoscopic image signal is preferably a red signal among the three primary color signals.

  In order to solve the above-described problems of the related art, the present invention provides a depth information generation apparatus having any one of the above configurations and a plurality of image data for a plurality of viewpoints based on the depth estimation data. Provided is a pseudo-stereoscopic image generation device including a viewpoint image data generation unit (2).

  Furthermore, the present invention generates disparity data indicating the disparity value of each pixel in the screen using a basic depth model for determining the disparity value of the entire screen in order to solve the above-described problems of the conventional technology. The central portion distribution ratio data indicating the degree to which the subject of interest included in the non-stereoscopic image signal is concentrated and distributed in the first central portion excluding the peripheral portion of the screen is calculated, and the size of the central portion distribution ratio data is calculated. Accordingly, correction is performed to increase the amplitude of the non-stereoscopic image signal in a second central portion that is the same as or different from the first central portion of the screen to generate a corrected non-stereoscopic image signal, and the parallax data And a corrected non-stereoscopic image signal are added to generate depth estimation data for each pixel in the screen.

  Furthermore, in order to solve the above-described problems of the conventional technology, the present invention uses a basic depth model for determining the parallax value of the entire screen in the computer, and the parallax indicating the parallax value of each pixel in the screen. Generating data, calculating center portion distribution ratio data indicating the degree to which the target subject included in the non-stereoscopic image signal is concentrated and distributed in the first center portion excluding the peripheral portion of the screen, A corrected non-stereo image is corrected by increasing the amplitude of the non-stereo image signal in a second central portion that is the same as or different from the first central portion of the screen according to the size of the central portion distribution ratio data. Generating a signal, and adding the parallax data and the corrected non-stereoscopic image signal to generate depth estimation data for each pixel in the screen. Providing depth information generating program according to claim.

  According to the depth information generation device, the depth information generation method, and the depth information generation program of the present invention, it is possible to generate depth information that provides a sufficient stereoscopic effect for an image having a subject of interest at the center of the screen. In addition, according to the pseudo stereoscopic image generation device of the present invention, it is possible to obtain a sufficient stereoscopic effect for an image having a subject of interest in the center of the screen.

It is a block diagram which shows one Embodiment of the pseudo | simulation stereoscopic image generation apparatus of this invention. It is a specific structural example of the depth estimation part 11 in FIG. 1, and is a block diagram which shows one Embodiment of the depth information generation apparatus of this invention. It is a figure which shows an example of the three-dimensional structure of the basic depth model (type 1) which the frame memory 114 in FIG. 2 hold | maintains. It is a figure which shows an example of the three-dimensional structure of the basic depth model (type 2) which the frame memory 115 in FIG. 2 hold | maintains. It is a figure which shows an example of the three-dimensional structure of the basic depth model (type 3) which the frame memory 116 in FIG. 2 hold | maintains. It is a figure which shows an example of the synthetic | combination ratio determination conditions by the synthetic | combination part 117 in FIG. FIG. 3 is a block diagram illustrating a specific configuration example of an object signal correction unit 118 in FIG. 2. It is a figure which shows an example of the area | region where the high frequency component frequency detection parts 81 and 82 in FIG. 7 each detect the frequency of a high frequency component. It is a block diagram which shows the specific structural example of the high frequency component frequency detection parts 81 and 82 in FIG. It is a figure for demonstrating the detection phase of the high frequency component by the high pass filter 801 in FIG. FIG. 8 is a block diagram illustrating a specific configuration example of a central portion distribution ratio calculation unit 84 in FIG. 7. FIG. 8 is a block diagram illustrating a specific configuration example of a leaky integration circuit 85 in FIG. 7. FIG. 8 is a characteristic diagram of a gain calculated by a gain calculation unit 86 in FIG. 7. FIG. 14 is a characteristic diagram of a gain limit value lim shown in FIG. It is a figure which shows the example of a display of the pseudo | simulation stereo image for demonstrating the effect by one Embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, an embodiment of a depth information generation device, a depth information generation method, a depth information generation program, and a pseudo stereoscopic image generation device according to the present invention will be described with reference to the accompanying drawings.

  In FIG. 1, input signals (object signals) that are R, G, and B signals are input to a depth information generation unit 1 that generates pseudo depth information. The depth information generation unit 1 constitutes a depth information generation apparatus according to an embodiment. The R, G, and B signals are a red signal, a green signal, and a blue signal of the three primary color signals.

  The R, G, and B signals input to the depth information generation unit 1 are image signals representing non-stereoscopic images to which depth information is not given explicitly or implicitly. The depth information generation unit 1 includes a depth estimation unit 11 that generates depth estimation data. The specific configuration and operation of the depth estimation unit 11 will be described in detail later.

  The stereo pair generation unit 2 receives the R, G, B signals that are input signals to the depth information generation unit 1 and the depth estimation data generated by the depth estimation unit 11. The stereo pair generation unit 2 generates a stereo pair of right eye image data and left eye image data.

  The stereo pair generation unit 2 is an example of a multi-viewpoint image data generation unit that generates image data of a plurality of viewpoints based on a non-stereo image signal. The depth information generation unit 1 and the stereo pair generation unit 2 constitute a pseudo stereoscopic image generation apparatus according to an embodiment.

  The stereo pair generation unit 2 uses the input R, G, B signal as it is as the right-eye image data. The stereo pair generation unit 2 generates left eye image data based on the right eye image data (R, G, B signals).

  The pixel shift unit 21 receives R, G, and B signals and depth estimation data. The pixel shift unit 21 shifts the pixels of the R, G, and B signals based on the depth estimation data.

  For example, when moving the viewpoint to the left, the closer the object displayed in front of the screen is, the closer it is to the inside (nose side) of the person viewing the image. Therefore, the pixel shift unit 21 moves the corresponding part of the pixel to the inside (right side) by an amount corresponding to the depth indicated by the depth estimation data. The closer to the back of the screen, the closer it is to the outside of the viewer. Therefore, the pixel shift unit 21 moves the corresponding part of the pixel to the outside (left side) by an amount corresponding to the depth.

  When pixels are shifted by the pixel shift unit 21, so-called occlusion may occur in which no pixels exist in the image. The occlusion compensation unit 22 compensates for the occlusion of the R, G, and B signals output from the pixel shift unit 21.

  The occlusion compensation unit 22 compensates for occlusion by a known method such as interpolation of pixels using pixels around the occlusion.

  The R, G and B signals output from the occlusion compensation unit 22 are input to the post processing unit 23. The post processing unit 23 performs post processing such as smoothing processing on the input R, G, and B signals. By performing the post processing by the post processing unit 23, the occlusion-compensated image becomes smooth and noise is reduced.

  The R, G, and B signals output from the post processing unit 23 become left eye image data. The left eye image output unit 24 outputs left eye image data, and the right eye image output unit 25 outputs right eye image data.

  A stereo image display device 3 is connected to the stereo pair generation unit 2. A stereo pair of the left eye image data output from the left eye image output unit 24 and the right eye image data output from the right eye image output unit 25 is input to the stereo image display device 3. The stereo image display device 3 displays a stereo image (stereoscopic image) based on the input stereo pair.

  The stereo pair generation unit 2 may generate the right-eye image data by shifting the left-eye image data (R, G, and B signals) by using the input R, G, and B signals as they are as the left-eye image data.

  The stereo pair generation unit 2 does not use the input R, G, and B signals as they are for both the right-eye image data and the left-eye image data, but the left and right viewpoint images obtained by shifting the pixels by the pixel shift unit 21. The data may be a stereo pair.

  A specific configuration example and the operation of the depth estimation unit 11 will be described with reference to FIG. The image input unit 111 receives R, G, and B signals. The image input unit 111 supplies only the R signal of the R, G, and B signals to the upper high frequency component evaluation unit 112, the lower high frequency component evaluation unit 113, and the object signal correction unit 118.

  The upper high-frequency component evaluation unit 112 evaluates how much high-frequency components are included in an upper 20% region of each frame. For example, the upper high frequency component evaluation unit 112 divides the upper 20% region into blocks of 8 horizontal pixels and 8 vertical pixels. The upper high frequency component evaluation unit 112 calculates the following expression (1) in each block, and sets the average of the blocks as the upper high frequency component evaluation value top_act. The upper high frequency component evaluation value top_act is input to the synthesis unit 117.

  The lower high-frequency component evaluation unit 113 evaluates how much high-frequency components are included in the lower 20% region of each frame. For example, the lower high frequency component evaluation unit 113 divides the lower 20% region into blocks of 8 horizontal pixels and 8 vertical pixels. The lower high frequency component evaluation unit 113 calculates the above equation (1) in each block, and sets the average of the blocks as the lower high frequency component evaluation value bottom_act. The lower high-frequency component evaluation value bottom_act is input to the synthesis unit 117.

  The frame memories 114 to 116 store a basic depth model that is a basis for generating a pseudo stereoscopic image. The basic depth models stored in the frame memories 114, 115, and 116 are referred to as Type 1, Type 2, and Type 3, respectively. 3, FIG. 4 and FIG. 5 show examples of three-dimensional structures of type 1, type 2 and type 3 of the basic depth model, respectively.

  The basic depth model is a model for determining the parallax value of the entire screen. The basic depth model can be configured by a table or a calculation formula that shifts each pixel on a plane in the pop-out direction or the depth direction, which has a non-planar shape characteristic as shown in FIGS.

  The synthesizing unit 117 performs basic depth models of types 1 to 3 output from the frame memories 114 to 116 at a predetermined synthesis ratio according to the values of the upper high-frequency component evaluation value top_act and the lower high-frequency component evaluation value bottom_act. Synthesize.

  The synthesizer 117 synthesizes the basic depth models of types 1 to 3 according to the synthesis ratio determination condition shown in FIG. The horizontal axis in FIG. 6 is the upper high-frequency component evaluation value top_act, and the vertical axis is the lower high-frequency component evaluation value bottom_act.

  As illustrated in FIG. 6, if the lower high-frequency component evaluation value bottom_act is equal to or less than the predetermined value bms, the synthesis unit 117 uses the type 3 basic depth model regardless of the value of the upper high-frequency component evaluation value top_act. . That is, the composition ratio of types 1 and 2 is set to zero.

  If the lower high frequency component evaluation value bottom_act is greater than the value bms and less than or equal to the predetermined value bml, the synthesis unit 117 synthesizes types 1 to 3 according to the value of the upper high frequency component evaluation value top_act as follows. . If the upper high-frequency component evaluation value top_act is equal to or less than the predetermined value tps, the combining unit 117 sets the combining ratio of type 1 to 0 and combines type 2 and type 3.

  If the upper high-frequency component evaluation value top_act is greater than the value tps and less than or equal to the predetermined value tpl, the synthesizer 117 synthesizes types 1 to 3. If the upper high frequency component evaluation value top_act is larger than the value tpl, the synthesizing unit 117 synthesizes the type 1 and the type 3 by setting the type 2 synthesis ratio to 0.

  If the lower high frequency component evaluation value bottom_act is larger than the value bml, the synthesis unit 117 synthesizes types 1 to 3 according to the value of the upper high frequency component evaluation value top_act as follows. If the upper high frequency component evaluation value top_act is equal to or less than the value tps, the synthesis unit 117 sets the synthesis ratio of types 1 and 3 to 0 and uses the type 2 basic depth model.

  If the upper high-frequency component evaluation value top_act is greater than the value tps and less than or equal to the value tpl, the synthesizer 117 synthesizes type 1 and type 2 with the synthesis ratio of type 3 set to 0. If the upper high frequency component evaluation value top_act is larger than the value tpl, the synthesizing unit 117 sets the synthesis ratio of types 2 and 3 to 0 and uses the type 1 basic depth model.

  Data indicating the parallax value of the entire screen synthesized by the synthesis unit 117 according to the synthesis ratio determination condition shown in FIG. 6 is input to the adder 119.

  A configuration in which a plurality of types of basic depth models are combined according to the upper high-frequency component evaluation value top_act and the lower high-frequency component evaluation value bottom_act is not an essential configuration, but is a preferable configuration. Any configuration that generates data indicating the parallax value of the entire screen using at least one type of basic depth model may be used.

  The object signal correction unit 118 corrects the R signal of the object signal as described later and outputs it as an R ′ signal. The R ′ signal is additional parallax value data added to the parallax value of the entire screen indicated by the data output from the synthesis unit 117.

  One reason for using the R signal is that there is a high probability that the magnitude of the R signal matches the degree of unevenness of the subject in an environment that is close to direct light and under the condition that the brightness in the screen does not differ greatly. . One of the other reasons is that warm colors such as red are progressive colors in chromaticity and are more easily recognized in the depth direction than cold colors.

  A specific configuration example and the operation of the object signal correction unit 118 will be described with reference to FIG. In FIG. 7, an R signal is input to the image feature detection unit 80. The image feature detection unit 80 includes high frequency component frequency detection units 81 and 82.

  As shown in FIG. 8, the high frequency component frequency detection unit 81 detects the frequency of the high frequency component in the region RnA that is the entire screen of one frame. The area RnA may be an effective video period. As shown in FIG. 8, the high frequency component frequency detection unit 82 detects the frequency of the high frequency component in the region RnB in the center of the screen that is narrower than the region RnA. The size of the region RnB may be set as appropriate.

  Specifically, the high frequency component frequency detection units 81 and 82 are configured as shown in FIG. In FIG. 9, an R signal is input to a high pass filter (HPF) 801.

  In the horizontal direction, the HPF 801 calculates the difference between the pixels x_m and x_p indicated by white at positions one pixel apart in the left-right direction with respect to the hatched current pixel as shown in FIG. To do. In the vertical direction, the HPF 801 calculates the difference between white pixels y_m and y_p that are separated by one pixel in the vertical direction with respect to the current pixel that is hatched, as shown in FIG. 10B. To do.

  The HPF 801 includes horizontal and vertical delay units for generating data of pixels x_m, x_p, y_m, and y_p shown in white in FIGS. 10A and 10B, and a subtractor that calculates a difference. . As can be seen from FIGS. 10A and 10B, the difference value for the current pixel with hatching is generated with a delay of one pixel in the horizontal direction and one line in the vertical direction. The difference value in the horizontal direction indicates a high frequency component in the horizontal direction, and the difference value in the vertical direction indicates a high frequency component in the vertical direction.

  The absolute value calculation unit 802 converts the difference value input from the HPF 801 into an absolute value according to the calculation formulas shown in Expressions (2) and (3). Act_H is the horizontal difference absolute value, and Act_V is the vertical difference absolute value. The absolute value calculation unit 802 calculates the high frequency component Act_HV by the calculation formula shown in Formula (4).

Act_H = abs (x_m-x_p) (2)
Act_V = abs (y_m-y_p) (3)
Act_HV = Act_H + Act_V (4)

  The binarization calculation unit 803 outputs “1” if the high-frequency component Act_HV input from the absolute value calculation unit 802 is greater than a predetermined threshold value, and outputs “0” if it is equal to or less than the threshold value. If the output of the binarization operation unit 803 is “1”, it means that the current pixel as the target pixel has a high frequency component.

  In the high frequency component frequency detection unit 81, the accumulation counter 804 accumulates and counts the output of the binarization calculation unit 803 for the period of the region RnA. The count value of the accumulation counter 804 indicates the total number of pixels in the region RnA that have been determined by the binarization calculation unit 803 that the high frequency component Act_HV is greater than the threshold value.

  In the high frequency component frequency detection unit 82, the accumulation counter 804 accumulates and counts the output of the binarization calculation unit 803 for the period of the region RnB. The count value of the accumulation counter 804 indicates the total number of pixels in the region RnB that have been determined by the binarization calculation unit 803 that the high frequency component Act_HV is greater than the threshold value.

  In the high frequency component frequency detection unit 81, the register 805 holds the count value from the accumulation counter 804 and outputs it as high frequency component frequency data Act_Num_A. In the high frequency component frequency detection unit 82, the register 805 holds the count value from the accumulation counter 804 and outputs it as high frequency component frequency data Act_Num_B.

  In the present embodiment, the high frequency component frequency data Act_Num_A and Act_Num_B are generated in each frame, but the high frequency component frequency data Act_Num_A and Act_Num_B may be generated for each of a plurality of frames. The image feature detection unit 80 may generate the high frequency component frequency data Act_Num_A and Act_Num_B for each predetermined unit (time unit) of the screen. However, it is preferable to generate the high-frequency component frequency data Act_Num_A and Act_Num_B for each frame.

  Here, the unit of the screen of image data has been described as a frame, but a field by interlace scanning may be used as a unit. In this case, the frame in this embodiment is read as a field.

  Returning to FIG. 7, the high frequency component frequency data Act_Num_A output from the high frequency component frequency detector 81 is input to the subtractor 83. The high frequency component frequency data Act_Num_B output from the high frequency component frequency detection unit 82 is input to the subtractor 83 and the central part distribution ratio calculation unit 84.

  The subtracter 83 subtracts the high-frequency component frequency data Act_Num_B from the high-frequency component frequency data Act_Num_A according to the calculation formula shown in the equation (5), and out of the region RnA of each frame, excludes the region RnB at the center of the screen. High frequency component frequency data Act_Num_ (AB) is calculated. High frequency component frequency data Act_Num_ (A-B) of the peripheral part is input to the central part distribution ratio calculation unit 84.

  Act_Num_ (A-B) = Act_Num_A-Act_Num_B (5)

  The central part distribution ratio calculation unit 84 has the subject of interest in the center of the screen based on the high frequency component frequency data Act_Num_B of the region RnB in the center of the screen and the high frequency component frequency data Act_Num_ (AB) of the peripheral part. Center part distribution ratio data Act_Center indicating the degree is calculated.

  In the present embodiment, the portion where the high-frequency component is detected in the image feature detection unit 80 is not a flat image such as a background but an image having a predetermined pattern (texture), and thus a large amount of high-frequency component is detected. The part is the subject of interest.

  As shown in FIG. 11, the central part distribution ratio calculation unit 84 includes a subtracter 841 and a lower limiter 842. The subtractor 841 subtracts the high-frequency component frequency data Act_Num_ (A-B) from the high-frequency component frequency data Act_Num_B according to the calculation formula shown in Formula (6) to calculate the central portion distribution ratio data Act_Center.

  Act_Center = Act_Num_B-Act_Num_ (A-B) (6)

  The lower limiter 842 sets the center part distribution ratio data Act_Center output from the subtractor 841 to 0 if the value is less than 0. From the lower limiter 842, center part distribution ratio data Act_Center having a predetermined value of 0 or more is output. The central portion distribution ratio data Act_Center shows a larger value as the degree to which a subject having a high frequency component is located in the central portion than the periphery of the screen is larger.

  The central portion distribution ratio data Act_Center is input to the leak type integration circuit 85. As shown in FIG. 12, the leaky integration circuit 85 includes an adder 851, a register 852, and multipliers 853 and 854. The leak type integration circuit 85 performs leak type integration processing in the time direction on the central portion distribution ratio data Act_Center with the configuration shown in FIG.

  The adder 851 adds the input center distribution ratio data Act_Center and the output of the multiplier 853. The register 852 holds the output of the adder 851. Multiplier 853 multiplies the output of register 852 by 15/16 and outputs the result to adder 851. The multiplier 854 multiplies the output of the register 852 by 1/16, and outputs center part distribution ratio data Act_Center_Leak that has been subjected to leaky integration processing.

  By applying leak type integration processing to the central portion distribution ratio data Act_Center by the leak type integration circuit 85, the R ′ signal generated by the object signal correction unit 118 does not change abruptly due to image changes. Accordingly, the stereo image generated by the stereo pair generated by the stereo pair generation unit 2 changes gradually, and a more natural image quality can be obtained.

  Although it is not essential to provide the leak-type integrating circuit 85, it is preferable to provide it.

  The central portion distribution ratio data Act_Center_Leak output from the leak type integration circuit 85 is input to the gain calculation unit 86. The gain calculation unit 86 also receives horizontal width data H_width and vertical width data V_width of image data input to the depth information generation unit 1 as image information.

  The gain calculation unit 86 calculates the gain according to the position of the screen and according to the value of the central portion distribution ratio data Act_Center_Leak. The setting of the gain according to the screen position will be described with reference to FIG. FIG. 13 shows gain characteristics according to the horizontal position of the screen. In FIGS. 13A and 13B, H_Num is a horizontal pixel number.

  FIG. 13A shows a conventional gain characteristic for comparison. The left end of the horizontal axis is the left end of the screen. Conventionally, the gain is constant from the left end in the horizontal direction to the right end indicated by H_width.

  On the other hand, in this embodiment, as shown in FIG. 13B, a constant gain 1 state indicated by a thick broken line from the left end in the horizontal direction to the right end indicated by H_width, and p1 at the center of the screen A state in which a gain exceeding 1 indicated by a solid line in the range of ~ p2 is obtained. The limit value lim of the gain in the range of p1 to p2 in the center of the screen is variable in the range of 1 to r2 according to the value of the center distribution ratio data Act_Center_Leak. In other words, the gain in the range of p1 to p2 in the center of the screen is in a state indicated by a maximum thick dotted line.

  When the gain characteristic shown in (b) of FIG. 13 is expressed by a calculation formula, Formula (7) is obtained.

If (H_Num <p1) Gain = (lim-1) × H_Num / p1 + 1;
If (H_Num <p2) Gain = lim;
If (H_Num <H_width) Gain = (lim-1) × (H_width-H_Num) / (H_width-p2) +1;
... (7)

  The positions of p1 and p2 in the center of the screen may be set as appropriate. The left end portion of the region RnB may be p1, and the right end portion may be p2. That is, the screen center portion here may or may not be the same as the screen center region RnB shown in FIG.

  FIG. 14 shows the relationship between the value of the central portion distribution ratio data Act_Center_Leak and the gain limit value lim. As shown in FIG. 14, the limit value lim is 1 if the value of the central portion distribution ratio data Act_Center_Leak is 0. As the value of the central distribution ratio data Act_Center_Leak increases, the limit value lim also increases sequentially.

  When the value of the central portion distribution ratio data Act_Center_Leak is r1, the limit value lim is the maximum r2. When the value of the central portion distribution ratio data Act_Center_Leak is equal to or greater than r1, the limit value lim is constant at the maximum r2.

  In FIG. 13, the gain is set to be different according to the position in the horizontal direction of the screen, but may be set to be different according to the position in the vertical direction of the screen. Also in the case where the gain is varied according to the position in the vertical direction, the gain in the central portion in the vertical direction may be increased as the value of the central portion distribution ratio data Act_Center_Leak increases as in FIG. The gain may be set to be different depending on the position in both the horizontal direction and the vertical direction of the screen.

  Returning to FIG. 7, the multiplier 87 receives the R signal and the gain calculated by the gain calculation unit 86. The multiplier 87 multiplies the R signal by the gain input from the gain calculator 86 and outputs the result as an R ′ signal. The R ′ signal is a corrected object signal obtained by correcting the R signal that is an object signal.

  As can be seen from the above, the object signal correction unit 118 (in particular, the gain calculation unit 86 and the multiplier 87) corresponds to the size of the central part distribution ratio data Act_Center (Act_Center_Leak) calculated by the central part distribution ratio calculation unit 84. Thus, the correction is performed so that the amplitude of the non-stereoscopic image signal (R signal) in the central portion of the screen is increased and the corrected non-stereoscopic image signal (R ′ signal) is output.

  The R ′ signal generated as described above is input to the adder 119 shown in FIG. The adder 119 adds the data indicating the parallax value of the entire screen output from the combining unit 117 and the R ′ signal, and outputs the result as depth estimation data.

  The effect by this embodiment is demonstrated using FIG. In FIG. 15, (a) is a pseudo stereoscopic image obtained by a conventional process, and (b) is a pseudo stereoscopic image obtained by the process of the present embodiment. The pseudo stereoscopic image in FIG. 15A is an image obtained by adding the R signal as it is to the data indicating the parallax value of the entire screen described in Patent Document 1.

  (A) and (b) of FIG. 15 express the shift amount 0 to 255 in a gray scale from black to white. The shift amount 0 is a state where the parallax is 0, and the shift amount 255 is a state where the parallax in the pop-out direction is the maximum.

  FIGS. 15A and 15B are examples of images in which the subject of interest is concentrated in the center of the screen. Conventionally, as shown in FIG. 15A, the degree of solidity of a subject of interest (particularly a flower image) is limited.

  On the other hand, according to the present embodiment, as shown in FIG. 15B, the shift amount of the subject of interest (particularly the flower image) increases and appears to pop out, so that the degree of the solid is large. It turns out that it is improving.

  As can be seen from the gain characteristics shown in FIG. 13B, the gain increases sequentially from the periphery to the center of the screen, and the gain decreases sequentially from the center to the periphery. The depth of the changes gently. Therefore, there is no sense of incongruity. According to the present embodiment, the depth of an object (subject) changes gradually toward the center of the screen, and a three-dimensional effect that is rounded can be obtained.

  According to the present embodiment, the center of the screen can be popped out according to the degree to which the subject of interest is present at the center of the screen, and the appropriate amount of popping out can be achieved according to the pattern of the image.

  In the present embodiment described above, a configuration in which the stereo pair generation unit 2 generates two-viewpoint image data and the stereo image display device 3 displays a stereoscopic image based on the two-viewpoint image data is shown. A stereoscopic image may be displayed by a so-called multi-viewpoint image display device that generates image data of three or more viewpoints and displays a stereoscopic image based on the image data of three or more viewpoints.

  The stereo image display device 3 includes any of a projection system using polarized glasses, a projection system or a display system combining time-division display and liquid crystal shutter glasses, a lenticular stereo display, an anaglyph stereo display, a head mounted display, etc. But you can.

  The projector system can be composed of two projectors that project respective image data of a stereo pair.

  The depth information generation apparatus or pseudo-stereoscopic image generation apparatus of the present embodiment may be configured by hardware, or may be configured by software by a computer program.

  When a computer performs the same processing as the above-described depth information generation apparatus and method and similar stereoscopic image generation apparatus and method, the depth information generation program and the pseudo stereoscopic image generation program are read from the recording medium and installed in the computer. Alternatively, the depth information generation program and the pseudo stereoscopic image generation program may be installed in the computer via a network.

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention.

1 Depth information generator (depth information generator)
2 Stereo Pair Generation Unit 3 Stereo Image Display Device 11 Depth Estimation Unit 21 Pixel Shift Unit 84 Center Part Distribution Ratio Calculation Unit 86 Gain Calculation Unit (Amplitude Correction Unit)
87 Multiplier (Amplitude Correction Unit)
112 Upper high-frequency component evaluation unit 113 Lower high-frequency component evaluation unit 114 to 116 Frame memory (parallax data generation unit)
117 Synthesizer (parallax data generator)
118 Object Signal Correction Unit 119 Adder

Claims (8)

Using a basic depth model for determining the parallax value of the entire screen, a parallax data generating unit for generating parallax data indicating the parallax value of each pixel in the screen;
A central part distribution ratio calculating unit that calculates central part distribution ratio data indicating the degree to which the subject of interest included in the non-stereoscopic image signal is concentrated and distributed in the first central part excluding the peripheral part of the screen;
A corrected non-stereo image is corrected by increasing the amplitude of the non-stereo image signal in a second central portion that is the same as or different from the first central portion of the screen according to the size of the central portion distribution ratio data. An amplitude correction unit for outputting a signal;
An adder that adds the parallax data generated by the parallax data generation unit and the corrected non-stereoscopic image signal output from the amplitude correction unit to generate depth estimation data for each pixel in the screen;
A depth information generating apparatus comprising:   The parallax data generation unit includes a synthesis unit that synthesizes parallax data indicating parallax values of the entire screen of each of a plurality of basic depth models based on characteristics of the non-stereoscopic image signal. Depth information generating apparatus described. An upper high-frequency component evaluation unit that generates an upper high-frequency component evaluation value by evaluating the amount of a high-frequency component included in a predetermined region on the upper part of the screen;
A lower high-frequency component evaluation unit that generates a lower high-frequency component evaluation value by evaluating the amount of a high-frequency component included in a predetermined lower area on the screen;
Further comprising
The combining unit combines disparity data indicating disparity values of the entire screen of each of the plurality of basic depth models according to the upper high-frequency component evaluation value and the lower high-frequency component evaluation value. The depth information generation device according to claim 2.   The central part distribution ratio calculating unit uses high frequency component frequency data indicating the frequency of high frequency components included in the peripheral part and high frequency component frequency data indicating the frequency of high frequency components included in the first central part. The depth information generating apparatus according to claim 1, wherein the central part distribution ratio data is calculated.   The depth information generating apparatus according to claim 1, wherein the non-stereoscopic image signal is a red signal of three primary color signals. The depth information generation device according to any one of claims 1 to 5,
A multi-viewpoint image data generating unit that generates image data of a plurality of viewpoints based on the depth estimation data;
A pseudo-stereoscopic image generation apparatus comprising: Using a basic depth model for determining the parallax value of the entire screen, generating parallax data indicating the parallax value of each pixel in the screen,
Calculating central portion distribution ratio data indicating the degree to which the subject of interest included in the non-stereoscopic image signal is concentrated and distributed in the first central portion excluding the peripheral portion of the screen;
A corrected non-stereo image is corrected by increasing the amplitude of the non-stereo image signal in a second central portion that is the same as or different from the first central portion of the screen according to the size of the central portion distribution ratio data. Generate a signal,
A depth information generation method characterized by adding the parallax data and the corrected non-stereoscopic image signal to generate depth estimation data for each pixel in the screen. On the computer,
Generating disparity data indicating the disparity value of each pixel in the screen using a basic depth model for determining the disparity value of the entire screen;
Calculating central part distribution ratio data indicating a degree to which a subject of interest included in the non-stereoscopic image signal is concentrated and distributed in the first central part excluding the peripheral part of the screen;
A corrected non-stereo image is corrected by increasing the amplitude of the non-stereo image signal in a second central portion that is the same as or different from the first central portion of the screen according to the size of the central portion distribution ratio data. Generating a signal;
Adding the parallax data and the corrected non-stereoscopic image signal to generate depth estimation data for each pixel in the screen;
A depth information generation program characterized in that