JP2012156788A - Depth estimation data generation device, depth estimation data generation program, and pseudo stereoscopic image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a depth estimation data generation device capable of obtaining a sufficient stereoscopic effect in every scene from a monotone image to a vivid image.SOLUTION: A depth estimation data generation device comprises: a saturation average value generation unit 12 that generates saturation average value data from color difference signal components; a luminance signal change value calculation unit 132 that calculates a difference between a luminance signal component and a predetermined value to calculate a luminance signal change value; an RB composite value calculation unit 133 that calculates an RB composite value obtained by performing composition so that an R component is positioned in front of a pseudo stereoscopic image and a B component is positioned behind the pseudo stereoscopic image; a gain generation unit 134 that generates a luminance signal gain for correcting the luminance signal change value and an RB gain for correcting the RB composite value based on the saturation average value data; and a depth estimation data calculation unit 135 that combines a calculation result of the luminance signal change value and the luminance signal gain and a calculation result of the RB composite value and the RB gain to calculate depth estimation data.

Description

  The present invention is for generating a pseudo three-dimensional image from a normal still image or moving image, that is, an image (non-stereo image) in which depth information is not given explicitly or implicitly such as a stereo image. The present invention relates to a depth estimation data generation device, a depth estimation data generation program, and a pseudo stereoscopic image display device.

  In a stereoscopic display system, in order to enable viewing of a non-stereo image by pseudo-stereoscopic view, a normal still image or moving image, that is, depth information for representing a stereoscopic image is expressed explicitly or implicitly like a stereo image. Also, a process of generating a pseudo three-dimensional image from an image (non-stereo image) that has not been given is performed.

  As an example of such a technique, there is a pseudo-stereoscopic image generation apparatus disclosed in Patent Document 1, for example. In the pseudo-stereoscopic image generation device of Patent Document 1, in order to determine a scene structure that is as close to reality as possible, a plurality of basic depth models that indicate depth values for each of a plurality of basic scene structures are prepared, A synthesis ratio is determined according to the high-frequency component evaluation values from the upper and lower high-frequency component evaluation units, and a plurality of basic depth models are synthesized according to the synthesis ratio. Then, the synthesized basic depth model and the R signal of the non-stereo image are superimposed by an adder to generate final depth estimation data, and processing based on this depth estimation data is performed on the non-stereo image. Another viewpoint image that gives a three-dimensional feeling is generated.

JP 2006-185033 A

  However, in the method disclosed in Patent Document 1 described above, the signal to be superimposed on the basic depth model is not limited to the R signal, but may be a B signal or a signal using both the R signal and the B signal. However, since this method is a method of selecting a signal to be superimposed, for example, when the R signal and the B signal are used in combination, if the image is a colorful image containing a lot of warm colors and cold colors, the stereoscopic effect can be greatly produced. However, in the case of a monotone image, there is a problem that the stereoscopic effect is reduced. Therefore, the technique disclosed in Patent Document 1 has a problem that a sufficient stereoscopic effect cannot be obtained for all scenes.

 Therefore, the present invention has been proposed in view of the above-described circumstances, and a depth estimation data generation apparatus and depth estimation data that can obtain a sufficient stereoscopic effect for any scene from a monotone image to a colorful image. It is an object to provide a generation program and a pseudo stereoscopic image display device.

  In order to achieve the above-described object, the depth estimation data generation device according to the present invention calculates saturation from a color difference signal component of an input video signal, generates an average value of the saturation, and generates saturation average value data. A saturation average value generation unit that calculates a difference between a luminance signal component of the input video signal and a predetermined value that is set in advance, and calculates a luminance signal change value; and An RB composite value calculating unit that calculates an RB composite value so that an R component is on the near side of the pseudo stereoscopic image and a B component of the input video signal is on the deep side of the pseudo stereoscopic image; and the luminance signal change A gain generation unit that generates a luminance signal gain for correcting the value and an RB gain for correcting the RB composite value based on the saturation average value data; the luminance signal change value; and the luminance signal gain; The calculation result and A depth estimation data calculation unit configured to calculate depth estimation data for generating a signal indicating a pseudo-stereoscopic image from the input video signal by synthesizing a calculation result of the B synthesis value and the RB gain; It is characterized by.

  The depth estimation data generation program according to the present invention calculates a saturation from a color difference signal component of an input video signal, obtains an average value of the saturation, and generates a saturation average value data. A luminance signal change value calculation step of calculating a luminance signal change value by obtaining a difference between a luminance signal component of the input video signal and a predetermined value set in advance; and an R component of the input video signal is the pseudo stereoscopic image An RB composite value calculating step for calculating an RB composite value that is combined so that the B component of the input video signal is on the back side of the pseudo-stereoscopic image, and a brightness for correcting the brightness signal change value A gain generation step of generating a signal gain and an RB gain for correcting the RB composite value based on the saturation average value data, and a calculation result of the luminance signal change value and the luminance signal gain A depth estimation data calculation step for calculating depth estimation data for generating a signal indicating a pseudo stereoscopic image from the input video signal by combining the calculation result of the RB synthesis value and the RB gain is executed on a computer. It is characterized by making it.

  Furthermore, the pseudo stereoscopic image display device according to the present invention generates the depth estimation data using the depth estimation data generation device according to claim 1, the depth estimation data, and the input video signal. A stereo pair generating device for generating another viewpoint image signal for displaying a pseudo-stereoscopic image by shifting the texture of the signal by an amount corresponding to the depth of the corresponding portion, the different viewpoint image signal, and the input video signal, And a stereo display device that displays a pseudo-stereoscopic image using the.

  According to the depth estimation data generation device and the depth estimation data generation program according to the present invention, the depth estimated from the luminance signal and the depth estimated from the R signal and the B signal are synthesized according to the average value of the saturation. Since depth estimation data is generated, a sufficient stereoscopic effect can be obtained for any scene from a monotone image to a colorful image.

  Further, according to the pseudo-stereoscopic image display device according to the present invention, the depth estimated from the luminance signal and the depth estimated from the R signal and the B signal are synthesized according to the average value of the saturation, and this synthesized Since the pseudo stereoscopic image is generated and displayed using the depth estimation data, it is possible to display an image having a sufficient stereoscopic effect for any scene from a monotone image to a colorful image.

It is a block diagram which shows the structure of the pseudo-stereoscopic image display apparatus which concerns on one Embodiment to which this invention is applied. It is a block diagram which shows the structure of the saturation average value generation part of the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a block diagram which shows the structure of the depth estimation part of the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a block diagram which shows the structure of the leak type | mold integration part of the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a flowchart which shows the procedure of the production | generation process of the saturation average value data by the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a flowchart which shows the procedure of the production | generation process of the depth estimation data by the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the calculation method of the luminance change value by the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the calculation method of the RB synthetic | combination value by the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the calculation method of the luminance signal gain and RB gain by the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied. It is a figure for demonstrating the effect by the depth estimation data generation apparatus which concerns on one Embodiment to which this invention is applied.

  Hereinafter, an embodiment to which the present invention is applied will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of a pseudo stereoscopic image display device according to the present embodiment.

  As shown in FIG. 1, the pseudo-stereoscopic image display device according to the present embodiment includes a depth estimation data generation device 1, a stereo pair generation device 2, and a stereo display device 3, and the depth information is explicitly shown. When a non-stereo image that is not given implicitly, such as a stereo image, is input, a left-eye image signal and a right-eye image signal are generated from the non-stereo image composed of a plurality of time-sequential images. And a pseudo stereoscopic image is displayed on the stereo display device 3 using the left eye image signal and the right eye image signal.

  Hereinafter, each of the depth estimation data generation device 1, the stereo pair generation device 2, and the stereo display device 3 constituting the pseudo stereoscopic image display device will be described.

[Configuration of depth estimation data generator]
As shown in FIG. 1, a depth estimation data generation apparatus 1 according to the present embodiment includes a matrix conversion unit 11 that converts an input RGB signal into a luminance signal and a color difference signal, and chroma from the color difference signal component of the input video signal. And a saturation average value generation unit 12 that generates saturation average value data and a depth estimation unit 13 that generates depth estimation data. The depth estimation data is data for generating a pseudo three-dimensional image from an image to which depth information is not given (also referred to as a non-stereo image).

  Here, the configuration of the saturation average value generation unit 12 will be described with reference to FIG. As shown in FIG. 2, the saturation average value generation unit 12 is calculated for each pixel, and a saturation calculation unit 121 that calculates the saturation (VC) of each pixel from the color difference signals BY and RY. An integration unit 122 that integrates the saturation values to calculate a saturation integrated value (VC_total) for one frame, and an average value of saturations by dividing the saturation integrated value (VC_total) by the total number of pixels in one frame ( VC_ave) and a register 124 that holds the calculated average value of saturation (VC_ave). In this embodiment, the saturation average value data is generated for each field or frame. However, the saturation average value data may be generated for a plurality of fields or a plurality of frames. Saturation average value data may be generated for each unit. However, it is desirable to generate saturation average value data for each field or frame.

  Next, the configuration of the depth estimation unit 13 will be described with reference to FIG. As shown in FIG. 3, the depth estimation unit 13 obtains a difference between a luminance type signal 131 and a predetermined value set in advance by using a leakage type integration unit 131 that performs a leakage type integration process on the saturation average value data. A luminance signal change value calculation unit 132 that calculates a change value (Y_out) and an RB composite value (RB_out) that is synthesized so that the R signal is on the near side of the pseudo stereoscopic image and the B signal is on the far side of the pseudo stereoscopic image. Saturation average value data (VC_ave_out) is calculated from the RB composite value calculation unit 133 to be calculated, the luminance signal gain Gy for correcting the luminance signal change value (Y_out), and the RB gain Grb for correcting the RB composite value (RB_out). ) Based on the gain generation unit 134, the calculation result of the luminance signal change value (Y_out) and the luminance signal gain Gy, the calculation result of the RB composite value (RB_out) and the RB gain Grb, and the depth estimation. Day And a depth estimation data calculating unit 135 for calculating a.

  Here, the circuit configuration of the leaky integrator 131 will be described with reference to FIG. As shown in FIG. 4, the leaky integration unit 131 includes an adder 41, a register 42, and multipliers 43 and 44. When the saturation average value data (VC_ave) is input, leak type integration processing is executed in the time direction. The leak type integration process will be described later.

[Procedure for generating saturation value data]
Next, the procedure of the saturation average value data generation process by the depth estimation data generation apparatus 1 according to the present embodiment will be described with reference to the flowchart of FIG.

  As shown in FIG. 5, in step S101, the matrix conversion unit 11 first converts the input video signal (RGB) into the luminance signal Y and the color difference signals U (BY) and V (RY) according to the equation (1). Convert to

Y = 0.299 × R + 0.587 × G + 0.114 × B
U = −0.169 × R −0.331 × G + 0.500 × B (1)
V = 0.500 × R − 0.419 × G − 0.081 × B
The converted color difference signals BY and RY are supplied to the saturation average value generation unit 12, and the luminance signal Y is supplied to the depth estimation unit 13.

Next, in step S102, when the color difference signals RY and BY are input to the saturation average value generation unit 12, the saturation (VC) of each pixel is calculated by the calculation formula shown in Expression (2). .

  When the saturation (VC) is calculated for each pixel, the saturation average value generation unit 12 integrates the calculated saturation values in step S103, and the integrated value of saturation for one frame (VC_total). Is calculated. In step S104, the calculated integrated value of saturation is divided by the total number of pixels in one frame to calculate an average value of saturation (VC_ave).

VC_ave = VC_total / total number of pixels (3)
When the saturation average value (VC_ave) is generated for each frame in this manner, the saturation value is stored in the register 124 and then supplied to the depth estimation unit 13 as saturation average value data. The depth estimation data generation device according to the present embodiment The generation processing of the saturation average value data by 1 ends.

[Depth estimation data generation processing procedure]
Next, a procedure of depth estimation data generation processing by the depth estimation data generation apparatus 1 according to this embodiment will be described with reference to the flowchart of FIG.

  As shown in FIG. 6, in step S201, the leak type integration unit 131 performs leak type integration processing on the saturation average value data (VC_ave) supplied from the saturation average value generation unit 12, and after leak type integration is performed. The saturation average value data (VC_ave_out) is output.

  Here, the leakage type integration process will be described with reference to FIG. As shown in FIG. 4, when VC_ave_in is input as input data, the data output from the register 42 is multiplied by 15/16 by the multiplier 43, and the value is added to the input data by the adder 41. The added result is stored in the register 42 and used in the same manner when the next VC_ave_in is input. Next, the data output from the register 42 is multiplied by 1/16 by the multiplier 44 and the final output data VC_ave_out is output. In this way, by performing leak type integration processing in the leak type integration unit 131, it is possible to provide a more natural image quality by gradually changing the image.

  Next, in step S202, a difference between the luminance signal Y supplied from the matrix converter 11 and a predetermined value set in advance is obtained to calculate a luminance signal change value (Y_out). Here, the process of calculating the luminance signal change value will be described with reference to FIG. As shown in FIG. 7, the horizontal axis in FIG. 7 is the input luminance signal Y (0 to 255), and the vertical axis is the output luminance signal change value (Y_out). In the present embodiment, 128 is subtracted from the luminance signal Y by the calculation formula shown in Formula (4).

Y_out = Y-128 (4)
The reason why the predetermined value is subtracted from the luminance signal Y is to realize the following two features.

1) White is an expanded color, and has a feature that depth is recognized in front of black.

2) Black is a contraction color and has a feature that the depth is recognized deeper than white.

  That is, by subtracting 128 from the luminance signal Y, black becomes a negative value, so that it is arranged on the back side in the pseudo-stereoscopic image and white is a positive value, so it is arranged on the near side in the pseudo-stereoscopic image. . Therefore, the above two features can be realized by calculating the luminance change value (Y_out). However, in this embodiment, the subtraction process is performed on the luminance signal Y, but the process may be performed on the G signal.

  Next, in step S203, the R and B signals of the input video signal are combined so that the R signal is on the near side of the pseudo stereoscopic image and the B signal is on the far side of the pseudo stereoscopic image. The composite value is calculated. Here, the calculation process of the RB composite value of the RB composite value calculation unit 133 will be described with reference to FIG. As shown in FIG. 8, the X axis is an input B signal (0 to 255), the Y axis is an input R signal (0 to 255), and the Z axis is a calculation formula shown in Expression (5). RB_rate calculated by the following.

RB_rate = (0.5 × R) + {0.5 × (255−B)} (5)
Further, the RB composite value (RB_out) is calculated from RB_rate by the calculation formula shown in Formula (6).

RB_out = RB_rate-128 (6)
By calculating the RB composite value (RB_out) in this way, it is possible to arrange a strong red portion on the near side of the pseudo stereoscopic image and a strong blue portion on the far side of the pseudo stereoscopic image. Further, since the output values of black and white (gray scale) are 0, the depth does not change. Therefore, a stereoscopic effect cannot be obtained with a monotone image.

  The reason for calculating the RB composite value in this way is because of the following three features.

1) The reason why the portion where the R signal is strong is arranged on the near side is that the probability that the magnitude of the R signal matches the unevenness of the subject in an environment close to direct light and under the condition that the brightness of the texture does not vary greatly. This is because there is a rule of thumb that is high.

2) The reason why the portion where the B signal is strong is arranged on the back side is that there is a general theory of the physical law (air perspective method) that the farther away due to the scattering of light rays, the closer it is to appear blue.

3) The warm color system is a forward color in chromatics, and has the feature that the depth is recognized in front of the cold color system (backward color). The cold color system (backward color) is recognized deeper than the warm color system. There is a feature that is.

  Therefore, by calculating the RB composite value, it is possible to give a stereoscopic effect to the pseudo stereoscopic image by these three features.

  Next, in step S204, the luminance signal gain Gy for correcting the luminance signal change value (Y_out) and the RB gain Grb for correcting the RB composite value (RB_out) are based on the saturation average value data (VC_ave_out). To generate.

  Here, a method of generating the luminance signal gain Gy and the RB gain Grb in the gain generation unit 134 will be described with reference to FIG. FIG. 9A is a diagram for explaining a method for generating the luminance signal gain Gy, and FIG. 9B is a diagram for explaining a method for generating the RB gain Grb.

  As shown in FIG. 9A, the horizontal axis represents the saturation average value (VC_ave_out), and the vertical axis represents the luminance signal gain Gy. As the saturation average value (VC_ave_out) increases, the luminance signal gain Gy decreases. It has the characteristic which becomes.

  On the other hand, in FIG. 9B, the horizontal axis represents the saturation average value (VC_ave_out) and the vertical axis represents the RB gain Grb, and the characteristic that the RB gain Grb increases as the saturation average value (VC_ave_out) increases. Is given. Then, the characteristics of the luminance signal gain Gy and the RB gain Grb are determined based on the calculation formula shown in Expression (7).

Gy + Grb = 1.0 (7)
However, this is merely an example, and the gain characteristic is not limited to the equation (7).

  When the luminance signal gain Gy and the RB gain Grb are determined in this way, next, in step S205, depth estimation data (Depth) is calculated by the calculation formula shown in formula (8).

Depth = (Y_out x Gy) + (RB_out x Grb) (8)
In this depth estimation data (Depth), the luminance signal change value (Y_out) is multiplied by the luminance signal gain Gy, the RB composite value (RB_out) is multiplied by the RB gain Grb, and then added.

  Here, since the luminance signal gain Gy and the RB gain Grb are generated based on the saturation average value data (VC_ave_out), in the depth estimation data of Expression (8), the depth estimation estimated from the luminance signal (Y). A synthesis ratio between the data and the depth estimation data estimated from the color signals (R, B) is determined based on the saturation average value data (VC_ave_out).

  When the depth estimation data is generated in this way, the depth estimation data generation processing by the depth estimation data generation apparatus 1 according to the present embodiment ends.

[Effect of depth estimation data generator]
Next, the effect of the depth estimation data generation device according to the present embodiment will be described with reference to FIG.

  As shown in FIG. 10, a case where a sky (monotone) image as shown in FIG. 10A is input as an input image will be described. FIG. 10B shows the relationship between the luminance level of the input image and the appearance frequency of the pixels, and it can be seen that the image is distributed in the intermediate luminance region.

  Here, a shift amount image subjected to the conventional processing is shown in FIG. This shift amount image is expressed in gray scale, black to white is displayed in 256 steps, the black side is moved in the depth (concave) direction, and the white side is moved in the protruding (convex) direction. To do. FIG. 10D shows the shift amount on the horizontal axis and the pixel frequency on the vertical axis, and the shift amount is concentrated at 128. That is, the parallax is 0, and as shown in FIG. 10C, the output image is a flat stereoscopic effect that cannot be obtained.

  On the other hand, in the shift amount image subjected to the processing according to the present invention shown in FIG. 10E, the saturation average value of the image is detected to determine the combination ratio of the luminance signal change value and the RB combination value. Therefore, since the image of FIG. 10E is a monotone image, the saturation average value is 0, the luminance signal gain Gy = 1, and the RB gain Grb = 0, and the ratio of the depth value estimated from the luminance signal is 100. % Depth estimation data can be automatically generated. As a result, as can be seen from the wide distribution shown in FIG. 10F, it is possible to generate an output image having a large difference in shading, that is, a large parallax and a sufficient stereoscopic effect.

  As described above, conventionally, a stereoscopic effect cannot be obtained with respect to a monotone image. However, according to the depth estimation data generation device 1 according to the present embodiment, a sufficient stereoscopic effect can be obtained even with a monotone image. Thus, a sufficient stereoscopic effect can be obtained for any scene from a monotone image to a colorful image.

[Configuration of Stereo Pair Generation Device and Stereo Display Device]
As described above, when the depth estimation data is generated by the depth estimation data generation device 1, it is possible to generate an image of another viewpoint based on the depth estimation data. Here, with reference to FIG. 1, the structure of the stereo pair production | generation apparatus 2 and the stereo display apparatus 3 which concern on this embodiment is demonstrated.

  As shown in FIG. 1, the stereo pair generation device 2 includes a texture shift unit 21 that shifts the texture of an input non-stereo image video signal based on depth estimation data, an occlusion compensation unit 22 that compensates for occlusion, A post-processing unit 23 that performs post-processing, a left-eye image signal generation unit 24, and a right-eye image signal generation unit 25 are provided.

  Then, the left-eye image signal and the right-eye image signal generated by the stereo pair generation device 2 are input to the stereo display device 3 and displayed as a pseudo stereoscopic image on the stereo display device 3.

[Example of operation of stereo pair generation device 2]
The stereo pair generation device 2 generates another viewpoint image based on the depth estimation data generated by the depth estimation data generation device 1 and the input video signal of the non-stereo image. For example, when moving the viewpoint to the left, for objects that are displayed in front of the screen, the closer the object, the closer to the viewer, the inner (nose side) of the image is seen. Just move. On the other hand, as for the object to be displayed at the back of the screen, the closer the object is to the outside of the person viewing the image, the texture of the corresponding part is moved to the left by an amount corresponding to the depth. A stereo pair is configured by using the image subjected to such processing as the left-eye image and the input non-stereo image as the right-eye image.

  That is, in the stereo pair generating apparatus 2 according to the present embodiment shown in FIG. 1, the texture shift unit 21 is firstly based on the depth estimation data output from the depth estimation unit 13, in order from the smallest value, that is, the one located in the back. The texture of the portion of the video signal corresponding to the value is shifted, for example, to the right by the shift amount S pixels indicated by the depth signal. When the depth signal is negative, the shift is made to the left by the shift amount S pixels indicated by the depth signal.

  The texture shift operation of the video signal based on the shift amount S in the texture shift unit 21 corresponds to the texture shift of the non-stereo image. In other words, this is a process of moving each pixel of the non-stereoscopic image to the left and right according to the value of the shift amount S that is the value of the depth estimation data.

  Here, there is a case where a portion where no texture exists, that is, an occlusion occurs due to a change in the positional relationship in the image due to the shift. For such a part, the occlusion compensation unit 22 fills the video signal with a video signal around the corresponding part of the video signal, or a known document (Kunio Yamada, Kenji Mochizuki, Kiyoharu Aizawa, Takahiro Saito: “Division by region competition method” Occlusion compensation based on the statistics of the texture of the recorded image ”, the Journal of Emotionology, Vol.56, No.5, pp.863-866 (2002.5)).

  The image subjected to occlusion compensation by the occlusion compensation unit 22 is subjected to post processing such as smoothing by the post processing unit 23 to reduce noise generated in the previous processing, and the left eye image signal generation unit 24 performs the left eye image generation. Output as a signal. On the other hand, the right eye image signal generation unit 25 outputs the input video signal as a right eye image signal.

  In this way, the stereo pair generation device 2 uses the depth estimation data generated by the depth estimation data generation device 1 and the input non-stereo image video signal to generate a left-eye image signal and a right-eye image signal. Stereo pairs can be generated. These left-eye image signal and right-eye image signal are output to the stereo display device 3, and the stereo display device 3 can display a pseudo-stereoscopic image with an improved pseudo-stereoscopic effect as a whole screen by a stereo pair.

  In addition, regarding the stereo pair, a stereo pair in which the left-eye image signal is an original image and the right-eye image signal is a different viewpoint image may be configured by flipping left and right. In the above processing, a stereo pair is formed in which either the right-eye image signal or the left-eye image signal is a video signal, and the other is a different viewpoint image signal that is generated. That is, it is also possible to configure a stereo pair using another viewpoint image signal moved to the right and another viewpoint image signal moved to the left.

[Example of operation of stereo display device 3]
The stereo display device 3 according to the present embodiment shown in FIG. 1 includes, for example, a projection system using polarized glasses, or a projection system or display system combining time-division display and liquid crystal shutter glasses, a lenticular stereo display, and an anaglyph method. A stereo display, a head-mounted display, and a projector system using two projectors corresponding to each image of a stereo image, and the left eye image signal and the right eye image signal generated by the stereo pair generation device 2 are input to the display or the like A pseudo stereoscopic image is displayed.

  In the description of the above embodiment, the stereo pair generation device 2 has been described with respect to the two viewpoints of the left-eye image signal and the right-eye image signal, but the present invention is not limited to this. In other words, when displaying on a display device capable of displaying two or more viewpoints, it is of course possible to generate a number of different viewpoint images corresponding to the number of viewpoints.

  In addition, as described above, it is possible to construct a multi-viewpoint stereoscopic image display system using a display device that can display two or more viewpoints. In this stereoscopic display system, a configuration equipped with audio output may be considered. In this case, for image content that does not have audio information, such as still images, an aspect in which an environmental sound suitable for an image is added can be considered.

  Furthermore, in this embodiment, the case where the depth estimation data generation device 1, the stereo pair generation device 2, and the stereo display device 3 are configured by hardware as shown in FIG. 1 has been described. The depth estimation data generation device 1, the stereo pair generation device 2, and the stereo display device are not limited to those configured by, for example, a CPU and software of a computer program for operating the CPU as described above. Of course, the third function may be achieved. In this case, the computer program may be taken into the computer from a recording medium, or may be taken into the computer via a network.

  The present invention has been described above with reference to one embodiment. However, the above embodiment is intended to exemplify an apparatus and method for embodying the technical idea of the present invention, and the technical idea of the present invention. Does not specify the material, shape, structure, arrangement, etc. of the component. The technical idea of the present invention can be variously modified within the scope of the claims.

DESCRIPTION OF SYMBOLS 1 Depth estimation data generation apparatus 2 Stereo pair generation apparatus 3 Stereo display apparatus 11 Matrix conversion part 12 Saturation average value generation part 13 Depth estimation part 21 Texture shift part 22 Occlusion compensation part 23 Post processing part 24 Left eye image signal generation part 25 Right eye Image signal generation unit 121 Saturation calculation unit 122 Integration unit 123 Normalization unit 124 Register 131 Leakage type integration unit 132 Luminance signal change value calculation unit 133 RB composite value calculation unit 134 Gain generation unit 135 Depth estimation data calculation unit

Claims (3)

A saturation average value generation unit that calculates saturation from the color difference signal component of the input video signal, calculates the average value of the saturation, and generates saturation average value data;
A luminance signal change value calculation unit that calculates a luminance signal change value by calculating a difference between a luminance signal component of the input video signal and a predetermined value set in advance;
An RB composite value calculation unit for calculating an RB composite value so that the R component of the input video signal is on the near side of the pseudo stereoscopic image and the B component of the input video signal is on the back side of the pseudo stereoscopic image; ,
A gain generation unit that generates a luminance signal gain for correcting the luminance signal change value and an RB gain for correcting the RB composite value based on the saturation average value data;
Depth for generating a signal indicating a pseudo stereoscopic image from the input video signal by combining the calculation result of the luminance signal change value and the luminance signal gain and the calculation result of the RB composite value and the RB gain. A depth estimation data generation apparatus comprising: a depth estimation data calculation unit that calculates estimation data. A saturation average value generation step of calculating saturation from the color difference signal component of the input video signal, obtaining an average value of the saturation and generating saturation average value data;
A luminance signal change value calculation step of calculating a luminance signal change value by obtaining a difference between a luminance signal component of the input video signal and a predetermined value set in advance;
An RB composite value calculating step for calculating an RB composite value so that the R component of the input video signal is on the near side of the pseudo stereoscopic image and the B component of the input video signal is on the back side of the pseudo stereoscopic image; ,
A gain generation step of generating a luminance signal gain for correcting the luminance signal change value and an RB gain for correcting the RB composite value based on the saturation average value data;
Depth for generating a signal indicating a pseudo stereoscopic image from the input video signal by combining the calculation result of the luminance signal change value and the luminance signal gain and the calculation result of the RB composite value and the RB gain. A depth estimation data generation program that causes a computer to execute a depth estimation data calculation step for calculating estimation data. The depth estimation data generation device according to claim 1, wherein the depth estimation data is generated.
Using the depth estimation data and the input video signal, a different viewpoint image signal for displaying a pseudo-stereoscopic image is generated by shifting the texture of the input video signal by an amount corresponding to the depth of the corresponding portion. A stereo pair generator,
A pseudo-stereoscopic image display device comprising: a stereo display device that displays a pseudo-stereoscopic image using the different viewpoint image signal and the input video signal.