JP4589277B2 - 3D display method - Google Patents

Description

  The present invention relates to a three-dimensional display method, and more particularly, to a high-quality three-dimensional stereoscopic image display method having a wide viewing angle characteristic in a DFD (Depth Fused 3D) type three-dimensional display method.

  As a conventional stereoscopic display method, a three-dimensional display method of a DFD (Depth Fused 3D) method capable of expressing a continuous depth only by changing the luminance ratio between the front and rear two front and rear surfaces has been proposed. (See Non-Patent Document 1 and Non-Patent Document 2 below)

As prior art documents related to the invention of the present application, there are the following.
Hideaki Takada, Shiro Suyama, Sakuichi Otsuka, Jinjo Kamihira, Shigenobu Sakai, “Three-dimensional display without new glasses”, 3D Image Conference 2000 Proceedings, 4-5, pp.99-102 (2000) Shiro Suyama, Hideaki Takada, "Development of 3D display based on new phenomenon", NTT Technology Journal, 2002.8, Vol.14 No.8, pp.74-77

In the three-dimensional display method of the DFD method described in Non-Patent Document 1 or Non-Patent Document 2, the display range of the display target object is basically two stacked display surfaces (for example, a transmissive display). This is a depth representation type stereoscopic display method in which a three-dimensional stereoscopic image is displayed between the two, and when viewed from a viewpoint with a two-dimensional image on the front and a two-dimensional image on the rear, they are overlapped (performed by being fused) ) Three-dimensional display is performed under the presentation conditions.
For this reason, in the three-dimensional display method of the DFD system, the front-view two-dimensional image (which also depends on the display position), the rear-surface two-dimensional image (which also depends on the display position), and the viewpoint position are restricted, and the viewing angle There is a problem that a wide stereoscopic display is difficult in principle.
The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to use a DFD (Depth Fused 3D) method capable of displaying a high-quality three-dimensional stereoscopic image having a wide viewing angle characteristic. It is to provide a three-dimensional display method.
The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

In order to solve the above-described problem, in the present invention, two-dimensional images obtained by projecting a display target object from the viewing direction of the observer are displayed on a plurality of display surfaces at different depth positions as viewed from the observer. In the three-dimensional display method for displaying a three-dimensional stereoscopic image by independently changing the luminance of the displayed two-dimensional image for each display surface, the pair of display surfaces adjacent to each other, When the display surface closer to the observer is the front surface and the display surface far from the observer is the rear surface, the texture image of the 3D stereoscopic image to be displayed and the depth information are used as input values, and the texture image The image processing is performed by referring to the depth value of the depth information corresponding to the texture image, applying image processing filters having different band pass characteristics according to the depth value, and performing the image processing. And texture images, from said depth information, and the luminance of the two-dimensional image to be displayed on the front, and calculates the luminance of the two-dimensional image to be displayed on the rear surface.
As a result, even if the viewpoint position of the observer is deviated from the front of the display device, the two-dimensional image displayed on the front surface and the double image displayed on the rear surface are not conspicuous, giving a natural three-dimensional effect. Can be obtained.

Further, according to the present invention, according to the mechanism of the DFD stereoscopic illusion phenomenon (the boundary of the image of the retinal image; the recognition of the edge depends on the low-pass component of the retinal image), the band pass characteristics are respectively set according to the depth value. As a different image processing filter, an image processing filter having a characteristic of cutting a high frequency spatial frequency of a texture image is used.
As a result, even when the viewing angle deviates from the image presentation condition in which the DFD illusion phenomenon occurs (for example, the normal state of FIG. 2-1 described later), the retinal images of the left and right eyes have the DFD illusion phenomenon (for example, Therefore, the DFD display presentation conditions are relaxed, and the viewing angle can be enlarged and the image quality can be improved.

Further, in the present invention, when the depth value is 0 or 1 for the purpose of reducing the amount of image processing calculation and maintaining the sharpness of the image (when only the rear surface or the front surface is displayed). Further, the image processing filter processing is omitted.
This facilitates the handling of DFD moving images that require high-speed processing. In addition, when a two-dimensional image is displayed on either the front surface or the rear surface, 2 by image processing using an image processing filter. It is possible to prevent blurring of a dimensional image and display a sharp and high-quality stereoscopic image.

The effects obtained by the representative ones of the inventions disclosed in the present application will be briefly described as follows.
According to the DFD (Depth Fused 3D) type three-dimensional display method of the present invention, it is possible to display a high-quality three-dimensional stereoscopic image having a wide viewing angle characteristic.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
In all the drawings for explaining the embodiments, parts having the same functions are given the same reference numerals, and repeated explanation thereof is omitted.
[Outline of conventional DFD 3D display device]
FIG. 3 is a block diagram showing a schematic configuration of an example of a conventional DFD display device. In the figure, reference numeral 901 denotes a transmissive display device constituting the front surface, 902 a transmissive display device constituting the rear surface, 903 a backlight, and 904 a pixel. The DFD display device shown in FIG. 3 is a display device in which two (front and rear) transmissive display devices are stacked.
The transmissive display device (901, 902) can be configured by, for example, a transmissive liquid crystal display. If the transmissive display device (901, 902) is a transmissive display device (organic EL, inorganic EL), for example. The liquid crystal display element) does not hinder the realization of any function.

FIG. 4 is a diagram for explaining a three-dimensional stereoscopic image displayed on a conventional DFD display device. In the figure, 1001 is a transmissive display surface (front surface), 1002 is a transmissive display surface (rear surface), 1003 is a front surface display image, 1004 is a rear surface display image, and 1005 is a three-dimensional display object (that is, a perceived three-dimensional display object). ), 1006 is an observation viewpoint (usually a center point between both eyes).
As shown in FIG. 4, two-dimensional images obtained by projecting the display target object from the direction of the observation viewpoint 1006 on the transmissive display surface 1001 and the transmissive display surface 1002 are respectively a pattern of the front image 1003 and a rear image 1004. It is a pattern.
Further, the respective two-dimensional luminances are distributed in accordance with the display position (depth position: Z) of the stereoscopic display object 1005. When the front and rear display surfaces are transmissive display surfaces, the luminance distribution according to the display position (depth position: Z) of the stereoscopic display object 1005 is the display position (depth position) of the stereoscopic display object 1005. : Distributes the transmittance according to Z), and means to change the luminance seen by the observer.
FIG. 5 is a diagram illustrating an example of an input / output image of a conventional DFD display device.
In the example shown in FIG. 5, a texture (RGB) image and a depth (Z) image of a three-dimensional stereoscopic image to be displayed are input, and luminance distribution processing is performed from them to generate a front image and a rear image.

[Outline of DFD Image Generation Processing of 3D Display Method of Embodiment]
FIG. 1-1 is a diagram for explaining an outline of the DFD image generation processing of the three-dimensional display method according to the present embodiment.
FIG. 1-1A is a diagram for explaining conventional DFD image generation processing. In the luminance distribution processing block 10, the luminance of an input texture (RGB) image is set as the luminance of the texture (RGB) image. A depth value (Z) for each pixel is distributed based on a depth (Z) image (depth map) expressed in the form of an image, and a front image (F) and a rear image (R) are output.
FIG. 1B is a diagram for explaining the DFD image generation processing according to the present embodiment. In the preceding stage of the luminance distribution processing block 12, a texture (RGB) image is converted to each depth value (Z) image depth value (Z) image. An RGB image preprocessing block 11 for performing image processing using an image processing filter having a band pass characteristic corresponding to each Z) is added.
The input of the GB image preprocessing block 11 shown in FIG. 1-1 (b) is a texture (RGB) image and a depth (Z) image (depth map), and the output is an RGB image preprocessed image (RGB ′). ) And depth (Z) images. Then, the texture image (RGB ′) after the preprocessing is newly used as the input of the luminance distribution processing block 12, and the front image (F) and the rear image (R) are generated.

FIG. 1-2 is a flowchart illustrating a processing procedure of the RGB image preprocessing block 11 illustrated in FIG. 1-1B.
First, for each Z value of the depth information, a corresponding image processing filter parameter is set and stored (step 101).
Next, when a texture (RGB) image and a depth (Z) image are input (step 102), one image processing filter is selected according to the Z value of the depth (Z) image, and the texture (RGB) image is selected. Image processing is performed on these pixels using the above-described image processing filter (step 104).
The above-described processing is executed for all the pixels (Step 103 and Step 105), and when the processing is completed for all the pixels, the image subjected to the image processing is stored as a new RGB image (RGB ′ image) ( Step 106).

FIG. 1-3 is a diagram for explaining an example of the image processing filter of the present embodiment. Although a Gaussian filter is used as an image processing filter in FIGS. 1-3, image processing having a characteristic of cutting other high-frequency spatial frequencies such as an average value filter, a two-dimensional discrete Fourier transform filter (two-dimensional DFT filter), etc. A filter may be used.
FIG. 1-3A is a diagram for explaining a general setting example (setting concept) when a Gaussian filter is used as an image processing filter.
This is an example in which the variance value of the Gaussian filter is changed according to the depth position of the object to be displayed (here, the Z value is normalized and 0 ≦ Z ≦ 1).
Under the condition of Z = 0.5 in which the texture (RGB) image is almost equally divided between the front and rear surfaces, the difference between the two-dimensional image displayed on the front surface and the two-dimensional image displayed on the rear surface is The dispersion value of the Gaussian filter is set large so that it becomes more conspicuous than the Z value.
On the other hand, when Z = 0 or 1, the texture (RGB) image is distributed in luminance only to either the two-dimensional image displayed on the front surface or the two-dimensional image displayed on the rear surface. There is no deviation between the two-dimensional image displayed on the screen and the two-dimensional image displayed on the rear surface.
In this case, the variance value of the Gaussian filter is set to 0 so that the RGB image is not blurred by the Gaussian filter.

FIG. 1B is a more specific image processing filter setting example.
When the value of Z is 0 ≦ Z ≦ Za, an image processing filter with σ = 0 is used, when Za <Z ≦ Zb, an image processing filter with σ = 1 is used, and when Zb <Z ≦ Zc, , Σ = 2, an image processing filter with σ = 1 is used when Zc <Z ≦ Zd, and an image processing filter with σ = 0 is used when Zd <Z ≦ 1. .
An example of a specific Gaussian filter is shown in FIG.
The first Gaussian filter is an example of an image processing filter in which σ = 0 and an input is directly output (in this case, the processing can be omitted).
The second Gaussian filter is an example of a 3 × 3 Gaussian filter with σ = 1.
The third Gaussian filter is an example of a 5 × 5 Gaussian filter with σ = 2.
Here, an example is shown in which the Z value setting is divided into three levels. However, the method of setting the values by dividing them into a plurality of other levels and changing the contents of the image processing filter is also easy. Needless to say, this is feasible.

FIG. 1-4 is a diagram illustrating an example of an intermediate output image (RGB ′) obtained by performing RGB image preprocessing on a texture (RGB) image in the present embodiment.
In the example shown in FIG. 1-4, the input image is an example of a white disk texture (RGB) image and a depth (Z) image having a constant value in increments of 0.25 from Z = 0 to 1. is there.
When Z = 0 or Z = 1, an image processing filter with σ = 0 is used. At this time, since the input texture (RGB) image becomes the intermediate output image (RGB ′) image as it is, the input image can be used as the output image as it is by dividing the case.
In this case, the white / black boundary (edge) of the disk of the texture (RGB) image is not particularly changed and remains a sharp boundary.
Assuming the case of Za <Z = 0.25 <Zb and Zc <Z = 0.75 <Zd, if Z = 0.25 or Z = 0.75, an image processing filter with σ = 1 is selected. To perform image processing. For this reason, the white (black) boundary (edge) of the disk of the intermediate output image (RGB ′) changes slightly gently due to the effect of the image processing filter.
In the case of Z = 0.5, an input RGB image is preprocessed using an image processing filter with σ = 2. At this time, the white / black boundary (edge) of the disk of the intermediate output image (RGB ′) changes more gently as compared with the case where the image processing filter with σ = 1 is used.

FIG. 1-5 is a flowchart illustrating a processing procedure of the luminance distribution processing block 12 illustrated in FIG.
After step 106 in FIG. 1-2, the luminance of the intermediate output image (RGB ′ image) is distributed using the intermediate output image (RGB ′ image) and the depth (Z) image, and the two-dimensional image displayed on the front surface is displayed. The luminance of each pixel and the luminance of each pixel of the two-dimensional image displayed on the rear surface are calculated (step 112).
The above-described processing is executed for all pixels (steps 111 and 113), and when processing is completed for all pixels, a two-dimensional image displayed on the front surface and a two-dimensional image displayed on the rear surface are displayed for each viewpoint. Are stored and output to the DFD display device (step 114).

FIG. 1-6 is a diagram illustrating an example of an output image obtained by performing luminance distribution processing on the intermediate output image (RGB ′) in the present embodiment.
In the example shown in FIGS. 1-6, the input image is an example of a white disk texture (RGB) image and a depth (Z) image having a constant value in increments of 0.25 from Z = 0 to 1. is there.
The intermediate output image (RGB ′) on which the preprocessing shown in FIG. 1-4 is performed is divided in luminance into a two-dimensional image (F) displayed on the front surface and a two-dimensional image (R) displayed on the rear surface. ing.
FIGS. 1-7 is a figure for demonstrating the modification of the image processing filter of a present Example.
In the image processing filter shown in FIG. 1-3, the variance value (σ) of the Gaussian filter is changed depending on the Z value, but the variance value (σ) of the image processing filter is changed depending on the Z value. It is also possible to make it constant.
FIG. 1-7 (a) is an example in which the dispersion value (σ) of the image processing filter is fixed. FIG. 1-7 (b) shows an image processing filter in that case as 5 × 5 of σ = 2. An example of setting a Gaussian filter is shown.
Here, a Gaussian filter is used as an image processing filter having a characteristic of cutting a high-frequency spatial frequency. However, other high-frequency filters such as an average value filter and a two-dimensional discrete Fourier transform filter (two-dimensional DFT filter) are used. Needless to say, it is possible to use an image processing filter having a characteristic of cutting the local spatial frequency.

FIGS. 2-1 is a figure for demonstrating the perception model (fusion of the depth direction) of a DFD illusion phenomenon.
In the normal state shown in FIG. 2-1, the edges C and D of the rear image and the edges A and B of the front image appear in the order of CL, AL, DL, and BL in the left eye image, Edges appear in the order of CR, AR, DR, BR. Since the order in which the edges appear in the left eye image and the right eye image is the same, each edge corresponds to each other and is perceived as a double image of the front surface and the rear surface.
On the other hand, in the state where the front image and the rear image overlap each other as shown in FIG. 2A, the appearance of the edges appears in the order of CL, AL, DL, BL in the left eye image, but AR, CR in the right eye image. , BR, DR, and the order in which edges appear in the left eye image and the right eye image is different. In such a state, the front edge and the rear edge are not matched, and the DFD illusion phenomenon occurs.
Here, for each of the front image and the rear image, as shown in the present embodiment, when the image processing for reducing the luminance change of the edge of the image is performed, for example, the right-eye image in the normal state of FIG. The brightness difference between the parts on both sides of the image (the part where only the rear image is reflected in the retina and the part where only the front image is reflected in the retina) , Both sides are enlarged. A change similar to that of the right eye image appears in the left eye image, but in the case illustrated in FIG. 2A, the effect of the right eye image is greater than that of the left eye image.
At this time, both the right eye and left eye retinal images approach the state in which the front image and the rear image overlap (DFD). Due to this effect, the restriction on the viewing angle of DFD is relaxed, and a stereoscopic illusion phenomenon similar to the DFD illusion phenomenon appears in both the normal state and the conventional case where the front and rear images are viewed as a double image. It can be.

FIG. 2-2 is a diagram for explaining a perception model (continuous depth change) of the DFD illusion phenomenon.
In a state in which the front image and the rear image overlap (DFD), both the left eye image and the right eye image are images having a step difference in luminance change, as shown in FIG. In the human visual recognition processing part, the edge of the image is detected using the low-frequency component of the retinal image, the corresponding points of the left eye and the right eye are taken, and stereoscopic perception is performed (FIG. 2-2). (See (b)).
By continuously changing the luminance ratio of the front and rear images, the perceived positions of the temporary edges of the left eye and right eye change continuously, so that continuous depth perception appears.
By performing the preprocessing of the texture (RGB) image shown in the present embodiment, the retinal image corresponding to FIG. 2-2 becomes a luminance curve represented by a physically continuous curve, and the DFD illusion phenomenon Similarly, continuous depth perception appears.

As described above, according to the present embodiment, two-dimensional images obtained by projecting the display target object from the observer's line-of-sight direction are displayed on a plurality of display surfaces at different depth positions as viewed from the observer, In a DFD three-dimensional display method for displaying a three-dimensional stereoscopic image by changing the luminance of the displayed two-dimensional image independently for each of the display surfaces, a front surface of a pair of adjacent display surfaces By applying band-pass filter image processing to the displayed two-dimensional image and the two-dimensional image displayed on the rear surface, the two-dimensional image of the front surface Applying different image processing filters according to the depth value for the problem of a double image in which the image looks double when observed from a position slightly shifted from the position where the two-dimensional image on the rear surface appears to overlap completely Image degradation (image While suppressing hair) to a minimum, it can be alleviated.
As a result, the viewing angle characteristics of the three-dimensional display method are improved, and the range that can be perceived as a three-dimensional stereoscopic image is expanded.
Further, when a two-dimensional image is displayed only on the front surface or rear surface from the Z value of the depth (Z) image, the preprocessing for applying the image processing filter to the original texture (RGB) image is omitted, thereby The burden on processing can be reduced, and the display of stereoscopic moving images can be easily realized in real time.

In this embodiment, the depth (Z) image does not necessarily have to be an image, and may be depth information including a depth value corresponding to the texture image. That is, the depth information is not received as image information, but need not be an image if the depth value is given in a form that can be associated with the texture image.
Further, it is not necessary to separately provide the texture image and the depth information, and it is also possible to pass the image as one image data including both the texture image and the depth value.
In the above description, the case where an RGB image is used as the texture image has been described. However, the RGB image is an example of a texture image and may be an image independent of each color or expressed by luminance, hue, and brightness. Alternatively, it may be a two-dimensional image.
As mentioned above, the invention made by the present inventor has been specifically described based on the above embodiments. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Of course.

It is a figure for demonstrating the outline | summary of the DFD image generation process of the three-dimensional display method of the Example of this invention. It is a flowchart which shows the process sequence of the RGB image pre-processing block shown to FIGS. 1-1. It is a figure for demonstrating an example of the image processing filter of the Example of this invention. In the Example of this invention, it is a figure which shows an example of the intermediate output image (RGB ') which performed RGB image pre-processing with respect to the texture (RGB) image. It is a flowchart which shows the process sequence of the luminance distribution process block shown to FIGS. 1-1 (b). In the Example of this invention, it is a figure which shows an example of the output image which performed the luminance distribution process with respect to intermediate output image (RGB '). It is a figure for demonstrating the modification of the image processing filter of the Example of this invention. It is a figure for demonstrating the perception model (fusion of the depth direction) of a DFD illusion phenomenon. It is a figure for demonstrating the perception model (continuous depth change) of a DFD illusion phenomenon. It is a block diagram which shows schematic structure of an example of the conventional DFD display apparatus. It is a figure for demonstrating the three-dimensional solid image displayed on the conventional DFD display apparatus. It is a figure which shows an example of the input-output image of the conventional DFD display apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,12 Luminance distribution processing block 11 RGB image pre-processing block 901 The transmissive display apparatus which comprises a front surface 902 The transmissive display apparatus which comprises a rear surface 903 Backlight 904 Pixel 1001 A transmissive display surface (front surface)
1002 Transparent display surface (rear surface)
1003 Front display image 1004 Rear display image 1005 3D display object (that is, perceived 3D display object)
1006 Observation viewpoint (usually the center point of both eyes)
F Front image R Rear image

Claims (3)

A two-dimensional image obtained by projecting the display target object from the viewing direction of the observer is displayed on a plurality of display surfaces at different depth positions as viewed from the observer, and the luminance of the displayed two-dimensional image is set to each of the above-described two-dimensional images. In a three-dimensional display method of displaying a three-dimensional stereoscopic image by changing each display surface independently,
Among a pair of display surfaces adjacent to each other, when the display surface closer to the observer is the front surface, and the display surface far from the observer is the rear surface,
Using the texture image of the 3D stereoscopic image to be displayed and the depth information as input values,
Step 1 for performing image processing on the texture image by referring to a depth value of the depth information corresponding to the texture image and applying an image processing filter having different bandpass characteristics according to the depth value;
Step 2 of calculating the luminance of the two-dimensional image displayed on the front surface and the luminance of the two-dimensional image displayed on the rear surface from the texture image after performing the image processing in step 1 and the depth information. A three-dimensional display method comprising:   The three-dimensional display method according to claim 1, wherein an image processing filter having a characteristic of cutting a high-frequency spatial frequency of a texture image is used as an image processing filter having different bandpass characteristics depending on the depth value. .   The three-dimensional display method according to claim 1, wherein when the depth value is 0 or 1, the step 1 is omitted.

Images