JP2007017558A - Apparatus and method for displaying stereoscopic image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and an apparatus for displaying a stereoscopic image by which masking relation and a glossy surface can be expressed and overcrowded adjustment inconsistency is dissolved. <P>SOLUTION: The apparatus for displaying the stereoscopic image is equipped with a 2-dimensional displaying means which displays an image for an observer's right eye and an image for an observer's left eye, an image selecting means which makes the right eye to easily see the image for the right eye or makes the left eye to easily see the image for the left eye and a 3-dimensional displaying means which penetrates the image for the right eye and the image for the left eye and displays the image for the right eye and the image for the left eye by superposing a depth image of each image on each image. The stereoscopic image seen by the observer is displayed by display while synchronizing the 2-dimensional displaying apparatus with the 3-dimensional displaying apparatus. Further, an edge part of the image is displayed at the 3-dimensional displaying means. The 3-dimensional displaying means may be a displaying means constituted by laminating the 2-dimensional displaying means which transmits light thereon to impart depth. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention displays a stereoscopic image on an electronic display device, and is capable of expressing a concealment relationship and a glossy surface, and does not cause a physiological contradiction between convergence and focus adjustment. The present invention relates to a stereoscopic image display method.

  The main physiological clues that humans perceive depth are the direction of the line of sight when looking at a certain point in space (convergence: the movement that becomes a cross-eyed when looking at nearby objects), and the image reflected in the left and right eyes The amount of deviation (binocular parallax) and the thickness of the crystalline lens (focal length) can be mentioned. In a stereoscopic display using glasses, a binocular parallax and an angle of convergence are realized by inputting different images to the left and right eyes using a polarizing filter, a liquid crystal shutter, and the like. However, these three-dimensional display methods have a problem in that a physiological contradiction occurs between convergence and focus adjustment, which causes the sickness and eye fatigue peculiar to stereoscopic vision.

  Several methods have been proposed as a method for solving this problem. One is a ray reproduction method. The most famous ray reproduction method is a holographic photograph. However, since information to be processed is enormous, it is difficult to realize this with an electronic display at the present time. As a method for realizing fine light beam reproduction that can solve the contradiction of congestion adjustment electronically, a super-multi-view three-dimensional display using a focused light source array (FLA) proposed by Yoshihiro Kashiwagi et al. 3D Image Conference 1996, Proceedings of Proceedings, pp.108-113, 1996) and Yasuhiro Takagi, 64 Eye 3D Color Display and Computer Displayed 3D Object Display (Non-Patent Document 2; 3D Image Super multi-view systems such as Conference 2002 Proceedings, pp. 85-88, 2002. are known. However, even with this method, the amount of data to be prepared is very large, and the resolution (number of pixels) of the image must be sacrificed accordingly.

  As a method of solving congestion adjustment contradiction while suppressing the amount of information to be processed, a method of combining a high frequency pattern with a cylinder lens proposed by Hideki Kakeya et al. Has been proposed (Patent Document 3). Although this method can significantly reduce the increase in the number of necessary data, since the high frequency pattern is superimposed on the original image, the deterioration of the image quality due to this is inevitable.

  Furthermore, as with the product Perspecta Spacial 3D of Actuality System./ rotating images at high speed, Stereoscopic Display Using Rapid Varifocal Mirror Oscillations by Alan C. Traub, (Non-patent Document 3; Applied Optics, vol.6, No. 6, June, 1967) and S. Suyama et al., Three-dimensional Display System with Dual-Frequency Liquid-Crystal Varifocal Lens, (Non-Patent Document 4; Jpn. J. April. Phys., 39, pp. 480- 484, 2000), or a high-speed variable focus optical system such as Depth Cube solid state 3D volumetric display by A. Sullivan, LightSpace Technologies (Non-Patent Document 5; Stereoscopic Display and Virtual Reality Systems XI, SPIE Vol. 5291, pp. 279-284, 2004), several methods for realizing a volume display by overlapping a plurality of screens have been proposed.

  In these methods, a light source or a real image or a virtual image of the light source is placed at a corresponding depth, so that there is no congestion adjustment contradiction. In addition, the amount of data required for display can be kept low compared to the ray reproduction method. However, as a drawback common to all the volume displays mentioned above, there is a problem that a hiding relationship (occlusion) between objects and gloss cannot be expressed. In order to realize volume display, naturally, volume information of the video space is required. Accurate 3D measurement in real time is difficult for the real world, and real-time display of real-time images is difficult to achieve with a volume display, as is the case with television that is prevalent in 2D images. .

JP 2001-218231 A JP 2005-165236 A JP 2005-165237 A

3D Image Conference 1996 Proceedings, pp.108-113, 1996 Proceedings of 3D Image Conference 2002, pp. 85-88, 2002. Applied Optics, vol.6, No.6, June, 1967 Jpn. J. April. Phys., 39, pp. 480-484, 2000 “Stereoscopic Display and Virtual Reality Systems XI”, SPIE Vol.5291, pp. 279-284, 2004

  As described above, in the conventional stereoscopic display system, there is no system that satisfies all three requirements at the same time: resolution of contradiction in congestion adjustment, suppression of data amount to be processed, concealment relation and glossy surface expression.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a method and apparatus for displaying a stereoscopic image that enables the expression of concealment relationships and glossy surfaces and eliminates contradiction in congestion adjustment.

  By using the present invention, it is easy to express a glossy surface and a concealment relationship in a volume display device that draws an image over multiple depths. In addition, it is possible to present a stereoscopic image in which contradiction of convergence adjustment is resolved and eye fatigue and sickness peculiar to conventional stereoscopic vision do not occur.

  It is known that when a human gazes at an object, information on an edge in an input image is used as a key for focus adjustment. Humans try to focus on the distance where the edge blur is least. Using this property, it is possible to guide the observer's focal position from the screen to the depth corresponding to the congestion.

  Specifically, based on the image you want to display, create two images, a smoothed image and an edge-extracted image. Binocular parallax and congestion are presented using a smoothed image. The edge image is arranged so as to overlap the stereoscopic image. In the conventional method, since the point where the edge blur is the smallest was on the screen surface, the focus adjustment was also fixed on the screen. However, in the present invention, the point where the edge blur is the smallest is on the edge image. This is a point on the depth where a stereoscopic image can be formed, and as a result, the focal point is guided on the stereoscopic image.

  Here, in order to overlap the stereoscopic image, it is necessary to display the edge image itself by stereoscopic display. In the present invention, a volume display technique is used to display the edge image. The volume display for displaying the edge can be realized, for example, by superposing a number of monochrome transmissive liquid crystals. This method does not perform all drawing on a single flat display, but also changes the depth to be drawn in accordance with the depth of the object to be displayed, so that it is possible to induce focus adjustment in addition to convergence. .

  Of course, it is also possible to express everything on the volume display, including areas other than edges. Certainly, if the 3D model of the object to be displayed is known, there is no problem in drawing. However, if the 3D model to be displayed is unknown, such as a stereoscopic display of a real image taken in real time, it is not necessary. Three-dimensional shape measurement is required. Stereo matching is an example of a method for measuring a three-dimensional shape of a real object in real time. However, if the target object does not have sufficient texture information, a good measurement result cannot be expected, and the calculation cost increases. Therefore, a volume display is inherently unsuitable for this application.

  However, in the present invention, since only the edge portion to be displayed is displayed on the volume display, a good measurement result by stereo matching can be expected, and the calculation cost can be reduced. Therefore, in the present invention, it is expected that not only a model whose shape is known but also a stereoscopic display of a real image combined with shooting and shape measurement in real time.

  Furthermore, since it is possible to project separate images for the right eye and left eye in the flat area (other than the edge area), it is possible to express a glossy surface, and in addition to the edge information, the occlusion relationship (the object in the foreground is behind) It is also possible to express a concealment relationship of hiding an object.

  For this reason, the stereoscopic image display apparatus of the present invention is a two-dimensional display means for displaying an image for the right eye and an image for the left eye of the observer, and the right (or left) eye is for the right eye (or left eye). Image selection means for making the image easy to see, and a three-dimensional display that transmits the image for the right eye and the image for the left eye, and displays the depth image of each image superimposed on the image for the right eye and the image for the left eye Means for displaying a stereoscopic image viewed from the observer by synchronously displaying the two-dimensional display device and the three-dimensional display device.

  Further, the image forming apparatus further includes an image forming system for forming an image that has passed through the image selecting means.

  Further, for viewing by a large number of people, the stereoscopic image display device of the present invention includes a two-dimensional display means for displaying a right eye image and a left eye image of a plurality of observers, and a right ( Alternatively, the left) eye transmits the right eye image (or the left eye image) and the right eye image and the left eye image, and the right eye image and the left eye image are transmitted. 3D display means for superimposing and displaying the depth images of the respective images, and displaying the two-dimensional display device and the three-dimensional display device in synchronism with each other so that a stereoscopic image viewed from the observer can be displayed. To display.

  The three-dimensional display means displays an edge portion of the image.

  Further, the three-dimensional display means may be a display means having a depth by stacking two-dimensional display means that transmits light.

  The stereoscopic image display apparatus of the present invention includes an edge extraction unit that extracts a three-dimensional position of each edge image and edge image from the image prepared for the right eye and the image prepared for the left eye, and the right eye described above. A two-dimensional image extracting means for extracting a two-dimensional image in which each edge image portion is suppressed from the images prepared for the left eye and the images prepared for the left eye, and the extracted two-dimensional image by the two-dimensional display device And synchronous display means for displaying the edge image on the three-dimensional display means in accordance with the display timing.

  The stereoscopic image display method of the present invention includes a step of extracting a three-dimensional position of each edge image and edge image from each image prepared for the right eye and each image prepared for the left eye, and for the right eye described above. A step of extracting a two-dimensional image in which each edge image portion is suppressed from the prepared image and the respective images prepared for the left eye, and the time when the extracted two-dimensional image is displayed on the two-dimensional display device. And displaying the above-mentioned edge image with a three-dimensional display means.

  Embodiments of the present invention will be described below in detail with reference to the drawings. In the following description, devices having the same function or similar functions are denoted by the same reference numerals unless there is a special reason.

  First, the outline | summary of this invention and 1st embodiment of this invention are demonstrated using FIG. FIG. 1 shows that among the images 1 and 2 drawn on the same screen of the two-dimensional display means, only the image 1 is observed in the right eye 4 and only the image 2 is observed in the left eye 5 of the observer 3. It comprises an image display device 6 having a mechanism for selecting an optical path and an image display device 8 which is a three-dimensional display means for displaying volume information 7 by drawing an image over multiple depths. As the image selection means of the image display device 6, glasses-type stereoscopic display system using a polarizing filter that uses polarized light orthogonal to the images 1 and 2, a liquid crystal shutter that displays the images 1 and 2 with a time difference, and a lenticular lens Known techniques such as a naked-eye stereoscopic image display device using a parallax barrier, a naked-eye stereoscopic image display device using a parallax barrier, and a naked-eye stereoscopic image display device disclosed in Patent Document 1 can be used. The autostereoscopic image display device of Patent Document 1 is a video display device that displays two types of images having parallax, a device that selectively transmits and separates the two types of images, and an optical system for an imaging system. The devices are arranged in order and combined with a device that detects the positions of the left and right eyes of the video viewer, and the display image of the video display device according to the movement of the positions of the left and right eyes of the video viewer In addition, the selective transmission device of the image is controlled, and different images including parallax are presented to the left and right eyes of the video viewer 3, and the real image plane at the hand of the video viewer using the imaging system By generating a real image in FIG. 14, a stereoscopic image having a depth at the hand of the video viewer 3 is presented.

  Further, the image display device 8 which is a three-dimensional display means for displaying volume information is required to be a light transmission type, and in addition to a known volume display using a high-speed variable focus optical system, a simpler device. It is also possible to use a display device in which monochrome TFT liquid crystal panels are stacked in multiple layers. Although the image display device 6 and the image display device 8 are not shown in the drawing, the display is synchronized by the synchronization circuit. In particular, when displaying a moving image, it is desirable that the image display device 6 and the image display device 8 are synchronized.

  In image 1 and image 2 displayed on the image display device 6, a portion other than the edge of the stereoscopic image desired to be displayed is displayed. In the case of an image having a noticeable edge, it is desirable to display the image by performing processing for suppressing the edge. On the other hand, the volume image 7 mainly displays an edge portion of the stereoscopic image. It is desirable to extract and display the edge of the image to be displayed. The image 1 and the volume image 7 are drawn so that the edge portion and the non-edge portion are aligned and overlapped when viewed from the right eye 3, and the original image corresponding to the viewpoint is restored, and the images 2 and 7 are drawn from the left eye 4. Drawing is performed so that the original image corresponding to the starting point is restored when viewed. The concealment relationship and gloss expression are realized by presenting different images 1 and 2 to the left and right eyes. On the other hand, when the edge is given as a visual stimulus, the focus of the human eye is known to be adjusted according to the depth so that the edge can be clearly seen. From this, it can be seen that the phenomenon that the focus adjustment point changes according to the depth of the stereoscopic image can be expressed by the volume image 7. As a result, the contradiction of congestion adjustment is resolved, and it is possible to show the viewer a stereoscopic image including a concealment relationship and glossy expression. That is, the concealment relationship and gloss expression are performed by the image display device 6 that is a secondary display unit, and the resolution adjustment contradiction is resolved by the image display unit 8 that is a three-dimensional display unit.

  The image display devices 6 and 8 are controlled by the display control units 22 and 23. In particular, the image display device 8 performs display in which the display position related to the depth is changed by the switching circuit 24. The display control units 22 and 23 are controlled by the synchronization control unit 21 so that the display of the image display devices 6 and 8 is synchronized. The display control unit 22 or 23 or the synchronization control unit 21 is supplied with a signal obtained by extracting the input display signal by the signal control unit 20. For example, the image signal in which the edge supplied to the display control unit 22 is suppressed and the image signal from which the edge supplied to the display control unit 23 is extracted are processed and generated by the signal control unit 20.

  In the case of a personal computer (PC) or the like, although the configuration is different from that shown in FIGS. 1 to 3, the clock signal in the PC plays the role of a synchronization signal and is processed simultaneously on one graphic board. By distributing the display signal for the two-dimensional display device to the display device for the three-dimensional display device, the two-dimensional display device and the three-dimensional display device are output in synchronization.

  In general, a three-dimensional display screen corresponding to one two-dimensional display screen may have a plurality of screens in addition to a single one depending on the configuration of the three-dimensional display device (for example, stacked liquid crystal display panels). The term “synchronization” as used above means that the three-dimensional display screen corresponding to the two-dimensional display screen is displayed synchronously.

  Hereinafter, a second embodiment of the present invention will be described with reference to FIG. In this embodiment, an image display device 9 capable of multi-eye stereo display is used instead of the binocular stereo display of the first embodiment. This makes it possible for multiple observers to observe a stereoscopic image according to a unique viewpoint, and reproduces motion parallax so that a single observer can view a stereoscopic image from various angles while moving. It is also possible. As the image display device 9, an application of the principle of integral photo, a parallax barrier method that realizes up / down / left / right suggestions, a method known in Patent Document 2, and the like can be used. A method known in Patent Document 2 is a fly-eye lens that diffuses so that light from light sources projected from adjacent image projectors and image projectors arranged vertically and horizontally are connected in close proximity to each other. And a convex lens optical system that generates a real image of each light source diffused so as to be substantially close to each other, and in accordance with the movement of the observation position, images from different image projection devices are switched and incident on each other without interruption. It is to be prepared.

  Next, a third embodiment of the present invention will be described with reference to FIG. In this embodiment, the convex lens optical system 10 for forming a real image is combined with the second embodiment. As a result, the display of the image display device 9 capable of multi-view stereo display can be generated as a stereoscopic image 11 in the air. Although the imaging position of the three-dimensional image 11 is different, a concave mirror optical system can be used instead of the convex lens optical system 10.

  There are various fields in which stereoscopic image display devices are used. For example, a display device for driving simulation of transportation, a display device for remote operation such as remote surgery, a display device for digital pets such as a more realistic simulation type fish tank with goldfish, etc. It can be used for display devices that can be shown. In the stereoscopic image display apparatus and stereoscopic image display method of the present invention, it is possible to express a concealment relationship and a glossy surface, and there is no physiological contradiction between convergence and focus adjustment. For this reason, on the side of viewing the display device, there is little fatigue even when used for a long time, and it is possible to perform a display with no sense of incongruity even at the first look.

1 is a schematic diagram showing an outline of the present invention and a first embodiment. It is a schematic diagram which shows 2nd embodiment of this invention. It is a schematic diagram which shows 3rd embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image presented to the right eye 2 Image presented to the left eye 3 Observer 4 Right eye 5 Left eye 6 Image display device 7 Volume information to be drawn 8 Image display device 9 Image display device 10 Convex lens optical system 11 Three-dimensional image generated in the air Real image 20 Signal control unit 21 Synchronization control unit 22 Display control 23 Display control 24 Switching circuit

Claims (7)

Two-dimensional display means for displaying an image for the right eye and an image for the left eye of the observer;
Image selection means for making the right (or left) eye easy to see the image for the right eye (or left eye);
Three-dimensional display means that transmits the right-eye image and the left-eye image, and displays the right-eye image and the left-eye image so that the depth images of the respective images are superimposed on each other.
A stereoscopic image display device that displays a stereoscopic image viewed from an observer by displaying the two-dimensional display device and the three-dimensional display device in synchronization with each other.   The stereoscopic image display apparatus according to claim 1, further comprising an imaging system that forms an image that has passed through the image selection unit. Two-dimensional display means for displaying a right eye image and a left eye image of a plurality of observers;
The right (or left) eye of each observer transmits the right eye (or left eye) image for easy viewing, and the right eye image and the left eye image are transmitted through the right eye (or left eye) image. Three-dimensional display means for displaying a depth image of each image superimposed on the image for left eye and the image for the left eye,
3. The stereoscopic image display device according to claim 1, wherein a stereoscopic image viewed from an observer is displayed by displaying the two-dimensional display device and the three-dimensional display device in synchronization with each other. 4.   The stereoscopic image display device according to claim 1, wherein an edge portion of an image is displayed on the three-dimensional display unit.   5. The stereoscopic image display device according to claim 1, wherein the three-dimensional display unit is a display unit that has a depth by stacking two-dimensional display units that transmit light. Edge extraction means for extracting the edge image and the three-dimensional position of the edge image from the image prepared for the right eye and the image prepared for the left eye;
Two-dimensional image extraction means for extracting a two-dimensional image in which each edge image portion is suppressed from the image prepared for the right eye and the respective images prepared for the left eye;
In accordance with the time when the extracted 2D image is displayed on the 2D display device,
The stereoscopic image display apparatus according to claim 5, further comprising: synchronous display means for displaying the edge image by a three-dimensional display means. It is a display method of the three-dimensional image display device according to claim 6,
Extracting each edge image and the three-dimensional position of the edge image from each image prepared for the right eye and each image prepared for the left eye;
Extracting a two-dimensional image in which each edge image portion is suppressed from the image prepared for the right eye and the image prepared for the left eye, and
In accordance with the time when the extracted 2D image is displayed on the 2D display device,
And a step of displaying the edge image on a three-dimensional display means.

Images