KR100930431B1 - A system for displaying stereoscopic 3D images, a method for generating stereoscopic 3D images, and a system for generating stereoscopic 3D images - Google Patents

Abstract

The multi-planar stereoscopic display system 10 and method uses a multi-surface optical device 32 that includes a plurality of individual optical elements 36-40 corresponding to a plurality of image depths to stereoscopic three-dimensional images 34 and 56. Create Image projector 20 selectively projects images 24-30 and 44-50 onto respective optical elements 36-40 to produce stereoscopic three-dimensional images 34 as seen by multisurface optical device 32. do. Psychological visual cues are added during the creation of three-dimensional image 34 and improve depth perception when viewing three-dimensional images. Floating image generator 54 may project a three-dimensional image with improved depth to produce a second stereoscopic three-dimensional image 56 that appears to float in space.

Stereoscopic display systems, anti-aliasing, optical elements, voxels, three-dimensional images

Description

System for displaying stereoscopic 3D images, methods for generating stereoscopic 3D images, and systems for generating stereoscopic 3D images {MULTI-PLANAR VOLUMETRIC DISPLAY SYSTEM AND METHOD OF OPERATION USING PSYCHOLOGICAL VISION CUES}

TECHNICAL FIELD The present invention relates to three-dimensional (3D) imaging, and more particularly, to a multi-planar display system using 3D anti-aliasing to produce stereoscopic three-dimensional images in space.

It is known that three-dimensional (3D) images can be created and appear in space. Typically, special eyewear such as goggles and / or helmets are used, but such eyewear can be a hindrance. In addition, such eyewear degrades the ability to see real 3D images due to the inherent nature of eye accessories. In addition, the use of such eyewear may result in alleviated eye fatigue by limiting the time of viewing the image, which is usually bulky and inconvenient to wear.

Therefore, there is a need to generate stereoscopic 3D images and displays without the disadvantage of using such eyewear.

Other stereoscopic systems generate such stereoscopic 3D images using, for example, self emissive stereo devices, ie voxels. Before providing an example of such a system, it is important to distinguish the word "voxel" that is misused from a 3D data component (hereafter referred to as "tridel"). A voxel is the actual shining point of light in a 3D data display and is similar to a pixel in a 2D display. Tridel, however, is an abstract 3D data type. In particular, voxels have positions of integers (i, j, k) and only have characteristics of color and brightness, while tridells are defined by a set of parameters defined at floating point positions (x, y, z) in virtual image space. Characterized. Thus, in the most general sense, Tridell is a 3D data type that includes any number of application-specific data types. For example, if a tridel is used to define polygonal vertices in 3D, the data parameters of this theoretical 3D data type are color (R, G, B) and opacity (A). As another example, if Tridell represents a data component of an image formed by a medical computed x-ray tomography scanner, the data parameter is x-ray opacity. . In another example, if the tridel represents a thermonuclear plasma, the data parameter may be the plasma density, temperature, and average rate of the plasma components.

From the above, it will be understood that in order to generate an image, ie 2D or 3D, each tridel must be mathematically processed into pixels or voxels. This process may include geometric transformations, including rotation, scaling, stretching or compression, perspective, projection, and viewpoint transformations, all of which move on the tridel's x, y, z coordinates. Further, in the process of determining the color and brightness of a pixel or voxel, the tridels can be averaged together when there are many in the space of one voxel or inserted between them when there are many pixels in the space of two tridels. . The distinction between tridel and voxels will be more clearly understood in view of the depth transformation described below to map the tridel's depth coordinates into voxel depth coordinates in the MOE device 32.

Returning to an example of another stereoscopic display system known in the art, an example of a stereoscopic imaging system is the system of 3D TECHNOLOGY LABORATORIES of Calif Mountain View, where the intersection of an infrared laser beam on a solid glass or plastic volume doped with rare earth impurity ions Create a voxel image. However, the nonlinear effect of generating visible light from two invisible infrared laser beams has a very low effect of about 1%, which requires a strong laser to produce bright images in large displays. Such strong lasers are potentially dangerous to the eyes and require an important protective enclosure around the display. Scanned lasers also typically have low resolution, low voxel counts, and the solid nature of the three-dimensional mechanism results in very heavy large systems.

Another stereoscopic display system manufactured by Actuality Systems, Inc., of Cambridge, Mass., Uses a linear array of laser diodes reflected from a high speed spin face mirror on a high speed spin projection screen. However, these high speed spin components, which are relatively large in size, must be carefully balanced to avoid vibration and, in some cases, major failures. Additionally, the size, shape, and orientation of the voxels in the display depend on their position, resulting in position dependent display resolution.

Another stereoscopic display system is provided by Neos Technologies, Inc. of Melbourne, Melbourne, which acousto-optically scans a laser beam onto a high speed spin spiral projection screen. These large spin components require a balance that is carefully maintained independent of display movement. Laser scanner systems have insufficient resolution and low rates, which strongly limits the number of voxels. Additionally, the size, shape, and orientation of the voxels in the display depend on their position, resulting in position dependent display resolution. In the end, the dramatically nonlinear nature of the display significantly increases the processing requirements for calculating other two-dimensional images.

Other types of 3D imaging systems are known, such as stereoscopic displays, which give each eye a slightly different projection view of the scene. The brain then fuses the separated images into a single 3D image. Some systems can only provide a single point of view and require special eyewear, or perform headtracking to remove the eyewear, but the 3D image can only be viewed by one viewer. Alternatively, the display may provide images in each zone that are appropriate at that point in time to multiple viewing zones of different angles, such as a multiview auto stereoscopic display. The user's eyes have a separate but adjacent viewing zone for viewing the 3D image, and the viewing zone should be very narrow to prevent chaotic annoyance as the viewer moves relative to the display. Some systems only have horizontal parallax / lookaround. In addition, depth-focus convergence mismatch can quickly give eye strain that severely limits viewing time. In addition, stereoscopic displays have limited field of view and cannot be used realistically with direct interaction techniques such as virtual real and / or force feedback interfaces.

Head mounted displays (HMD) are typically used in virtual real world applications where a pair of video displays exhibit a suitable projection field of view for each eye. A single HMD can only be used by one person at a time, providing limited vision for each eye. Headtracking should be used to provide parallax.

Another display system includes a holographic display in which an image is produced through the interaction of coherent laser light with a pattern of very fine lines known as holographic grating. This grating changes the direction and intensity of the incident light to appear to come from the position of the object being displayed. However, typical optical holograms contain vast amounts of information, so updating the holographic at high speed is computationally intensive. In the case of holographic displays having a relatively large size and sufficient field of view, the pixel count is generally greater than 250 million.

Thus, there is a need for high quality stereoscopic 3D imaging that has a computationally acceptable requirement for processing systems and has improved viewing and implementation.

In addition, in three-dimensional imaging, the use of discrete voxels causes portions of the image to appear jagged due to pixelation, for example, for features of transitions between discrete depths in stereoscopic 3D images. There is a need for a method of smoothing transitions between portions of stereoscopic 3D images.

Disclosed are a multi-planar stereoscopic display (MVD) system and method of operation for generating stereoscopic three-dimensional images. MVD systems include multi-surface optical devices that include a plurality of individual optical elements arranged in an array; An image projector for selectively projecting a set of images onto each optical element of the multisurface optical device to produce a first stereoscopic three dimensional image viewed in the multisurface optical device; And a stray image generator for projecting the first stereoscopic three-dimensional image from the multisurface optical device to produce a second stereoscopic three-dimensional image that appears to float in space at a location away from the multisurface optical device.

Each of the plurality of individual optical elements of the multisurface optical device includes a liquid crystal element having controllable variable translucency. An optical element controller is also provided to control the translucency of the liquid crystal element so that a single liquid crystal element is controlled to have an opaque light scattering state for receiving and displaying each image of the image set from an image projector, and the remaining liquid crystal elements are opaque liquid crystal elements. Controlled to be substantially transparent to enable viewing of the image displayed on the image.

The optical element controller rasters through the liquid crystal element at high speed for multiple imaging cycles, selecting one liquid crystal element therefrom to become an opaque light scattering state during a particular imaging cycle, and the opaque light scattering state receives a set of images continuously and a three-dimensional depth To move through the liquid crystal device for generating a three-dimensional three-dimensional image having a.

The image projector projects a set of images into the multisurface optical device to produce an overall first stereoscopic three-dimensional image in the multisurface optical device at a rate greater than 35 Hz to prevent human perceiveable image flicker. For example, the volume rate may be about 40 Hz. In one embodiment, for example, if about 50 optical elements are used at a volume rate of about 40 Hz, the image projector projects each of the image sets onto each optical element at a rate of about 2 Hz.

The image projector includes a projection lens for outputting a set of images. The projector also includes an adaptive optical focus system for focusing each of the image sets on each optical element to control the resolution and depth of projection of the image set from the projection lens. Alternatively, or in addition, the image projector includes a plurality of laser light sources for projecting red, green, and blue light respectively to generate and project the image set in multiple colors.

In addition, a 3D anti-aliasing method is used to soften portions of the projected image upon transition between optical elements within a multisurface optical device. Anti-aliasing adjusts the display of the voxel upon transition between optical elements such that the color value of the voxel is modified as a function of the distance of the voxel from the optical elements, so that the transition between parts of the stereoscopic three-dimensional image is smooth.

1 illustrates the disclosed multi-planar stereoscopic display system.

2 shows an optical element based liquid crystal having a transparent state.

3 shows the optical element of FIG. 2 in a scattering opaque state; FIG.

4-7 illustrate successive displays of images on multiple optical elements to form stereoscopic 3D images.

8 shows a membrane light modulator.

9 illustrates an adaptive optical system used in an image projector.

10 illustrates the adaptive optical system of FIG. 9 in relation to a multiple optical element system.

11 is a side cross-sectional view of a pixel of a ferroelectric liquid crystal (FLC) spatial light modulator (SLM).

12-14 show the angular orientation of the axes of the FLC SLM of FIG.

15 is a flow chart of a method for generating a multiplane data set.

FIG. 16 illustrates 3D anti-aliasing of voxels in a plurality of optical elements. FIG.

FIG. 17 shows a voxel display in the absence of 3D anti-aliasing. FIG.

18 shows a voxel display in the case of 3D anti-aliasing.

FIG. 19 is a graph comparing apparent depth with and without 3D anti-aliasing. FIG.

20 is a flowchart of a method of implementing 3D anti-aliasing.

21 and 22 illustrate the generation of 3D images with translucent front objects in the absence of anti-aliasing.

Figures 23 and 24 show the generation of 3D images with translucent front objects in the presence of anti-aliasing.

With reference to FIG. 1, a multi-planar stereoscopic display system 10 is provided that produces a three-dimensional (3D) image that is essentially stereoscopic, i.e., the 3D image occupies a distinct and limited volume of 3D space, where the image appears. Exists in. Therefore, this 3D image is true 3D, as opposed to an image perceived in 3D due to the optical illusion of vision, such as by stereographic methods.

The 3D image generated by the system 10 may have a very high resolution and may be displayed in a wide range of colors, having properties related to viewing the real object. For example, such a 3D image has horizontal and vertical moving parallax or lookaround, allowing the viewer 12 to move to receive more visual cues to maintain the 3D appearance of the 3D image.

In addition, the viewer 12 does not need to wear any special eyewear, such as a stereographic visor or glasses, to view advantageous 3D images because such eyewear is disruptive and causes eye fatigue. Moreover, the 3D image has a continuous field of view horizontally and vertically, with a horizontal field of view equal to 360 ° in a given display configuration. Additionally, the viewer can be at any arbitrary viewing distance from the MVD system 10, without losing 3D sensing.

The multi-planar stereoscopic display system 10 may be coupled to the system 10 or may be operatively connected to the system 10 via a communication channel, for example, from a remote location and in any network, such as a conventional telecommunication link or the Internet. It includes an interface 14 for receiving 3D graphics data from a graphics data source 16, such as a computer that can be connected via. The interface 14 may be a PCI bus, or an Accelerated Graphics Port (AGP) interface, available from INTEL, Santa Clara, Calif. Other interfaces include NuBus high-performance in Apple Macintosh computers and other systems, as well as VME backplane interconnect bus systems standardized to the IEEE 1014 standard, Small Computer System Interface (SCSI), and Indusrty Standard Aichitecture (ISA) interfaces. An extended bus system, an extended ISA (EISA) interface, a universal serial bus (USB) interface, an open or dedicated interface, as well as a FireWire bus interface standardized to the IEEE 1394 standard, which provides high-speed communications and isochronous real-time data services from a computer. have.

Interface 14 passes 3D graphics data through a multi-planar stereoscopic display (MVD), which includes a large high speed image buffer. The three-dimensional image, shown as a stereoscopic 3D image, is converted by the MVD controller 18 into a series of two-dimensional image slices at varying depths through the 3D image. The frame data corresponding to the image slice is then quickly output from the high speed image buffer of the MVD controller 18 to the image projector 20.

MVD controller 18 and interface 14 may be implemented with a computer, such as an OCTANE graphics workstation commercially available from SILICON GRAPHICS, Inc. of Mountain View, CA. Other general purpose computer based systems may also be used, such as, for example, a personal computer (PC) using a 195 MHz instruction condensed computing (RISC) microprocessor. Thus, it should be understood that the disclosed MVD system 10 and its components are not limited to the specific implementation and implementation of hardware and / or software.                 

The graphical data source 16 optionally provides an application program interface (API) to provide 3D image data in a format suitable for the MVD controller 18 of the computer via an input / output (I / O) device such as the interface 14. And a graphics application of a computer operating the device driver. MVD controller 18 may be, for example, hardware and / or software implemented in a personal computer and optionally using an expansion card for specialized data processing.

For example, an expansion card in MVD controller 18 converts a 3D data set from graphics data source 16 into a series of two-dimensional image slices forming a multi-plane data set corresponding to slices 24-30. Graphics hardware and / or software. The 3D image 34 is therefore generated at a real time or near real time update rate for real world applications such as surgical simulation, air traffic control or military command and control. Such expansion cards may also include a geometric engine for manipulating 3D data sets and texture memory for texture mapping of 3D images.

Prior to sending the image data to the image projector 20, the MVD controller 18 and optionally the graphic data source 16 are used to smooth the shape to be displayed in the 3D image 34 and xy perpendicular to the z axis. Due to the display pixelation caused by the original discontinuous voxel structure of the MOE device 32 with the optical elements 36-42 aligned in the plane, for example, jagged lines of depth between parallel planes along the z direction are removed. To avoid, 3D anti-aliasing can be performed on the image data. As data corresponding to image slices 36-42 is generated, the image elements may appear close to the edges of planar transitions, ie transitions between optical elements, for example optical elements 36-38. To avoid sudden transitions in certain image elements, slices 24 and 26 each contain images of specific image elements 44-46 such that the image elements smooth the transitions and the 3D image 34 appears more continuous. Are shared between both planes formed by the optical elements 36-38. The brightness of the image element on each successive optical element varies with the position of the image element in the image data.

Graphic data source 16 and MVD controller 18 may also perform zero run encoding through interface 14 to maximize the transfer rate of image data to MVD controller 18 for image generation. It should be understood that other techniques for conveying image data may be used, such as delta (Δ) compression, as well as the Moving Image Expert Group (MPEG) data communication standard.

The 3D image contains a resolution image on the order of 50 SVGA updated at a rate of 40 Hz, causing a raw data rate of greater than 2 GB / sec to be displayed. This raw data rate can be significantly reduced by not transmitting zero. Stereoscopic 3D images are typically represented by a large number of zeros associated with the interior of the object, the background object and the surrounding empty space blocked by the front object. The graphic data source 16 is represented by a run of zero represented by a zero-run flag (ZRF) or zero-run code and thus the associated run length. Therefore, the count of zero can be sent for display without sending zero. The 3D data image buffer in the MVD controller 18 can be initialized to store all zeros, and as the image data is then stored in the image buffer, the direction of the ZRF flag is equal to the run length of zero by the MVD controller 18. Let the jump ahead in the buffer by the data position and the number of pixels. The 3D data image buffer includes 3D data to be output to the image projector 20, which may include an SLM driver for operating the SLM to generate a two-dimensional image.

Image projector 20 is a multiple optical element (MOE) device 32 for selective imaging to produce a first stereoscopic three-dimensional image 34 visible to viewer 12 to appear in the space of MOE device 32. It has an associated optics 22 for projecting two-dimensional slices 24-30 of the 3D image at high frame rates and in a time-series manner. The MOE device 32 is one optical element that receives and displays each slice during each frame rate cycle, under the control of the MVD controller 18, and the slices 24-30 as two-dimensional images 44-50 shown. A plurality of optical elements 36-42 that selectively receive each one. The number of depth slices generated by the MVD controller 18 is equal to the number of optical elements 36-42, ie each optical element represents a unit of depth resolution of the stereoscopic 3D image to be generated and displayed.

Optical elements 36-42 may be, for example, nematic, ferroelectric, or cholesteric materials, or other such as cholesteric textures using a modified Kent State known in the art for such compositions. It may be a liquid crystal display made of a polymer stabilized material.                 

As a set of displayed images, the overall display of each of the slices 24-30 by the optical elements 36-42 of the MOE device is viewed directly by the viewer 12 at a sufficiently high frame rate, as described below. And, without a stereographic headset, and instead of individual two-dimensional images 44-50, occupies a rate higher than about 35 Hz to detect continuous stereoscopic 3D images 34. Thus, in the illustration of FIG. 1, the image 44-50 may be a cross section of a sphere, and the 3D image 34 generated in this way may be an image of the optical element 36-42 forming the MOE device 32. It will appear as a sphere to the viewer 12 disposed in the middle.

In other embodiments, images 44-50 may be generated to display the entire image with mixed 2D and 3D appearance, such as 2D text as captions below the sphere, or 2D text in the sphere. One application is the characterization of both 2D and 3D for the viewer 12 to view the GUI, such as MICROSOFT WINDOWS 95, which has a 2D screen appearance as a virtual flat screen display and has a 3D image like a sphere appearing on a virtual flat screen display. It may be a graphical user interface (GUI) having a.

The first stereoscopic 3D image 34 can be viewed within an orientation range. In addition, light 52 from the first stereoscopic 3D image 34 is further processed by the real image projector 54, so that the first stereoscopic 3D image floating spatially from the MOE device 32. Generate a second stereoscopic 3D image 56 that is seen by the observer 12 as if it were an image nearly identical to 34. The actual image projector 54, or alternatively the floating image projector, is a set of optics for condensing light 52 emitted from the MOE device 32 to reimage the 3D image 34 into free space and / or Or a mirror. The actual image projector 54 may be a high definition stereoscopic display (HDVD) comprising a conventional spherical or parabolic mirror to create a signal viewing zone located on the optical axis of the MOE device 32.

For example, the actual image projector system may be the device disclosed in Prince's U.S. Patent 5,552,934 and Crabtree, IV, U.S. Patent 5,572,375, each of which is incorporated herein by reference. In an alternative embodiment, the holographic optical device has the same function as a conventional spherical or parabolic mirror for generating floating image 56, but one viewing zone is in a center region aligned with the optical axis and Since different viewing zones can be used by the actual image projector 54 having a plurality of viewing zones on the other side of the optical axis, the plurality of 3D floating images 56 can be viewed by the plurality of observers.

In another alternative embodiment, the actual image projector 54 may comprise a holographic optical element (HOE), a hologram in the conventional sense that does not show a recorded image of an existing object. Instead, the HOE acts as a conventional optical element, such as a lens and / or mirror, that receives, reflects, and redirects incident light back in a new direction. Compared with conventional optical elements such as glass or plastic, the HOE is very light weight, inexpensive to regenerate, and may have unique optical properties not available in conventional optics. For example, since the HOE can produce multiple images of the same object at different angles from the selected optical axis, the field of view of a display using a relatively small HOE does not increase the optical size required for conventional optical devices. It can be increased dramatically without. Thus, using at least one HOE as the actual image projector 54, the MVD system 10 can be manufactured to provide a 360 ° field of view for a relatively compact system. In addition, in the case of the image projector 20 using the laser light source, the HOE is particularly compatible with the laser light source with high performance due to the wavelength selectivity of the HOE.

Since one of the stereoscopic 3D images 34 and 56 is shown to the observer 12 as having volume, depth, and optionally color, the multi-plane stereoscopic display system 10 is adapted to tactile animation for teaching surgery. As in the examples described below, it can be applied to virtual entity and haptic / tactile applications. The real image projector 54 allows the floating 3D image 56 to accept virtual interactions immediately. The MVD system 10 may include a user feedback device 58 for receiving hand movements from the observer 12 in response to the observer 12 attempting to manipulate one of the images 34, 56. This hand movement is a user feedback as a control signal sent to the interface 14 to the MVD controller 18 to change one or both of the images 34 and 56 to appear to respond to the observer 12's movement. It may be translated by the device 58. Alternatively, user feedback device 58 may be effectively connected to graphics data source 16, which may include a 3D graphics processor, to change one or both of images 34, 56.

Multiple new interaction technologies improve the performance of the MVD 10 using the actual image projector 54. For example, a force feedback interface developed by SENSIBLE DEVICE, INC. In Cambridge, Massachusetts, USA, allows the MVD system 10 to provide the ability to actually feel and manipulate the 3D images 34, 56 by hand. It is a powerful technology. With proper programming, the viewer 12 can sculpt a three-dimensional image as if the image is clay using a system called DIGITAL CLAY, a product of DIMENSIONAL MEDIA ASSOCIATES, the assignee of the present application.

Another application part of the MVD system 10 with the force feedback interface is a surgical simulator and trainer that allows the user to view and perceive three-dimensional virtual anatomy, to practice innovative new procedures to obtain a certificate as a surgeon. And even animations such as reactions to virtual heartbeats and virtual stabbing actions by the user, for example to perform telesurgery over the Internet using internet communication protocols.

Thus, the tactile effect can be combined with animation to provide a simulation and real time simulation of a user working on the 3D image generated by the MVD system 10. For example, the viewer 12 may be a surgeon teaching a medical student, where the surgeon observes and manipulates the first 3D image 23 in virtual reality, whereby the students may view the first 3D image ( A corresponding 3D image 56 that has been manipulated correspondingly by the actual image projector 54 in response to the change in 34 is observed. The students can then individually start manipulation of the image 34, such as the image of the heart, which may be the heart beating by image animation, such as the 3D image 34, 56. The surgeon can then rate and observe the students performing the image manipulation as if the images, like simulations of heart surgery, were real.

MOE device

In the above-described embodiment, the MOE device 32 consists of a stack of single pixel liquid crystal displays (LCDs) made of glass, such as optical elements 36-42 separated by glass, plastic, liquid, or air spacers. do. Alternatively, optical elements 36-42 may be constructed of plastic or other materials having various advantages, such as such a lightweight structure. Glass, plastic, and / or air spacers may be combined with glass LCDs in an optically continuous configuration to remove reflections at the internal interfaces. The surfaces of the LCDs and spacers may be optically coupled by either optical contact, index matching fluid, or optical cements. Alternatively, the spacers may be replaced with a liquid such as water, mineral oil, or index matching fluid. Such liquids can be circulated through an external cooling device that cools the MOE device 32. Also such liquid-spaced MOE devices 32 may be transported or installed in an empty sieve to reduce the overall weight. And a spacing liquid may be added after the device.

In a preferred embodiment, the optical elements 36-42 are planar and rectangular. However, it may alternatively be curved or of some form, such as a cylinder. For example, cylindrical LCD displays may be manufactured by other techniques, such as extrusion, and may be superimposed on one another. The separation distance between the optical elements 36-42 may be constant, or in alternative embodiments such that the depth of the MOE device 32 is greatly increased without increasing the number of optical elements 36-42. It can also be a variable. For example, because the eyes of viewer 12 lose depth sensitivity with increased viewing distance, optical elements positioned further away from viewer 12 may be spaced further apart. The algebraic spacing may be performed such that the spacing between the optical elements 36-42 increases linearly with a distance from the viewer 12.

The optical elements 36-42 have a property that can be quickly switched electronically by, for example, the MOE device driver of the MVD controller, or a clear, highly transparent state as shown in FIG. It is composed of a liquid crystal form with the property of being opaque and capable of switching between highly scattered states. 2 and 3 with a cross section of the optical element 36 being described, for example, the liquid crystal molecules 60-64 may be substrates 66-68, which may be glass, plastic, or air spacers. It may also float between, and may also have transparent conductive layers 70 and 71 applied to each of the substrates 66-68.

The conductive layers 70 and 71 have excellent transparency and low resistance, but are sputtered of indium tin oxide (ITO) having a relatively higher refractive index compared to the refractive index of the glass or plastic substrates 66-68, or It may also consist of a thin film deposited. The difference in refractive index between these materials may result in reflections at the interfaces. For that reason, additional coatings or layers of anti-reflection (AR) material may optionally be disposed between the conductive layers 70, 71 and the substrates 66-68 to reduce the amount of reflected light, such as unwanted reflections. It may be arranged. For example, an AR layer having an optical thickness of 1/4 (76 nm) of the wavelength of typical light, or having an index of refraction of about 1.8, reduces reflection at the interface of the substrate-conductive layer to a very low level.

By using AR coatings, the spacer material may be removed such that the space between the optical elements 36-42 is left in air or vacuum, thereby reducing the weight of the entire MOE device 32. Such AR coatings may be vacuum deposited, or deposited, or sputtered dielectrics. Alternatively, AR coatings may be applied by spin coating, dip coating, or menisus coating with SOL-GEL.

Referring to FIG. 2, using such conductive layers 70, 71, for example, a voltage source 72 from the MVD controller 18 may be provided between the substrates 66-68 of the optical element 36. 74). The substrates 66-68 of this optical element 36 allow liquid crystal molecules 60-64 to line or send light 76 through the optical element 36 with little or no scattering. In addition, the optical element 36 is sufficiently transparent.

Referring to FIG. 3, for example, removal of the voltage 72 opens between the conductive layers 70, 71, such as opening a rapidly switchable switch 78 controlled by the MVD controller 18. It may also be caused by opening the circuit. During the removal of such a voltage 72, the liquid crystal molecules 60-64 are randomly oriented so that light 76 is randomly scattered to produce scattered light 80. In this arrangement, the optical element 36 appears opaque, and thus may receive or display the image 44 focused there by the image projector 20 as a projection screen.

In an alternate embodiment, referring to FIGS. 2-3, the example optical element 36 connects the conductive layer 70 proximate to the first substrate 66 to ground in the transparent state shown in FIG. 2, while It may be activated by connecting the conductive layer 71 proximate the second substrate 68 to a voltage supply device, such as a voltage in the range of about 50V to about 250V. In order for the optical element 36 to be scattered as shown in FIG. 3 and switched while being in an opaque state, voltage application is reversed. That is, conductive layer 71 is grounded for a predetermined delay such as about 1 ms to about 5 ms, and then conductive layer 70 is connected to the voltage supply device. The process is reversed again to restore the optical element 36 to a transparent state. Thus, an average direct current (DC) current or voltage is not applied to the optical element 36 and may lead to failure by having a constant applied voltage. There is also no continuous alternating current (AC) current or voltage that generates heat to the optical elements or increases the power required.

In operation, only one of the optical elements 36-42 of the MOE devices 32 forms a plane and a surface that are being scattered at a given time and in an opaque state. As the image projector 20 projects slices 24-30 at high speed such that one slice per cycle is emitted through the projection cycle, the scattering plane is quickly rasterized through the depth of the MOE device 32 to effectively vary it. In-depth projection screens are formed, while the remaining transparent optical elements allow viewer 12 to see the displayed image from the received image slices 24-30.

As shown in FIGS. 4 to 7, as the continuous frame data is supplied from the MVD controller 18 to the image projector 20 to generate an image 82-88 therefrom, the MVD controller 18 is configured as an optical element. By synchronizing the switching of (36-42), the optical element 36 becomes opaque as the image 82 is emitted thereon as shown in FIG. 4, and the optical element 38 has the image 84 as shown in FIG. It becomes opaque as it is emitted thereon, and the optical element 40 becomes opaque as the image 86 is emitted thereon, as in FIG. 6, and the optical element 42 has an image 88 thereon as shown in FIG. 7. As released it becomes opaque. The MVD controller 18 can intervene a delay between the supply phases of each set of frame data to the image projector, causing the given optical element to become opaque, causing the image projector 20 to remove from the sets of frame data during the delay. Each has sufficient time to generate the corresponding image 82-88.

With reference to FIGS. 4-7, one optical element is opaque and displays the image thereon, while the other optical elements are transparent, for example on FIG. 4 on optical element 36 through at least optical element 38. An image 82 may be seen, similarly an image 84 may be viewed through at least optical element 40 of FIG. 5, and image 86 may be viewed at least through optical element 42. Because images 82-88 are displayed at high speed by image projector 20 on optical elements 36-42 that are switched to opaque transparent states at relatively high speeds, images 82-88 are single stereoscopic 3D. Form an image 34.

To form a continuous stereoscopic 3D image 34 in which no flicker is perceived, each optical element 36-42 will receive that image and switch to an opaque state at a frame rate greater than 35 Hz. Thus, to refresh and / or update the entire 3D image, the frame rate of the image projector 20 will be greater than approximately N × 35 Hz. For a stack of 50 LCD components forming a MOE device 32 with a discrete optical element frame rate of 40 Hz, the total frame rate of the image projector 20 will be greater than approximately 50 × 40 Hz = 2 Hz. High performance and / or high quality stereoscopic 3D imaging by the MVD system 10 will require the image projector to have a larger frame rate on the order of 15 Hz.

In one embodiment, the images 82-84 of FIGS. 4-7 are displayed sequentially, such sequential frame alignment updates the range of depths once per volume period, thereby optical elements 36-42 in the MOE device 32. ) To update the total volume. Such sequential frame alignment will be sufficient even at the lowest frame rate conditions, such as a frame display rate of about 32 Hz for still images 82-88 and 45 Hz for images 82-88 representing motion. In an alternate embodiment, semi-random plane ordering will be performed to lower image jitter and reduce motion artifacts, even though each optical element is still updated only once per volume period. The range is updated at higher frequencies. Such semi-random planar alignment includes multiplane interlacing in which the images are irradiated in the even plane and then the odd number plane, increasing the perceived volume rate without increasing the frame rate of the image projector 20.

The MOE device 32 maintains the image resolution inherent in the image projector 20 to provide a high fidelity three-dimensional image. The liquid crystal panel 36-42 has high transparency, is clear and free from haze, and can switch quickly between opaque scattering states in clear transparent states, so that light and images from the image projector 20 are efficiently And sufficiently scattered.

In other embodiments, MOE device 32 may be manufactured to be lightweight. The liquid crystal panel 36-42 may be composed of a pair of glass substrates, the inner surface of which is coated with transparent conductive layers 70 and 71, and with an insulating layer thereon. The polymer alignment layer may be optimally disposed on the insulating layer. Between the substrates of a given liquid crystal panel, a thin layer of liquid crystal constituent is disposed about 10-20 microns thick.

Most of the volume and weight of the panels are related to the organic substrate, which causes potentially very heavy MOE devices 32 as the transverse size and number of panels increase. The configuration of the liquid crystal panels 36-42 with a plastic substrate is one solution to the increase in weight. Another method involves using a processing method that creates an optical element of the MOE device 32 by a roll-to-roll process on a very thin plastic substrate.

By using such a relatively lightweight component for the MOE device 32, it may be folded when the MOE device 32 is not in operation, thereby making the MVD system 10 portable. In addition, the optical elements 36-42 may include other inorganic materials in addition to or in lieu of liquid crystal technology, such as an ITO layer organically coated by spin or dip coating.

High Speed Frame Rate Image Projector

The maximum resolution and color depth of the three-dimensional images 34 and 56 generated by the MVD system 10 are directly determined by the resolution and color depth of the high speed frame rate image projector 20. The role of the MOE device 32 is to primarily convert a series of two-dimensional images from the image projector 20 into 3D stereoscopic images.

In one embodiment, image projector 20 includes an arc lamp light source using a short arc. Light from the lamp is separated into red, green and blue elements by color separation optics and used to illuminate three separate spatial light modulators (SLM). After modulation by the SLM, the three color channels are recombined into a single beam and projected into the MOE device 32 from an optic 22, such as a focus lens, so that each corresponding two-dimensional image from the slices 24-30 is optical. Displayed on the corresponding one of the elements 36-42.

In another embodiment, image projector 20 includes a high power solid state laser in place of an arc lamp and color separation optics. Laser light sources have a number of advantages including improved efficiency, high directional beams, and single wavelength operation. In addition, the laser light source produces high saturation, sharp colors.

In another embodiment, other techniques may be used to implement the SLM provided that high speed operation is achieved. For example, high speed liquid crystal devices, modulators based on micro-electromechanical (MEM) devices, or other light modulation methods may be used to provide such high frame rate imaging.

For example, digital light processing (DLP) technology from SILICON LIGHT MACHINES (Sunny Vale, Calif.) And analog ferroelectric LCD devices from BOULDER NONLINEAR SYSTEMS (Boulder, CO) are used for output by the image projector 20. Can be used to modulate the image. In addition, the SLM may be a ferroelectric liquid crystal (FLC) device, and polarization biasing of the FLC SLM may be implemented.

In order to obtain an ultra-high resolution image in the MVD system 10 and to display each corresponding image at an appropriate depth to the optical element, the images 44 to 50 are appropriate for each corresponding optical element of the MOE device 32. And rapidly refocus. To meet this refocusing requirement, an adaptive optical system is used, which is available from G. Vdovin's "http://guernsey.et.tudleft.nl/focus/index.html" available on the Internet. Known devices, such as the fast focusing device described in " Fast focusing of imaging optics using micromachined adaptive mirrors. &Quot; As shown in FIG. 8, the film light modulator (MLM) 90 acts as a mirror having controllable reflection and focusing characteristics as the elastic thin film 92. The film 92 consists of a metal film coated with a conductive reflecting layer coated with a reflector such as plastic, nitrocellulose, "MYLAR", or aluminum in tension. Electrode and / or piezo actuator 94 is located substantially adjacent membrane 92. The electrode 94 is flat or flat to extend substantially two-dimensionally with respect to the surface of the film 92. The film 92 is mounted substantially adjacent to the electrode 94 by a mounting structure 96 such as a substantially elliptical mounting ring, a circular ring.

Electrode 94 may be a high voltage of approximately 1,000 volts from voltage source 98. This voltage may vary within a predetermined range to pull and / or repel the film 92. The film 92, which can be brought to ground potential by being connected to ground 100, is deflected by static electricity or deformed into a curved shape such as a parabola shape. When deformed in this state, the film 92 operates as a focusing optical device having a focal length, so that the projection distance can be changed at high speed by changing the electrode voltage. For example, the curved surface of the membrane 92 may have a focal length equal to half the radius of curvature of the curved membrane 92, the radius of curvature being the tension on the membrane 92, the physical properties of the membrane 92 material. , The separation of the film 92 from the electrode 94, and the voltage applied to the electrode 94.

In one embodiment, the film 92 is always biased towards the electrode 94. Alternatively, by placing a window having a transparent conductive layer on the opposite side of the film 92 from the electrode 94 and then applying a fixed voltage to the window, the film 92 is bidirectionally ie away from the electrode 94. Or by deflecting toward electrode 94, the range of image focus is wider. 1976 by Martin Yellin, SPIE CONFERENCE PROCEEDINGS, vol. 75, pp. 97-102, discloses a technique for controlling the variation of the film 92 in various directions.

The optical effect of the deflection of the MLM 90 is magnified by the projection optical device 22, causing the image projected from the object plane to be focused at a variable distance from the image projector 20 at a high refocusing rate. In addition, the MLM 90 may maintain an almost constant image magnification over the full focus range.

Referring to FIG. 9, the MLM 90 is part of an adaptive optical system, for example a quarter wave plate 104 and a beam splitter 106 to focus an image for the projection optical device 22. Adjacent to. Image 110 from object or object plane 112 is horizontally polarized by beamsplitter 106 through polarizer 108 and then thereafter passes through quarter wave plate 104 for reflection and focusing. Light enters the film 92 with circularly polarized light. After reflection, the focused image 114 thus passes through the quarter wave plate 104 to become light 114 polarized 90 ° in the direction of the incident light 110. Beamsplitter 106 then reflects light 114 towards projection optical device 22 to form an object image. By using this quarter wave plate 104 and polarizer 108 together with the MLM 90, the adaptive optical system can be folded relatively compact, leaving the MLM off axis or at a distance from the projection lens 22. Mounting 90 can be avoided.

The image may be focused from the projection optical device 22 to the normal projection surface 116 at a normal distance FN, which image is between the minimum distance FMIN and the maximum distance FMAX from the minimum projection surface 118. Can be refocused at a high rate while maintaining a high resolution image from the maximum projection surface 120.

As shown in FIG. 10, an image projector 20 that includes an adaptive optical system having an MLM 90, a quarter wave plate 104, and a polarizer 108 may selectively and rapidly capture a 2D slice of a 3D image. 2D slice is incident on the spacer 122 between the optical elements 36 to 42 of the MOE device 32 since the 2D slice can be focused on at least one optical element with a high focusing accuracy. Will not be.

As another embodiment, referring to FIGS. 9-10, the image projector 20 may include an SLM 124 having a plurality of pixels 126 to modulate the light 110 from the object plane 112. . A switchable half-wave plate is formed by using a switchable half-wave plate to produce an alignment layer with a 90 degree orientation on the front and back substrates of the SLM 124. The liquid crystals of the TN SLM align to the alignment layers on each side, and then gently bond between the two substrates to form a half-cycle helix. If the helix pitch is selected near the wavelength of light, the helix acts as a half-wave plate and rotates the incident light by polarizing it by 90 °. Applying a sufficient electric field to the TN SLM redirects most of the liquid crystal material between the two substrates to a point perpendicular to the substrate, thereby reversing the helix and breaking the half-wave plate, thereby ignoring the polarization rotation of the incident light. If the TN liquid crystal material inherently lacks polarization, it causes the TN SLM to be insensitive to the sign of the applied voltage, and any applied voltage produces the same reduction in the waveplate operation, thus delaying the delay, which is a magnitude function of the applied voltage. It operates as a wave plate.

Alternatively, as shown in FIG. 1, the SLM 124 may include each pixel 126 having an FLC material 128 positioned over a semiconductor substrate, such as a silicon substrate 130, and an electrode located therebetween. Or a ferroelectric liquid crystal (FLC) based device composed of a plurality of pixels 126 having 132. The electrode 132 may be made of aluminum. Transparent conductor 134 is positioned over FLC ash 128 and connected to a voltage source, such as a 2.5 V operating voltage. A cover slide 136, for example made of glass, is positioned over the transparent conductor 134.

FLC SLMs composed of these pixels 126 operate in a similar manner to twisted nematic (TN) SLM, where applying an electric field between the electrode 128 and the conductor 134 rotates the polarization of the incident light. The degree of rotation is proportional to the applied voltage and varies between 0 ° and 90 °. In combination with an external polarizer, such as polarizer 108, rotating the SLM 124 polarized intensifies the incident light.

Unlike TN SLMs, FLC SLMs have inherent polarization, giving FLC SLMs the desired thickness to form a waveplate with a delay independent of the applied voltage. The FLC SLM operates as a waveplate with an orientation that is a function of both the magnification and the sign of the applied voltage.

For the pixel 126 of the FLC SLM 124 of FIG. 11, a half waveplate of the FLC SLM 124 is implemented to have an unpowered orientation of about 22.5 ° with respect to the horizontal reference axis. The incident light polarization is rotated by 45 degrees. When powered, the transparent conductor 134 is biased at 2.5V, which corresponds to half the voltage range of the electrode 132 of the pixel 126.

12 to 14, the orientation of the principal axis of the half-wave plate formed by the pixels 126 of the FLC SLM 124 is 0 °, 45 °, respectively at 0 V, 2.5 V, and 5 V; It is shown to have 90 ° polarization.

Both TN SLM and FLC SLM are direct current balanced to maintain correct operation. Applying a continuous DC electric field to the pixel 126 causes the alignment layer on the substrate to be destroyed by impurity ion bombardment, resulting in damage to the pixel 126. To prevent such damage, the sign of the electric field is periodically and / or irregularly made at a frequency of about 100 Hz for the TN SLM and about 1 Hz for the FLC SLM. The TN SLM has low sensitivity to the sign of the electric field, so that the image passing through it maintains a constant appearance even when the electric field is reversed. However, since the FLC SLM is sensitive to the sign of the electric field, gray scale inversion occurs, which means that the black region of the image turns white and the white region turns black when the SLM is DC balanced.

In order to prevent the gray level inversion during the DC balance of the SLM 124, when the polarization of the incident light is biased, the positive and negative images generated by applying an electric field to the pixel 126 have the same appearance. SLM 124 and / or individual pixels 126 have static half-wave plates 138 located in front of SLM 124 to receive incident light 110. The wave plate 138 is oriented to rotate the polarized light of the incident light by 22.5 °, so that the grayscale has the maximum brightness when applying either 0 V or 5 V to the electrode 132, and 2.5 to the electrode 132. When V is applied, it has the minimum brightness. In another embodiment, the maximum brightness of the polarized biased FLC SLM 124 is determined by using FLC material 128 with a 45 ° static orientation to prevent the maximum brightness from decreasing by including the waveplate 138. It can be equal to the maximum brightness of the unbiased SLM without plate 138.

As described above, in another embodiment of the image projector 20, a laser such as a colored laser and / or a solid state color laser may be used for the object surface 112. Such lasers include, for example, blue and green solid state lasers, and are currently being utilized in other information storage and retrieval technologies such as CD ROMs and laser video systems.

In another embodiment of the image projector 20, adaptive optics are used in the head-up display to produce 3D images that can be moved toward or away from the viewer 12 rather than having a fixed depth. If the MOE device 32 is not used, the 2D image slices 24-30 are projected directly into the eye of the viewer 12 to appear at the correct depth. By quickly displaying these slices 24-30 in the viewer 12, the viewer 12 sees the 3D image. In this embodiment of the MVD system 10, the adaptive optical device of the image projector 20 and other components is very compact to be incorporated into a head-up display for existing helmet-mounted displays, or instrument panel mounting systems of vehicles.

In another embodiment, slices 24-30 are generated and projected so that several images 44-50 are displayed on one or more optical elements 36-42, respectively, and the MOE is not a single depth corresponding to a single optical element. The device 32 oversamples the depth by displaying the image over a range of depths. For example, it is more advantageous if the MOE device 32 has more planes of the optical elements 36-42 than the number of image slices 24-30, so the number of images 44-50 is an image slice. More than (24-30). For example, slices 24 are displayed as images 44-46 on both optical elements 36-38, respectively. This oversampling produces a 3D image 34 with a more continuous appearance without increasing the number of optical elements 36-42 or the frame rate of the image projector 20. Such oversampling can be done, for example, by switching the plurality of optical elements to be in an opaque state such that a single projection slice is received at each of the plurality of opaque optical elements during each of the plurality of projection cycles.

Creation of 3D images from multisurface data sets

Received by the MVD controller from the graphics data source 16 to produce a set of 2D image slices 24-30 that will be represented as a set of 2D images 44-50 to form a 3D image 34. A multi-surface data set is generated from the 3D image data. Each of the slices 24-30 is represented by an appropriate depth within the MOE device 32; That is, slices 24-30 are selectively projected onto particular ones of optical elements 24-42. When slices 24-30 of 3D image 34 are close enough, image 34 appears as a continuous 3D image. In addition, the optional multisurface anti-aliasing described herein may be used to enhance the continuous appearance of the 3D image 34.

A method for computing a multi-surface data set (MPD) is performed by the MVD system. In particular, the MVD controller 18 performs a method such as combining information from the depth (or z) buffer and the color buffer of the frame buffer of the graphics data source 16, which may be a graphics computer. The method also includes fixed depth of operation and anti-aliasing.

With reference to FIG. 15, the method may include steps for interaction with a user 12 operating an optional user feedback device 58 or an MVD system 10 such as a GUI to select and / or manipulate an image to be displayed. Respond at 140. Such operation and / or mutually, MVD system 10 executes the rendered image in step 142 from image data stored in a frame buffer, which may be, for example, the memory of MVD controller 18. The frame buffer includes sub-buffers such as a color buffer and a depth buffer. In a typical rendering process, the graphics computer computes the color and depth of each pixel at the same (x, y) location in the depth buffer. If the depth of the new pixel is less than the depth of the previously computed pixel, the new pixel is closer to the viewer, so that the color and depth of the previous pixel are changed to the color and depth of the new pixel in both the color buffer and the depth buffer, respectively. If all objects in a scene are rendered to the data set for image formation, the method continues with steps 144-152. Alternatively or additionally, the image rendered in the frame buffer can be displayed to the viewer as a 3D image on a 2D computer screen as an introduction to generating the 3D image into a stereoscopic 3D image 34, resulting in 3D to the viewer 12. The image to be generated as the image 34 is selected.

In executing the method for the MPD operation, data from the color buffer is read in step 144 and data from the depth buffer is read in step 146. For example, the frame buffer may have the same number of pixels in the x-axis and y-axis, which may be determined by the pixel size of the optical element 36-42 as image slices 24-30 of the desired size. If the number of pixels per unit size is not the same between the frame buffer and the image slices 24-30, then the data in the color and depth buffers have the same resolution as the MVD system 10 with the desired pixel size of the image slices 24-30. Is scaled in step 140 to have. The MVD controller 18 includes an output buffer in memory for storing the final MPD generated from the data of the color and depth buffer, which may be scaled data as described above.

The output buffer stores a data set corresponding to a 2D image having the same resolution and color depth as the image 44-50 to be projected by slices 24-30. In a preferred embodiment, the number of images 44-50 is equal to the number of planes formed by the optical elements 36-42 of the MOE device 32. After the MPD operation is completed and the pixels of the 2D image are sorted in the output buffer in step 150, the output buffer is moved to the MVD image buffer, which can be persisted in memory in the image projector 20, from which The 2D image is converted into image slices 24-30 to form the 3D image 34 to be shown to the viewer 12 as shown. The method then returns to step 140 concurrently with the generation of the 3D image 34, for example, to process the new input, and from there on, for example, the 3D image to generate an active 3D image. (34) is updated or changed.

The MVD system 10 operates in two modes: variable depth mode and fixed depth mode. In the variable depth mode, prior to generating the stereoscopic 3D image by the MVD system 10, the steps of determining the maximum depth value Zmax and the minimum depth value Zmin corresponding to the extreme depth value of the 3D image on individual 2D screens ( The depth buffer is tested prior to the MPD operation included in 146. In fixed depth mode, Zmax and Zmin are values assigned by viewer 12 during application startup or during interaction to represent the back and front limits respectively of 3D image 42 generated by MVD system 10. to be. The variable mode symbol causes all objects visible on the 2D screen to be displayed on the MOE device 32 irrespective of the range of the depth of change or the depth of the image due to the interactive manipulation of the scene with the object.

Using the values of Zmax and Zmin according to the MPD method described herein, the depth value in the depth buffer can be offset and scaled in step 148 such that a pixel with a depth of Zmin has a scaled depth of 0 and Zmax A pixel having a depth of 된다 will have a scaled depth equal to the number of planes of the optical elements 36-42 of the MOE device 32. Such pixels with scaled depth in step 150 examine the integer portion [di] of the scaled depth value di and match the color value from the color buffer to the appropriate (x, y) coordinates in the MPD slice 24-30. Is then sorted and stored in the output buffer. This color value may represent the brightness of the associated pixel or voxel.

Based on the above, it will be apparent to those skilled in the art that the same effect can be achieved using a selected subset of the optical elements 36-42 of the MOE device 32. However, in the preferred embodiment, all the optical elements 36-42 of the MOE device 32 are utilized.

As discussed above, taking into account the distinction between voxels and tridells, the process of mapping the depth of the tridel from virtual space in the display to its voxel depth coordinates actually occurs in two steps. The first step involves the conversion of Tridel's virtual depth coordinate z to the actual depth coordinate z 'in the multi-plane display. The second step involves converting the continuous z 'value of the tridel into the discrete depth coordinate k of the particular display voxel k. The reason will be apparent below.

The conversion from z to z 'may be performed at MVD controller 18 or graphic data source 16. This conversion is preferably performed by software (application, API, or device driver) or graphics card hardware within the MVD controller 18 because it is displayed somewhat independently. Similarly, the conversion from z 'to k may be performed at MVD controller 18 or graphic data source 16. However, this conversion will often be performed at the MVD controller 18 or by hardware or firmware because it depends on the particular parameter of the display.

However, in a system where the multi-plane frame buffer is actually on the graphics card of the graphics data source 16, the conversion from z 'to k must be performed in the graphics card hardware. In this case, the graphics card must be able to query the MVD controller 18 as a k mapping characteristic at z 'so that they can be used during the process from tridel to voxel.

The virtual depth coordinates in the graphical data source 16 can potentially have a range far deeper than the physical depth of the stereoscopic display. For example, a scene of a house and a street may have a virtual depth range of 50 meters, while the MOE device 32 may only have a physical depth of 0.3 meters. In addition, the mapping of the physical depth z 'to the virtual depth z of Tridel may take the form of any function provided a single value is provided. For example, in the variable depth mode described above, the simplest mapping scales the entire virtual depth range D _V linearly within the depth D _D of the MOE device 32 with the same constant scale parameter as D _D / D _V. Match. Similarly, in the fixed depth mode described above, the first 0.3 meters of virtual space can be mapped to display on a constant scale of one. Portions of the scene with a depth greater than D _D may not be displayed, or may be drawn as a 2d background on the deepest side of the display.

Another useful mapping is to provide high resolution and reduced resolution at higher depth values for nonlinear and low depth values. For example, the square root function has a resolution that decreases as z increases, providing the highest resolution near zero. In the example using the preceding values for D _V and D _D , use the following mapping for z in the range of 50 meters from O:

In general, any single valued function can be used for a map from z to z 'and it will be left to the programmer or viewer to determine how to make the best z to z' map for a particular image or application.

What is needed is a method for calculating the discrete voxel depth k from the required physical depth z 'of the Tridel to produce an image in the MOE device 32. The MOE device 32 is composed of many optical elements or image planes (N _Planes ) that occupy a range of physical depths between 0 and D _D. In the simplest case, the planes can be spaced equally by Δ = DD / (N _Planes- 1). This makes the relationship between z 'and k simple, linear and equal to k = z' / Δ. However, it may sometimes be desirable to increase the space on the face with increasing depth from the viewer. In this case, the relationship between z 'and k becomes nonlinear. For example, the space between planes k and k + 1 is given by:

And the overall depth of the display is

The physical depth z 'of face k is

The equation can be given as follows for k.

Since the negative root is a negative value and is clearly a non-physical solution, it can be determined by inspection that the positive root of the equation is used to calculate the voxel depth k from the physical depth z '. The voxel depth can be calculated “on the fly” from the above equation as voxel data is sent to the display, but in advance since the range of both z ′ and k is known from the design of the MOE device 32. Using a calculated lookup table is more effective.

It is generally noted that the above equation does not represent an integer value as a result. This may be allowed because multi-planar anti-aliasing acts to determine how bright the voxel at the depth k associated with the tridel at the virtual depth z can be divided among two adjacent display voxels. Recall that the integer portion of k determines the plane pair in which the brightness of the tridel is specified and how the fractional portion of k determines the brightness between the two planes. For example, if the tridel at (i, j) has a value like 5.34, 34% of the brightness of the tridel will be found on the voxel at (i, j, 6) and at (i, j, 5) Will be found on the voxel.

Utilizing the disclosed MPD method, the stereoscopic 3D image 34 generated by the MVD system 10 may be incomplete. In other words, if an object or region is not visible when viewing the viewer's eye viewing the corresponding 3D image on a 2D computer screen, the object or region there is completely removed. In the stereoscopic display generated by the MVD system 10, the viewer 12 of FIG. 1 can be moved at a viewing angle allowing viewing of previously hidden objects, thereby allowing the user to look around the image one by one. 10) has an advantage over the method of displaying a conventional 3D image as a 2D image.

In an alternative embodiment the MPD method implements anti-aliasing using a fractional value of the scaled depth value, i. Assigns to adjacent MVD image slices. For example, if the scaled depth value is 5.5 and each slice corresponds to a discrete depth value, half of the pixel brightness is assigned to each of slice 5 and slice 6. Alternatively, if the scaled depth is 5.25, 75% of the color values are assigned to slice 5 and 25% of the color values are assigned to slice 6 because slice 5 is the "closer" scaled depth.

Different degrees of anti-aliasing may be appropriate for different visualization tasks. The degree of anti-aliasing can vary from one extreme value. That is, the fractional depth value for assigning color values is ignored to another extreme value using all fractional depth values. Alternatively, the degree of anti-aliasing can be varied to be any value between such extremes. Such variable anti-aliasing can be performed by multiplying the fractional value of the scaled depth by the anti-aliasing parameter and then negatively setting the resultant value to half the value of the anti-aliasing parameter. The final color value can be determined by clamping or clamping the negatively offset value to be within a specified range, such as between 0 and 1. A 1 anti-aliasing parameter is one in which anti-aliasing is fully implemented, and an infinite anti-aliasing parameter is one without anti-aliasing. Anti-aliasing parameters less than one may also be implemented.

When scaling depth buffer values, perspective projection can be used as specified in the Open Graphics Library (OpenGL) multisurface software interface for graphics hardware that supports rendering and imaging. This perspective projection results in nonlinear values in the depth buffer. In order to achieve an accurate relationship between the virtual depth and visual depth of the 3D image 34, the MVD controller 18 takes such nonlinearities into account when calculating the scaled depth in step 148. Alternatively, orthographic projection may be used to scale the depth buffer value at step 148.

Those skilled in the art will understand that there are many factors that contribute to the human vision of perceiving an object or scene in three dimensions. Among these factors are physical visual cues and psychological visual cues. As an example, the physical visual cue causes but is not limited to the following physical effects.

The three-dimensionality of the scene is associated with the fact that slightly different images are provided for each eye. This binocular effect, or so-called stereopsis, is an important physical cue that is processed to add to the display of three-dimensionality by the brain. Also, when viewing a real three dimensional scene, the viewer's eyes must change their focus so that they focus on different depths in the three dimensional scene. This difference in the focus of the eye, sometimes referred to as eye accommodation, is another physical visual cue that allows the brain to conclude that a three-dimensional scene is shown. A closely related physical cue is visual convergence, meaning that both eyes are focused on the same point. In viewing an actual three-dimensional scene, the amount of visual convergence varies with the eye focus on different depths in the three-dimensional scene. This provides another cue to the brain that the scene being shown is three dimensional.

Another example of a physical queue arises from the fact that an actual real three-dimensional scene requires the viewer's movement to see different parts of the three-dimensional scene. This so-called "image look around" or motion parallax is also another physical cue associated with a real three dimensional scene that adds to the brain the perception that the scene shown is a real three dimensional.

Physical visual cues are uniquely presented in the stereoscopic three-dimensional images disclosed herein as they create and occupy a volume of space, as illustrated by the above effects. These physical cues can distinguish these images from images appearing in three dimensions, but in fact are given on two-dimensional displays such as television screens or computer monitors.

By their nature, the stereoscopic three-dimensional image display disclosed herein produces an image that can be measured but with a fine depth of field. This depth can be adjusted by varying the geometry of the MOE device 32, including the number and spacing of the plurality of optical elements 36-42 contained therein, but not in the perceived stereoscopic image of the stereoscopic image coexisted by the MOE 32. Depth is necessarily limited by practical considerations.

It is possible in the art to create and emphasize the depiction or depth of three-dimensionality in a two-dimensional image in addition to the physical visual cues provided to the brain when viewing a real three-dimensional scene, or by the use of one or more psychological cues. Known. By way of example, psychological visual cues can also be provided by rendering apparent depth to objects in the scene in a scene with suitable shading and / or shadowing to give the scene a three-dimensional appearance as a whole.

A common psychological cue is the use of forced perspective. In current 2D monitors, perspectives are formed that require computation with visualization of 3D data that creates a depth of field so that objects farther away from the viewer appear smaller and parallel lines converge. In the disclosed MVD system 10, the 3D image 34 is formed to generate the above-described depth of field in a perspective that requires calculation, so that the depth of the 3D image 34 is improved.

In addition, a three-dimensional appearance may be provided to the scene by rendering the object in the scene such that it has a surface texture whose resolution decreases with the apparent distance of the object from the viewer. This provides an apparent “fuzziness” of the surface that increases as the apparent depth increases in the scene. By increasing the degree of haze associated with a distant object or shifting the color of the distant object towards blue as the apparent distance increases, the addition of ambient effects during the rendering of a scene such as a landscape is closely related to the psychological visual cue. Another psychological cue that gives the scene a three-dimensional apparent depth is to reduce the brightness of the perceived object over distance or as it loses focus.

Another psychological visual cue is the use of occlusion, which means that parts of distant objects can be obscured by objects in front of them. Since the front part of the image cannot block light from the background part of the image, the stereoscopic display cannot provide real psychological blockage in the 3D image. Thus, if the front and background portions of the 3D image are fully formed, the background portion will be seen through the front and the front portion will appear translucent rather than solid. However, a quasi-occlusion effect can be formed by not forming part of the background image that is otherwise obscured by the front image. Thus, the apparent blocking effect can be obtained by this technique at least within an angle to the selected viewing axis.

Although the use of psychological visual cues is well known to artists and artists who wish to impart three-dimensional characteristics to two-dimensional pictures and the like, such psychological visions, when combined with the physical cues inherently provided by the three-dimensional display disclosed herein It has been found that the cue provides a 3D image where the apparent depth can sometimes exceed the physical depth of the MOE device 32 by a large factor.

For example, an image of the interior of the 3D box can be given as a 3D stereoscopic image by the system described herein. Given the box in a geometrically correct manner, the inside of the box will look no deeper than the depth of display (ie, the depth of the MOE device 32). However, by using forced perspective during the rendering of the 3D box before forming the stereoscopic image, the deeper parts of the image are given on a reduced scale, so that the 3D box can be formed to look significantly deeper than otherwise seen as a three-dimensional image. Can be.

As another example, since the image of a road retracting into a distance within a stereoscopic display is implemented by low pass filtering during the rendering process, through the combination of the physical depth of the display and the use of forced perspective and reduction of image resolution with distance It can make it look more realistic.

As is apparent from the above, it may be desirable to add one or more of the aforementioned psychological visual cues as well as other during the rendering of the scene before projection of the scene to form a stereoscopic 3D image.

In the implementation of the MVD system, psychological visual cues can be added by using commercially available software applications such as 3D Studio Max, SoftImage, and Lightwave during the rendering process in the MVD system 10. These software applications may reside in the graphics data source 16 or the MVD controller 18, or may be included in a separate standalone processor that is a functional part of the MVD controller 18. For example, a psychological visual cue that can be added by being configured with many renderings of each rendered scene so that a background blur due to the short depth of focus is created with a camera that orbits around the point of focus.

Also, haze, blue shift of light with depth, dimming of brightness with depth, and psychological visual cue of depth of focus (i.e., peripheral psychological cue) may be added to the input processor of graphic data source 16 or MVD controller 18. Or in real time by a separate processor that is part of the MVD controller 18. In particular, the image data transferred to the display frame buffer may be stored in such a way that images of different depths are in separate storage areas. This makes it possible to introduce depth cues by performing depth dependent image processing. For example, haze can be added by reducing the contrast of deeper images. The blue shift can be added by doing blue to shift the color balance of the deeper image. Darkening can be added by reducing the brightness of deeper images. Blur of the depth of focus can be added by applying an image of increasing distance on one side of the depth of focus to a Gaussian blur filter of increasing intensity.

Physical and / or psychological depth cues are emphasized such that the display of 2D images is “3D” apparent, as disclosed, for example, in US Pat. No. 5,886,818, which is often directed to enhancing projected 2D images to represent floating in space. Can be added. However, it was not previously recognized that physical and psychological depth cues can greatly enhance the 3D appearance of stereoscopic 3D images formed by the systems and techniques disclosed herein, although not limited to those described above. Thus, despite the fact that stereoscopic 2D images are formed by these systems and techniques, 3D imagers of physical and / or psychological depth cues are added during the image rendering process, which is much more substantial than otherwise. This forms a 3D stereoscopic image recognized as three-dimensional.

In yet another embodiment, some of the images 44-50 may be replaced with an optical element to display overdepth its depth by displaying for a depth having a range of the MOE device 32 instead of a single depth corresponding to a single optical element. Slices 24-30 are created and projected to be displayed individually in one or more of 36-42). For example, oversampling may be advantageous if the MOE device 32 has more planes of the optical elements 36-42 than the number of image slices 24-30, so the number of images 44-50 is It becomes larger than the number of image slices 24-30. For example, slices 24 are individually displayed as images 44-46 on both optical elements 36-38. This oversampling produces a more continuous apparent 3D image 34 without increasing the number of optical elements 36-42 or the frame rate of the image projector. Such oversampling can be performed, for example, by switching the plurality of optical elements to be in an opaque state so as to receive the projected slice onto the individually opaque optical elements once during individual multiple projection cycles.

Alternative embodiment of the MVD system

In one alternative embodiment, the MOE device 32 includes ten liquid crystal panels 36-42, dimensions 5.5 inches (14 cm) long, 5.25 (13.3 cm) wide, and 2 inches (4.8 cm) deep. Has Image projector 20 includes an acoustic-light laser beam scanner that uses a pair of ion lasers to produce red, green, and blue light, which are modulated and then scanned by high frequency sound waves. The laser scanner can vector-scan 166,000 points per second at a resolution of 200 x 200 points. When combined with 10 planar MOE devices operating at 40 Hz, the MVD system 10 can generate a 3D image with a total of 400,000 voxels, i.e. 3D image elements. A color depth with a 24-bit RGB resolution is obtained at an image update rate of 1 Hz. Using an actual image projector 54, a field angle of 100 ° x 45 ° can be obtained.

In another alternative embodiment, the MOE device 32 includes 12 liquid crystal panels 36-42, is 6 inches (15.2 cm) long, 6 inches (15.2 cm) wide, and 3 inches (7.7 cm) deep. Has phosphorus dimensions. Image projector 20 includes a pair of Texas Instruments DLP video projectors, which are designed to operate in field sequential color mode to produce grayscale images at a frame rate of 180 Hz. By interlacing the two projectors, an efficient single projector is formed at a frame rate of 360 Hz, producing 12 stereoscopic planar images at a rate of 30 Hz. The lateral resolution obtained is 640 x 480 points. When combined with 12 planar MOE devices 32 operating at 30 zHz, the MVD system 10 produces a gray 3D image with a total of 3,686,400 voxels. A color depth with 8 bit gray scale resolution is obtained at an image update rate of 10 Hz. Using the actual image projector 54, a field angle of 100 ° x 45 ° can be obtained.

In a further alternative embodiment, the MOE device 32 includes 50 liquid crystal panels 36-42, is 15 inches (38.1 cm) long, 13 inches (33.0 cm) wide, and 10 inches (25.4 cm) deep. Has phosphorus dimensions. Image projector 20 includes a high speed analog ferromagnetic LCD available from Boulder Nonlinear Systems. This LCD displays a very fast frame rate of 10 kHz. The lateral resolution obtainable is 512 x 512 points. When combined with 50 planar MOE devices 32 operating at 40 zHz, the MVD system 10 produces a gray 3D image with a total of 13,107,200 voxels. Color depth with 24-bit RGB resolution is obtained at an image update rate of 10 Hz. Using the actual image projector 54, a field angle of 100 ° x 45 ° can be obtained. Depending on such resolution and uninterrupted volume rate of 40 Hz, the MVD system 10 has display capabilities equivalent to a conventional monitor having a size of 20 inches (50.8 cm) in the diagonal direction.

In another embodiment, the optical elements 36-42 may have lateral resolution of 1280 × 1024 and depth resolution of 256 planes. The system will be operable in a depth interlaced mode in which the faces replaced by the completed volume updated at a rate of 37.5 Hz are written at a rate of 75 Hz total. This interlacing provides a higher sense volume rate without increasing the frame rate of the image projector 20.

In another embodiment, the MOE device 32 includes 500 planes for significantly greater depth resolution, and 2048 × 2048 pixel lateral resolution, resulting in voxel coefficients of more than two billion voxels. In this configuration, the size of the MOE device 32 is 33 inches (84 cm) by 25 inches (64 cm) in width by 25 inches (64 cm) in depth, which is the same as a conventional display having a diagonal of 42 inches (104 cm). The image projector 20 in this embodiment includes the Grationg Light Value technology of SILICON LIGHT MACHINES, which provides a frame rate of 20 kHz.

Virtual interactive applications

Other embodiments of the MVD system 10 incorporating a user feedback device 58, such as a power feedback interface, provide 3D images 34, 56 at a location where the viewer 12 is displaying 3D images 34, 56. ) So that you can feel it by touching it directly. The MVD system 10 can generate high resolution 3D images 34, 56, and this virtual interaction is performed in the MVD system 10 using a suitable power feedback device to produce a very high resolution surface texture. Hard surfaces, ie, surfaces created by the viewer 12 that represent resistance and / or low compliance in light of the virtual reality movement of the surface portion.                 

Accordingly, the user feedback device 58 includes a high resolution position encoder and a high frequency feedback loop to not only recognize the power feedback of the viewer 12, but also display the movement of the hand of the viewer 12 due to the change in the 3D images 34 and 56. ) Preferably, the user feedback device 58 is a power-feedback-induced-globe to minimize the mass and inertia of the reduced mass and bulk and associated components to impede the movement of the viewer 12. Includes lightweight compact virtual reality components such as:

Such a user feedback device can significantly reduce the weight of any available component written by the viewer 12, including a lightweight carbon mixture. Moreover, very compact and high resolution fiber optics, or capacitive position encoders, can be used in place of the bulky optical position encoders known in the art for determining some position of the viewer 12, such as hand or head orientation. .

The components available for viewer 12 include an embedded process system for controlling user feedback device 58, thereby reducing the processing overhead of MVD controller 18 and / or interface 14. By using an embedded processor whose only task is to implement an interface, the feedback rate for the entire MVD system 10 can be over 100 kHz. When combined with very high resolution encoders, the MVD system has an extremely high fidelity power feedback interface.

Using virtual interaction techniques by the MVD system 10 capable of displaying such stereoscopic 3D images 34, 56, the 3D GUI allows the viewer 12 to access and manipulate the 3D data directly. do. Known interface devices such as data globes, video gesture recognition devices, and the FISH SENSOR system available from MassachusettS, MIT MEDIA LAB of Cambridge, can be used to directly display 3D data, for example, in 3D graphics and computer aided design (CAD) systems. Used to make it operable.

For such 3D image and data manipulation, the MVD system 10 also provides a 3D cursor anywhere in the display volume around the image 34 in the same way that the viewer 12 moves his hand within a given space. In addition to 3D pointing devices to move the 3D mouse device, such as SPACE BALL available from Massachusetts, Lowell's Spacetec, Inc. Alternatively, via the user feedback device 58, the MVD system 10 may interpret the movement of the hand of the viewer 12 as a 3D cursor.

In one embodiment, the user feedback device 58 may include components for sensing the position and orientation of the hand of the viewer 12. For example, viewer 12 may hold or use a position sensor, such as a magnetic sensor available from POLYHEMUS, INC., And / or other forms of sensor, such as a position sensor integrated within a virtual reality data globe. Alternatively, the position of the hand is sensed within the display volume of the 3D image 34 by the use of radio frequency sensors such as computer image processing and sensors developed at MIT MEDIA LAB. To avoid forced action, the user feedback device 58 displays the displayed 3D image (in the same way as the 2D movement of a conventional 2D mouse on a flat surface of the desktop to control the position of the 2D cursor on the 2D screen of a personal computer). It is possible to detect the movement of the hand or finger of the viewer 12 in a fairly small sensing space physically separated from 34.

Advantages of MVD Systems

Using the MVD system 10, 3D images 34, 56 are generated to provide natural observation by the viewer 12, ie the 3D images 34, 56 virtually minimize eye strain and work. It has all the depth cues associated with the observation of the real thing, making it observable for a prolonged period of time.

The MVD system 10 is a high resolution / voxel coefficient by an MVD device 32 that provides at least one size dimension, for example, 3,000,000 or more voxel sizes, for many stereoscopic displays known in the art. To provide. In addition, by appropriately using the linear geometry for displaying the 3D image 34, such as the MOE device 32 having a rectangular cross section suitable for displaying the image slices 24-30 as the 2D images 44-50. The MVD system 10 provides a coordinate system that matches the internal coordinate system of various known graphics computers and graphics applications, which can easily maximize computer performance and display updater rate without requiring additional conversion software. I use it. Further, in the preferred embodiment, the image voxels of the MOE 32 are the same and constant in shape, size, and orientation, thus eliminating image distortions in the 3D image 34.

Unlike multiview autostereoscopic displays known in the art, the MVD system 10 provides vertical and horizontal parallax for a wide range of fields of view, in which the 3D image is surrounded by a multidimensional field of view rather than just one. Allow them to watch. Also, unlike the multiview auto stereoscopic display, the clock of the MVD system 10 is continuous in all directions, i.e., when the viewer 12 moves relative to the MOE device 32, it is embarrassing in the 3D image 34. There is no jump.

In addition, due to the static structure of the optical elements 36-42 in the MOE device 32, the image distortion of the MOE device 32, display vibration, And there are no moving parts leading to even fatal mechanical defects.

The MVD system 10 may avoid blocking, ie blocking by front objects of light emitted by the rear objects. A limited form of blocking, called computational blocking, can be produced by selecting a particular aspect to improve the image composition and the ratio of the display, and not showing a surface that is not visible from that aspect. However, when the viewer 12 tries to look at the front objects, rear objects not shown are not visible. In one embodiment, the MVD system 10 is interspersed with a scattering optical element that displays an image with other optical elements in a scattering state that absorbs back light to create a blocking state. To compensate for the lack of blocking. Guest-host polymer-dispersed liquid crystal is used in the optical elements 36-42, where the dye is mixed with the liquid crystal molecules and causes the color of the material to change by the applied voltage.

Since the use of the actual image projector 54 requires space to extend up to the MOE device 32, the MVD system 10 also has little contrast degradation due to the ambient lighting of the MVD system 10, which is a MOE The amount of ambient light reaching the device 32 is reduced, thereby preventing contrast degradation.

As an alternative, contrast degradation may increase the illumination from the image projector 20 in proportion to the ambient lighting, or absorbent plastic enclosure around the MOE device 32 to reduce image brightness to an observable level. Can be reduced by installing Ambient light must pass through the absorbing enclosure twice to reach the viewer 12, once inward and after interspersing the optical elements 36-42 of the MOE device 32. Conversely, light from the image projector 20 forming the image 44-50 passes through the absorption enclosure only in the direction of the viewer 12, thus reducing the light loss, which is due to ambient light. Is a function of square root of.

An alternative embodiment of reducing the influence of ambient light is an enclosure with three narrow spectral band passes of red, green, and blue, or a high absorption band of light that is highly effective in reducing the effects of such ambient light. Use of absorbent materials. Since the narrow band light from the laser light source passes through the MOE device 32 without being attenuated after scattering while most of the light band light from the ambient light is absorbed, an effective operation in consideration of the ambient light is performed by the laser in the image projector 20. Obtained by using a light source.

Anti-aliasing in MOE Devices

In another alternative embodiment, referring to FIG. 16 as described herein, before transmitting the image data to the optical elements 160-168 of the MOE device 32 after the image projector 20, the MVD controller (18) or alternatively, graphics data source 16 performs 3D anti-aliasing on the image data to planarize the shapes to be displayed within 3D image 34 on optical elements 160-168. Using 3D anti-aliasing, the system 10 displays pixelization due to inherent discrete voxels of the MOE device 32 with the optical elements 160-168 arranged from the xy plane to the z-axis, For example, it avoids imaging jagged lines or incomplete regions between parallel planes 162-164 along the z-axis direction.

As data corresponding to the image slice is generated, the image element 170 appears at the edge of the planar transformation, ie between the optical elements, eg, between the optical elements 162-164. For purposes of illustration only, the construction of the optical elements 160-168 and voxel 170 shown in FIGS. 16-18 have been exaggerated to more clearly describe and illustrate the disclosed anti-aliasing systems and methods, It can be seen that the optical elements 160-168 have relatively little space between them.

In order to prevent accidental changes in a 3D image composed of a particular image element 170 and a voxel and / or image element 170, they are projected onto the optical elements 162-164 from the projector 20 and described herein. Both slices are formed such that each of the images 172-174 on the optical elements 162-164 each includes an image element 170 or a portion or derivative thereof, so that the image element 170 is an optical element 162. Shared between both planes formed by -164, which smoothes the change and makes the 3D image 34 of FIG. 1 appear more continuous. The brightness of the image elements 172-174 on the continuous optical elements 162-164 varies with the position of the image elements 172-174 in the image data.

Referring to FIG. 16, the number N of the optical elements 160-168 are flat LCD surfaces, also classified as P1, P2, P3, ... PN, and reach a distance D which is the width of the MOE device 32. Thus, each of the optical elements 160-168 is positioned along the z-axis from the common reference point with distances D1, D2, D3, ... DN such that DN-D1 = D. For example, the common reference point is the optical element 160 adjacent to the projector 20 along the z axis, so that D1 = 0 and DN = D. As an alternative, the distance of the optical elements 160-168 is measured through the lens 22 of the projector 20, and the offset distance D _OFFSET from the optical element 160 and the lens 22 is from the optical element 160. Subtracted from the absolute distances D1, D2, D3, ... DN of the optical elements 160-168 from the lens 22 to obtain the relative distance. Therefore, D1 = D _OFFSET . Optical elements 160-168 may also be disposed at regular intervals, or alternatively, the spacing between optical elements 160-168 may vary.

As described herein, the depth value of each voxel 170 is measured along the z-axis at the lens 22 or from a reference point in the optical element 160, which depth value is associated with the associated color value stored in the color buffer. Along with the depth buffer. For example, the depth value Dv is associated with the voxel 170.

In order to effect anti-aliasing, and therefore to smooth the appearance of the voxel 170 lying between the optical elements 162-164, the distance DA between the depth value DV and the optical elements 162-164. , DB) are determined respectively, and this interval is used to generate anti-aliasing parameters. The anti-aliasing parameters are then used to generate two voxels 172-174 respectively on the optical elements 162-164, with the color values of the corresponding voxels 170 modified by the anti-aliasing parameters. To generate respective color values for the two voxels 172-174.

17 shows a voxel display without anti-aliasing. As shown in FIG. 17, voxels 176-178 on optical element 162 and voxels 180-184 on optical element 164 form sharp transitions at the boundaries defined by voxels 178-180. do. If the spacing between the optical elements 162-164 is large, the appearance of the image 34 may be markedly uneven or broken by the combination of the marked voxels 176-184. For example, the voxels 178-180 are closer to, for example, the voxels 178 but not closer to the optical element 162, and the voxels 180 not closer to the optical element 162 but closer. 164). These intermediate depth values may then be converted to discrete depth values D2 and D3 of the optical elements 162-164, respectively, to represent voxels 178-180. In addition, in FIG. 17, the color values of the voxels 178-180 do not change, and thus the intensity of the colors of the voxels 178-180 may be irregular for other optical paths. The transition voxels 178-180 may be omitted due to their intermediate depth, but the 3D image 34 composed of the voxels 176, 182-184 may appear to have holes or tears.

When using anti-aliasing, as shown in Figure 18, both transition voxels 178-180 can be used to generate new voxels 178A-178B, 180A-180B, which are voxels 178A-180A. Is displayed on the optical element 162 and voxels 178B-180B are displayed on the optical element 164. In addition, as shown in FIG. 18, the color values of the voxels 176 and 182-184 do not change, while by performing anti-aliasing, the color values of the new voxels are determined by the new voxels 178A-178B and 180A-180B. Each can be modified to have an adjusted color to smooth image transitions in the xy plane across different depths. Thus, as shown in FIG. 19, the voxels 176-184 have sharp transitions at sharp depths along the curve 176 for imaging in FIG. 17, while the voxels 176, 178A-178B, 180A in FIG. 18. -180B, 182-184 has a relatively smoother transition at a clear depth along the curve 188. For purposes of explanation only, the curves 186-188 are not superimposed in FIG. 18 for clarity but it should be understood that the clear depths of the voxels 176, 182-184 in FIG. 18 are the same with or without anti-aliasing. do.

In FIG. 19, the voxels 178A-178B of FIG. 18 are intermediate to the depths of the voxels 178A-178B and the optical elements 162-164 with a clear depth 178C that corresponds to the initial depth of the voxel 178 in FIG. 17. ) Form an image across but not closer to the optical element 162. Similarly, the voxels 180A-180B of FIG. 18 are intermediate to the depths of the voxels 180A-180B and the optical elements 162-164 are shown at a sharp depth 180C corresponding to the initial depth of the voxel 180 in FIG. 17. An image is formed across but not closer to the optical element 164.                 

Anti-aliasing is not limited to the two nearest adjacent optical elements, but the voxels 178-180 generate a plurality of corresponding voxels on each of the plurality of optical elements 160-168 and, for example, the curves of FIG. 188) can be used to provide a depth transition curve that can be flatter than 188). For example, the depth transition curve 188 by anti-aliasing can be approximated to a sigmoid or tangent function.

Referring to FIG. 16, in order to perform anti-aliasing on the voxel 170, at least one depth adjustment value λ is generated that is a function of the distance of the voxel 170 from the at least one optical element. In one embodiment, an adjustment value [lambda], [mu] may be generated that is a function of the scale value of the distances DA, DB from each optical element 162-164. These adjustments (λ, μ) are used to modify the color value (Cv) associated with the voxel 170 to generate new color values (CA, CB) associated with the newly generated voxels 172-174, respectively. The voxels 172-174 have respective xy positions on the optical elements 162-164 which are the same as the xy positions of the voxels 170.

The color value of the voxel may specify at least the brightness of the voxel to be displayed. In addition, the voxel 170 may be associated with a series of parameters including at least one scalar amount that specifies the brightness of the colored voxel. Therefore, the color value can be corrected by multiplying the color value and the adjustment value. For example, when the color value Cv = 12 brightness units and the adjustment value λ = .5, the corrected color value CA is determined to be Cvλ = (12 brightness units) × (.5) = 6 brightness units.

In one embodiment, the distance Dv is scaled to be a depth value of 1 to N, where N is the number of optical elements 160-168, and each integer value 1 to N is, for example, shown in FIG. The indices of the labels P1, P2, P3, ... PN shown in 16 correspond to specific elements of the optical elements 160-168. The adjustment values λ, μ are determined from the scaled depth values. If the optical elements 160-168 are evenly spaced at a constant spacing S along the distance D:

Therefore, the scaled distance of the voxel 170 is represented by Equation 2 below.

Here, Dv is an absolute distance measured from the lens 22 or another reference point. For example, if lens 22 is in z-axis orientation, optical element 160 may be at a distance D1 = D _OFFSET .

Since D _SCALED is a real value such that 1 ≦ D _SCALED ≦ N, the fractional portion of D _SCALED in the range between 0 and 1 represents the relative distance of the optical elements 162-164. In the case of the optical elements 162-164 bounding voxels 170 on either side along the z axis, the indices of the optical elements 162-164 are the following Equations 3 and 4, respectively. [X] is a floor or integer function of the value of variable X. That is, the function that makes the maximum integer less than X.

_Since the fractional part of D _SCALED is _expressed by Equation 5 below,

Color values CA and CB representing respective luminance associated with the voxels 172 and 174 are assigned to Equations 7 and 8, respectively.

Here, the sign ": =" represents the assignment of a new value.

For example, for voxel 170 with depth Dv = 9.2 units from lens 22, offset D _OFFSET = 3.0 units, and MOE device 32 has N = 5, D = 20, spacing S = 5 units, and D _SCALED = 2.24 according to equation (2), with five evenly spaced optical elements extending to the length of 20 units. The voxel 170 is located between the optical elements with exponent [D _SCALE ] = 2 and [D _SCALE ] + 1 = 3 as in Equations 3 and 4, and thus has labels P2 and P3 in FIG. Optical elements 162-164 are identified as optical elements as new voxels 172-174 are displayed corresponding to voxels 170.

In the present embodiment, from the equations (5) and (6), the fractional value of the scaled depth is lambda = .24, and mu = .76. Therefore, (1-λ) =. 76, (1-μ) =. 24, and from the equations (7) and (8), the color value of the voxel 172 is CA = .76 of the original voxel 170 luminance. , Cv = 76%, and the color values of voxel 174 are CB = .24 and Cv = 24% of the original voxel 170 luminance. Thus, because the voxel 170 is closer to the optical element 162 than to the optical element 164, the corresponding new voxels 172-174 have a distributed luminance, so that the close optical element 162 has two Most of the color is displayed between the voxels 172-174, and a small but not zero amount of the distant optical element 164 is due to the transition of the 3D stereoscopic image between the optical elements 162-164 in the voxel 170. Contribute to making it appear.

In the case of the voxel 170 where the depth value is accurate on the optical elements 160-168, anti-aliasing is not required. Therefore, equations (2) to (4) are changed to integer values, (5) and (6) result in adjustment values [lambda], and [mu] are 0 and 1 or 1 and 0 respectively, so that adjustments to the color values are made. Is not performed at all. In order to avoid unnecessary calculations, the MVD controller 18 may check whether the calculation in Equation 2 is an integer within a range of 1 percent, which is a predetermined error tolerance, and if so, the voxel 170 is optical. It is determined or considered to be exactly in one of the elements 160-168. The anti-aliasing procedure is terminated for the voxel 170 currently being processed, and the procedure may continue to process another voxel of the 3D image 34.

In this embodiment using Equations 1 to 8, constant intervals and other characteristics of the MOE device 32 are known, and the distance DV and MOE device characteristics of the voxel 170 are expressed by Equations 3 and 4 below. Because it determines which optical element will delimit the voxel 170, a search for the nearest adjacent optical element is not necessary.

In another alternative embodiment, for the optical elements 160-168 of the MOE device 32 having a constant spacing, or a variable and / or non-uniform spacing, the math described below in relation to Equations 7 and 8 Anti-aliasing can be performed using equations (9)-(13). For example, in the case of a MOE device having a variable spacing and / or a variable offset of the MOE device from the projector 20 and the lens 22, an on-the-fly manner during the modification of the spacing and configuration of the optical elements 160 to 168. The anti-aliasing method can be performed. In an alternative embodiment, since the distance / depth of the optical elements _160-168 may vary, the anti-aliasing method uses two adjacent optical elements having distance / depth values D _NEAR1 and D _NEAR2 , such as By searching for the depth value of each of the optical elements 160 to 168 with respect to, at least two optical elements that define the boundary of the voxel 170 currently being processed are determined.

The variable values D _NEAR1 and D _NEAR2 may be integer indexes that specify the relevant optical elements from the optical elements _160-168 . For example, in FIG. 16, D _NEAR1 = 2 and D _NEAR2 = 3 and correspond to the optical elements 162 to 164 that delimit the voxel 170 along the Z axis.

The depth adjustment values λ, μ are determined as follows.

는 값 또는 변수 X의 절대값 또는 크기 함수이다.here,

Is the absolute value or magnitude function of the value or variable X.

The depth adjustment values from equations (10) and (11) are both positive real numbers that satisfy the following expressions (12) and (13).

The depth adjustment value scales the irregularity and / or the variable distance between the optical elements, and then generates voxels 172 to 174 corresponding to the color values adjusted in Equations 7 and 8. As shown in equations (10) and (11), the depth adjustment values [lambda], [mu] are based on the interpolation of the depth of the voxel 170 within the depth range of the voxels 172 to 174 associated with the optical elements 162 to 164. .

In the above example with a constant interval, equations 9 to 13 are applied using D _V = 9.2 units, D _NEAR1 = D ₂ = 8 units, and D _NEAR2 = D ₃ = 13 units.

This is consistent with the adjustment value using the equations (1) through (8). Alternative embodiments are useful if the size and spatial characteristics of the MOE device 32 and the optical elements 160-168 vary, but determine suitable adjacent optical elements 162-164 to create new voxels 172-174. A search to do is required.

20 illustrates a method of implementing 3D anti-aliasing for reading the corresponding depth value DV and color value CV from depth and color buffer, respectively, in step 190 for the current voxel to be displayed, such as voxel 170. A flowchart is shown. The method may then determine whether the spacing between the optical elements is constant in step 192, for example, where the configuration settings of the MVD controller 18 are constant or non-uniform distributions. 160-168, and / or the MVD controller 18 and the MOE device 32 may indicate whether to operate in a variable interval mode, as described herein.

If the interval is constant, the method scales the depth value DV in step 194 to within the index range of the optical elements 160-168 using Equations 1 and 2, and then determines the nearest optical element and In (196), the boundary of the depth value DV is determined using Equation 3 and Equation 4. In other cases, if the spacing is not constant at step 192, in alternative embodiments, step 196 may be performed without step 194, ie searching through the distance / depth values of each of optical elements 160-168. The procedure can be used to determine the optical element that satisfies Equation (9). In another alternative method, step 192 may optionally be implemented or omitted, depending on the configuration and mode of operation of the MVD controller 18 and the MOE device 32.

The method then uses depth 5 and / or second depth adjustment values λ at step 198 using Equations 5 and 6 or Equations 10 and 11, according to embodiments implemented as described herein. Determine the value μ. The method then adjusts the color values in step 200 for the voxels on the nearest bounding optical element using depth adjustment values or values using equations (7) and (8), and then the method comprises steps (202). Display the adjusted voxel on the nearest optical element with the adjusted color value in.

In yet another alternative embodiment, a medium degree of anti-aliasing can be implemented. For example, the adjustment values λ, μ can be fixed, for example, to a value of 5 such that half of the brightness of the voxels 170 is assigned to each of the voxels 172-174. This intermediate anti-aliasing may result in an apparent depth such as the intermediate depth 180D corresponding to the intermediate transition curve, as shown by curve 189 in FIG. 19.

In another alternative embodiment, the degree of anti-aliasing is another one that utilizes all fractional depth values λ, μ, at one extreme, ie, at the extremes ignoring the fractional depth values λ, μ to assign color values. The extremes can vary, or the degree of anti-aliasing can vary to any value between these extremes. This variable anti-aliasing can be performed by dividing the fragment portion λ of the depth scaled by the anti-aliasing parameter P and negatively offset the final value therefrom. That is, after α is calculated in equations (5) and (10), the variable value λ _VAR is calculated as follows.

The final color value can be determined by fixing and clamping the negative offset value to be within a predetermined range, such as between 0 and 1. Thus, equations (7) and (8) are modified for variable anti-aliasing as follows.

20, steps 198-202 may execute equations (14) through (16), respectively, to provide variable anti-aliasing.

The anti-aliasing parameter of P = 1 corresponds to full anti-aliasing, and the anti-aliasing parameter of P → ∞ does not correspond to any anti-aliasing, which is infinite, which can be implemented computationally with any high number of values. . Anti-aliasing parameters of less than one may also be implemented. For example, when P = 1, anti-aliasing as described above with respect to Equations 1 to 13 is implemented.

In another embodiment, for an anti-aliasing of λ = .24 and an anti-aliasing parameter of 3, λ _VAR = .08 by equation (14), and therefore, as in equations (15) and (16), a voxel ( C _A2 = .92 C _V = 92% of the color value of 170), while C _B2 = .08 C _V = 8% of the color value of the voxel 170. Compared to many of the examples described above, this variable anti-aliasing increases the contribution of voxel 172 at apparent depth from 76% to 92%, while voxel 174 is 24% or about 1/4 to 10%. Has a reduced contribution to less than%. In another example, when P → ∞, anti-aliasing is removed, resulting in λ _VAR = 0.00 by equation (14). Thus, as in Equation 15 and Equation 16, C _A2 = 1.0 C _V = 100% of the color value of the voxel 170 while C _B2 = 0.00 C _V = 0% of the color value of the voxel 170. to be. Thus, any voxel 170 that lies between the optical elements 162-164 is displayed on the closer optical element 162 without anti-aliasing, and if P → ∞, a second voxel farther from the reference point. Step 202 in FIG. 20 may further include a step of not generating, and thus not displaying. For example, voxel 174 is not generated.

In another alternative embodiment using variable anti-aliasing, the method in FIG. 20 may include indicating a new voxel only when the adjusted color value is greater than a predetermined threshold T. FIG. For example,

to be.

For example, T can be 0.5, so that a color contribution of less than about 5% is achieved on optical elements 160-168, for example when voxels having such color values are switched to opaque / scattering mode. It can be considered negligible because it is displayed in. Thus, this negligible contribution to the entire 3D image is discarded, and non-contributing voxels are not displayed, which can reduce the number of voxels to be displayed and improve the computational processing of the 3D image.

In yet another embodiment, the MVD system 10 may generate a 3D image 34 having a translucent appearance of portions of the 3D image 34. That is, the images 44-50 displayed on the optical elements of the MOE device 32 have the appropriate shading and color so that a portion of one image appears translucent, while another portion of the second image renders the translucent portion. Appears to see through. Such translucent appearance can be created with or without anti-aliasing.

When generating the 3D image 34, the method used by the MVD system 10 may, for example, openGL frame buffer data, such as color and depth (or z) buffers of the frame buffer of the graphics data source 16. MPD calculation is performed. The value in the depth buffer is the depth of the corresponding pixel in the color buffer and is used to determine the location of the voxel or pixel, such as voxel 170 of FIG. 16, indicated in MOE device 32. This MPD calculation is suitable in situations where it is desirable that portions of the image of the background objects of the stereoscopic image 34 from the MOE device 32 are not represented if such images are blocked by the images of the front objects. In order to generate an image in the MOE device 32 where the images of the front objects are translucent so that an image corresponding to the blocked background object is shown, an alpha channel technique is used, where the parameter α (alpha) is dependent on the value of α. The color of the pixel / voxel in the color buffer is determined by combining the color of both the front and background objects. The total opacity is obtained by α = 1 and the total translucency is obtained by α = 0. While using such alpha channel imaging to produce color images from a color buffer that appears to be accurate, the depth values in the depth buffer do not change, and thus still correspond to the depth of the image of the foremost objects. In known display systems, unaltered depth prevents proper image display within a stereoscopic display system since there are several surfaces at various depths that can be displayed using only one depth value. This disclosed MVD system 10 avoids this impossible point of the prior art when displaying multiple surfaces at various depths for a single depth value, for example a stereoscopic image 34 having a translucent object or part thereof. ). This disclosed MVD system 10 uses additional properties of OpenGL to generate a clip plane located in model space of the MVD system 10, whereby the representation is for example the plus side opposite the minus side, It only needs to be created on the predetermined side of each clip plane.

As shown in FIGS. 21-24, for a MOE device 32 having N planes 204-212, which may be numbered indices 1 to N, and with a uniform spacing Δ therebetween, stereoscopic images Scenes such as (34) face each other, are separated by a distance Δ, and rendered N times in a clip plane centered on a predetermined MOE plane among the planes 204-212 in model space. Thus N different images are generated and the corresponding color buffer is retrieved from the frame buffer to be sent to the MVD controller 18. When sending a color buffer to the MVD controller 18 for display on the MOE device 32, the MVD system 10 has an inherent alpha value associated with the MOE device used to generate the 3D stereoscopic image 34. As such, the alpha channel will be turned off.

Rendering by the clip plane may be implemented without anti-aliasing as shown in FIGS. 21 and 22, where the clip planes 214-216 are used corresponding to image portions located closer to the viewer 218. With portions of the image 34 displayed on the first plane 206, portions of the image 34 are positioned on the first plane 206 located between the clip planes 214-216. Created and displayed. With the image portions between the clip planes 220-222 displayed on the second plane 208, the new portions of the image 34 are clipped for display on the second plane 208 away from the viewer 218. Created between the planes 220-222 and located between the clip planes 220-222.

In order to implement anti-aliasing in the above way using an alpha channel, other features of OpenGL, such as the ambient effect of implementing fog-like imaging used in anti-aliasing, are used. The fog characteristics allow the color of each image object to be determined by the density of the fog, combined with the depth of the model for the color of the fog and the depth range associated with the specified FAR and NEAR values for the fog.

Fog functions available in OpenGL include linear, exponential, and exponential square functions. The disclosed MVD system 10 may use such functions as well as combinations of such fog functions, such as the superposition of the linear fog functions 224-227 shown in FIGS. 23-24. In the exemplary embodiment shown in FIGS. 23-24. Each of the combinations of linear fog functions 224-227 starts with a value of 0 corresponding to the black setting at the near depth of the fog and corresponds to the true color setting at a distance of (FAR-NEAR) / 2 from the near depth position. Proceed in a linear fashion with a value of 1. The fog function then drops to zero at the far depth of the fog. In such a fog function, and with the clip plane separated by a distance of 2Δ from the center located in a given MOE plane, at model intervals at which the image 34 is displayed and spaced apart, the image 34 is rendered N times, each time Data from the color buffer is sent to the corresponding plane of the MOE device 32.

In an exemplary embodiment, the combination of linear fog functions and the processing of the voxel image data with such a combination are performed by synthesizing the image for a given optical element, such as plane 206 of FIG. 23, with at least two rendering paths. . During the first path, two clip planes are placed on first optical plane 228 and first clip plane 228 located on optical element 204 having an immediately rendered image before current optical element 206. With the second clip plane positioned there is separated by the distance Δ. The linear fog function 224 with increasing distance with NEAR less than FAR is used as the clip plane described above, which is rendered to the first set of images for the optical element 206. During the second path, the two clip planes are positioned on the optical element 208 with the first clip plane located on the current optical element 206 and the image immediately rendered after the current optical element 206. It has a clip plane 230 and a second clip plane located on the current optical element 206 and are separated by a distance Δ.

Two sets of images rendered with different linear fog functions 224-225 are added together to the MVD system 10 and displayed on the optical element 206.

For rendering to the first image on the first plane 206 as shown in FIG. 23, the fog functions 224-225 are centered relative to the first plane 206 and from the clip planes 228-230. The images of and the depths between them have their corresponding color values transformed into the corresponding values of the fog functions 224-225 at the associated depths. After rendering the added image on the optical element 206 using the functions 224-225, the MVD system 10 causes it to be rendered as a continuous image on the second plane, as shown in FIG. 226-227 are moved to the center of the second plane. The images from the clip planes 232-234 and the depths between them have their corresponding color values changed by the corresponding values of the fog function 226 at the associated depths. The MVD system 10 allows for continuous movement of the fog function and proceeds to the corresponding clip plane for color adjustment of each representative image using the alpha channel method. In alternative embodiments, the fog function may be implemented for different planes 204-212, such as depth perception of the 3D stereoscopic image 34 displayed with a higher fog density as the distance from the viewer 218 increases. Increase the effect.

For example, referring to FIG. 23, for image 236 at depth 238, denoted D and having respective color values Ci for each portion of the image, the value of fog function 224 at depth D. Since 240 is αD, the adjusted color value displayed for image 236 is αDCi. Since the color value Ci may be a depth adjusted color value as in Equation 7 and Equation 8 and / or Equation 15 to Equation 18, the alpha channel adjustment may be selectively implemented in step 200 of FIG. In addition, anti-aliasing using the above-described alpha channel technique may be performed.

In the above, a novel and inventive multi-planar stereoscopic display system 10 and a method of operation have been described using the preferred embodiment. However, various changes and substitutions may be made without departing from the scope of the present invention. For example, while the preferred embodiment has been described using planar optical elements, such as flat panel liquid crystal displays, it is also conceivable to use curved optical elements in the same manner as described above.                 

MVD system 10 is part of a serial application of U.S. Provisional Application No. 60 / 082,442, filed on April 20, 1998, and U.S. Patent No. 5,572,375, filed on Nov. 4, 1996, in co-pending. It may be implemented using the devices and methods disclosed in US patent application Ser. No. 08 / 743,483, filed and co-pending. The MVD system can be implemented using the apparatus and methods disclosed in US Patent Application 09 / 004,722, filed Jan. 8, 1998 and co-pending. Each of the foregoing provisional applications, applications, and patents is herein incorporated by reference. Accordingly, the present invention is described by way of explanation, not limitation.

Claims (46)

A system for displaying stereoscopic three dimensional images of a three dimensional scene having a virtual depth D _V , (a) a controller for generating data representing a series of two-dimensional slices of varying depths through the three-dimensional scene; (b) a multi-surface optical device having a physical depth D _D and comprising a plurality of optical elements; (c) an image projector for selectively projecting the series of two-dimensional slices onto each optical element of the multi-surface optical device in response to the data to display a first stereoscopic three-dimensional image viewed by the multi-surface optical device; The virtual depth D _V of the three-dimensional scene is mapped to correspond to the physical depth D _D of the multi-surface optical device; And (d) a projector for projecting the first stereoscopic three dimensional image formed on the multisurface optical device to produce a second stereoscopic three dimensional image visible in an outer space of the multisurface optical device And a stereoscopic three dimensional image of the three dimensional scene having a virtual depth D _V. The method of claim 1, wherein the virtual depth D _V of the 3D scene is the multi-displaying the three-dimensional three-dimensional image of a three-dimensional scene with a virtual depth D _V, is mapped to a linear relationship to the physical depth D _D of the surface optical device system. The method of claim 1, wherein the virtual depth D _V of the 3D scene is the multi-display a stereoscopic three-dimensional image of a three-dimensional scene on the physical depth D _D of the surface optical device with a virtual depth D _V, is mapped to a non-linear relation System. The method of claim 1, wherein the plurality of optical elements are systems for displaying a stereoscopic 3D image of a 3D scene with a virtual depth D _V, which is disposed at the same interval along the depth. The method of claim 1, wherein the plurality of optical elements are systems for displaying a stereoscopic 3D image of a 3D scene with a virtual depth D _V, which is arranged in a non-uniform spacing along the depth. 5. The method of claim 4 or 5, the depth in the first three-dimensional three-dimensional image can also display a three-dimensional three-dimensional image of a three-dimensional scene with a virtual depth D _V, is mapped to correspond to one of said plurality of optical elements System. In a system for displaying a stereoscopic three-dimensional image of a three-dimensional scene, (a) a controller for generating data representing a series of two-dimensional slices of varying depths through a selected portion of the three-dimensional scene, the selected portion corresponding to a maximum virtual depth value Z _MAX and a minimum virtual depth value Z _MIN ; (b) a multi-surface optical device having a plurality of arranged optical elements; And (c) an image projector for displaying a first stereoscopic three-dimensional image viewed in the multisurface optical device by projecting the series of two-dimensional slices onto each optical element of the multisurface optical device in response to the data. Including but not limited to: The slice associated with the minimum depth value Z _MIN is projected onto the front optical element of the multisurface optical device, the slice associated with the maximum depth value Z _MAX is projected onto the rear optical element of the multisurface optical device, and the remaining slices are each A system for displaying a stereoscopic three-dimensional image of a three-dimensional scene projected between optical elements of the. 8. The system of claim 7, wherein portions of the three-dimensional scene that exceed the maximum virtual depth value Z _MAX are displayed as two-dimensional images on a rear optical element. 8. The system of claim 7, wherein the values of Z _MAX and Z _MIN are mutually specified by a viewer. 8. The system of claim 7, wherein the values of Z _MAX and Z _MIN are specified by a viewer during application initiation. The device of claim 8, wherein the slice associated with the minimum depth value Z _MIN is projected onto the foremost optical element of the multisurface optical device and the slice associated with the maximum depth value Z _MAX is the last of the multisurface optical device. A system for displaying a stereoscopic three-dimensional image of a three-dimensional scene projected onto a room optical element. 9. The second stereoscopic three dimensional image according to claim 7 or 8, wherein the first stereoscopic three dimensional image formed on the multisurface optical device is projected to appear to float in space at an outer position of the multisurface optical device. A system for displaying stereoscopic three-dimensional images of a three-dimensional scene, further comprising a floating image generator for generating a digital image. In the method for generating a stereoscopic three-dimensional image of a three-dimensional scene, (a) generating data representing a series of two-dimensional slices of varying depths through a selected portion of the three-dimensional scene, the selected portion corresponding to a maximum depth value Z _MAX and a minimum depth value Z _MIN ; (b) projecting the series of two-dimensional slices on each optical element of the multi-surface optical device to display a first stereoscopic three-dimensional image as seen by the multi-surface optical device. Including but not limited to: The slice corresponding to the minimum depth value Z _MIN is projected onto the front optical element of the multi-surface optical device, and the slice generated at the maximum depth value Z _MAX is projected onto the rear optical element of the multi-surface optical device and the remaining slices Is a projection between each optical element, producing a stereoscopic three-dimensional image of a three-dimensional scene. The method of claim 13, wherein portions of the three-dimensional scene that exceed the maximum depth value Z _MAX are projected as two-dimensional images on a rear optical element. The method of claim 13, wherein the values of Z _MAX and Z _MIN are mutually specified by a viewer. The method of claim 13, wherein the values of Z _MAX and Z _MIN are specified by a viewer during application launch. 15. The device of claim 13 or 14, wherein a slice corresponding to the minimum depth value Z _MIN is projected onto the foremost optical element of the multi-surface optical device and the slice corresponding to the maximum depth value Z _MAX is the multi-surface optical device. A method of generating a stereoscopic three-dimensional image of a three-dimensional scene, projected onto the rearmost optical element of the. 15. The method of claim 13 or 14, wherein the first stereoscopic three-dimensional image is projected from the multisurface optical device to produce a second stereoscopic three-dimensional image that appears to float in space at an outer position of the multisurface optical device. The method of claim 3, further comprising a three-dimensional image of the three-dimensional scene. In a system for generating a stereoscopic three-dimensional image, (a) a controller that renders a three-dimensional scene to provide at least one psychological vision cue and generates a series of two-dimensional slices through the rendered three-dimensional scene; (b) an optical device into which the two-dimensional slice can be projected; (c) an image projector for selectively projecting the series of two-dimensional slices onto the optical device to produce a first stereoscopic three-dimensional image Including but not limited to: And the at least one psychological visual cue causes the apparent depth of the first stereoscopic three dimensional image to exceed the physical depth of the optical device. The system of claim 19, wherein the at least one psychological cue is provided by including shading in the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by including a shadow in the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by occluding a portion of the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by including a forced perspective in the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by reducing the resolution of a surface texture of a farther portion of the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by increasing the haziness of a distant portion of the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by shifting the color of the distant portion of the rendered three-dimensional scene towards blue. 20. The system of claim 19, wherein the at least one psychological cue is added by reducing the brightness of a distant portion of the rendered three dimensional scene. 20. The system of claim 19, wherein the at least one psychological cue is provided by reducing the focus of a farther portion of the rendered three dimensional scene. The system of claim 19, wherein the optical device comprises a plurality of optical elements. 30. The system of claim 29, wherein the plurality of optical elements are arranged along the depth axis. 20. The apparatus of claim 19, further comprising a stray image generator for projecting the first stereoscopic three-dimensional image from the optical device to produce a second stereoscopic three-dimensional image that appears to float in space at an outer location of the optical device. , A system for generating stereoscopic three-dimensional images. In the method for generating a stereoscopic three-dimensional image of a three-dimensional scene, (a) rendering the three-dimensional scene to provide at least one psychological visual cue; (b) generating image data corresponding to a two-dimensional image slice set through the three-dimensional scene; (c) providing the image projector with the image data corresponding to the two-dimensional slice set of the three-dimensional scene; (d) selectively projecting each of the two-dimensional slices from the image projector onto an optical device to produce a first stereoscopic three-dimensional image Including but not limited to: And wherein the at least one psychological visual cue causes the apparent depth of the first stereoscopic three dimensional image to exceed the physical depth of the optical device. 33. The method of claim 32, wherein the at least one psychological cue is provided by adding shading during the rendering of the three dimensional scene. 33. The method of claim 32, wherein the at least one psychological cue is provided by adding a shadow during the rendering of the three dimensional scene. 33. The method of claim 32, wherein the at least one psychological cue is provided by blocking portions of the scene during rendering of the three-dimensional scene. 33. The method of claim 32, wherein the at least one psychological cue is provided by adding forced perspective during the rendering of the three dimensional scene. 33. The method of claim 32, wherein the at least one psychological cue is provided by reducing the resolution of a surface texture of a farther portion of the three-dimensional scene during rendering. 33. The method of claim 32, wherein the at least one psychological cue is provided by increasing haze of a distant portion of the three-dimensional scene during rendering. 33. The method of claim 32, wherein the at least one psychological cue is provided by shifting the color of the farther portion of the three-dimensional scene towards blue during rendering. 33. The method of claim 32, wherein the at least one psychological cue is provided by reducing the brightness of a distant portion of the three-dimensional scene during rendering. 33. The method of claim 32, wherein the at least one psychological cue is provided by reducing the focus of a distant portion of the three-dimensional scene during rendering. 33. The method of claim 32, wherein the optical device comprises a plurality of optical elements. 43. The method of claim 42, wherein the plurality of optical elements are arranged along a depth axis. 33. The three-dimensional scene of claim 32, further comprising a stray image generator for projecting the first stereoscopic three-dimensional image to produce a second stereoscopic three-dimensional image that appears to float in space at an outer location of the optical device. To generate a stereoscopic three-dimensional image. In the method for generating a three-dimensional three-dimensional image using a multi-surface optical device comprising a plurality of optical elements, (a) rendering a three dimensional scene having a depth axis; (b) generating data corresponding to a set of two-dimensional image slices of varying depths through the three-dimensional scene, each of the two-dimensional image slices having a plurality of tridels each having a depth coordinate z; Defined by-; (c) mapping a depth coordinate z of each tridel to a corresponding depth coordinate z 'in the multi-surface optical device; (d) mapping a depth coordinate z 'of each tridel to at least one voxel having a depth coordinate k corresponding to one of the plurality of optical elements; And (e) selectively projecting the voxel at the depth coordinate k on a corresponding optical element to produce a first stereoscopic three dimensional image seen in the multi-surface optical device A method of generating a stereoscopic three-dimensional image using a multi-surface optical device comprising a plurality of optical elements, including. 46. The multisurface of claim 45, further comprising projecting the first stereoscopic three-dimensional image to form a second stereoscopic three-dimensional image visible in an outer space of the multisurface optical device. A method of generating stereoscopic three-dimensional images using an optical device.

Images