WO2017160484A1 - 3d display apparatus, method, and applications - Google Patents
3d display apparatus, method, and applications Download PDFInfo
- Publication number
- WO2017160484A1 WO2017160484A1 PCT/US2017/019064 US2017019064W WO2017160484A1 WO 2017160484 A1 WO2017160484 A1 WO 2017160484A1 US 2017019064 W US2017019064 W US 2017019064W WO 2017160484 A1 WO2017160484 A1 WO 2017160484A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- display screen
- display
- imaging component
- component
- lens
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/0875—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more refracting elements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/0006—Arrays
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
- G02B30/26—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type
- G02B30/27—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type involving lenticular arrays
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
- G02B30/34—Stereoscopes providing a stereoscopic pair of separated images corresponding to parallactically displaced views of the same object, e.g. 3D slide viewers
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0129—Head-up displays characterised by optical features comprising devices for correcting parallax
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0132—Head-up displays characterised by optical features comprising binocular systems
- G02B2027/0134—Head-up displays characterised by optical features comprising binocular systems of stereoscopic type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0132—Head-up displays characterised by optical features comprising binocular systems
- G02B2027/0136—Head-up displays characterised by optical features comprising binocular systems with a single image source for both eyes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0185—Displaying image at variable distance
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/0006—Arrays
- G02B3/0037—Arrays characterized by the distribution or form of lenses
- G02B3/0056—Arrays characterized by the distribution or form of lenses arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses
Definitions
- aspects and embodiments of the invention most generally pertain to visual display apparatus and systems, image display methods, and applications. More particularly, aspects and embodiments are directed to 3D near-eye display systems for Virtual Reality (VR) and Augmented Reality (AR) systems, methods pertaining thereto, and applications thereof.
- VR Virtual Reality
- AR Augmented Reality
- 3D near-eye displays for VR and/or AR applications use stereoscopic methods to realize 3D imaging effects by separately presenting two offset images to the left and right eyes of the viewer.
- one micro-display e.g., OLED, LCOS, etc.
- portable display such as a smartphone screen that is split into two sections, etc.
- display' is used as an image source along with suitable optics to form a virtual image viewable by each eye.
- Fig. 1 shows an example of a Google Cardboard VR viewer wherein a smartphone screen is split into two sections to provide two offset images.
- An image display is generally composed of an array of pixels (e.g., 1080 x 1200) as minimal display- source elements.
- the display and optics are located in front of each eye to form the near-eye display.
- the display and optics are typically not directly located in front of each eye; rather, the virtual image is re-directed to the eye through a partial reflector (or diffractive element functioning as a partial reflector) so that the eye can also see the real world.
- Figs. 2a and 2b respectively, illustrate near-eye displays for one eye of a viewer for VR and AR applications.
- Figs. 3a and 3b respectively, illustrate differences between natural viewing and stereo viewing.
- vergence and focal distance are equal.
- the viewer adjusts the vergence of the eyes to look at an object, and the eyes focus to sharpen the retinal image.
- vergence and accommodation are neurally coupled. Specifically, accommodative changes evoke changes in vergence (accommodative vergence), and vergence changes evoke changes in accommodation (vergence accommodation).
- Zone of Comfort As a relationship between distance of vergence and distance to the screen (accommodation distance).
- Certain VR/AR applications such as, e.g., a computing platform suggest that the vergence distance of the 3D image should be less than 0.5 meter so that the viewer's hand can reach it easily.
- the ZoC suggests that the accommodation distance of the virtual screen should be less than one (1) meter for 0.5 meter comfort vergence distance.
- an aspect of the invention is a 3D display apparatus.
- the 3D display apparatus includes a display screen component comprising a display screen pixel array adapted to display a display screen image; a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and a controllable movement component coupled to at least one of the imaging component and the display screen, wherein the imaging component and the display screen are controllably movable relative to each other, further wherein upon a controlled movement of the imaging component relative to the display screen, a location of the virtual or the real image along an optical axis is controllably changed.
- the embodied apparatus may include some or all of the following features, limitations, and/or characteristics alone or in combination as one skilled in the art would understand:
- microlens imaging component is movable along an optical, z, axis in front of the display screen
- microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen;
- microlens imaging component is in a fixed location and the display screen is movable; -wherein the display screen is movable along an optical, z, axis behind the microlens imaging component;
- the display screen is movable in a plane, x-y, perpendicular to an optical, z, axis behind the microlens imaging component;
- both the microlens imaging component and the display screen are movable
- microlens imaging component is movable along an optical, z, axis in front of the display screen, and the display screen is movable in a plane, x-y, perpendicular to an optical, z, axis behind the microlens imaging component;
- microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen, the display screen is movable along an optical, z, axis behind the microlens imaging component;
- each of the microlenses has a lateral dimension between 0.5x and 2x a lateral dimension of a single display pixel
- each of the microlenses has a lateral dimension that is equal to the lateral dimension of a single corresponding display pixel
- the pitch of the microlenses is equal to or larger than the lateral dimension of each of the microlenses.
- the near-eye display system includes a 3D display apparatus, comprising a display screen component comprising a display screen pixel array adapted to display a display screen image; a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and a controllable movement component coupled to at least one of the imaging component and the display screen, wherein the imaging component and the display screen are controllably movable relative to each other; and a viewing lens fixedly disposed adjacent the microlens imaging component, wherein a final virtual image of the display that is viewable by a viewer will be formed by the viewing lens at a controllably variable distance, Z )fur from the viewing lens relative to a movable position of the microlens imaging component, where Z ) , is the accommodation distance of the viewer
- the viewing lens is a positive lens system comprising at least a piece of positive lens including a Fresnel lens or a group of lenses to form a positive lens group; -wherein the movement of microlens imaging component along the z-axis is time modulated (z(t)) in the manner so that the corresponding final virtual images from the viewing lens can form a 3D image viewable by the viewer.
- An aspect of the invention is a method for displaying an image.
- the method includes the steps of providing a near-eye display system, comprising a near- eye display apparatus, further comprising a display screen component comprising a display screen pixel array adapted to display a display screen image; a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and a controllable movement component coupled to at least one of the imaging component and the display screen, wherein the microlens imaging component and the display screen are controllably movable relative to each other; and a viewing lens fixedly disposed adjacent the microlens imaging component, so as to form a final virtual image of the display at a controllably variable distance, Z)terrorism from the viewing lens; and moving the microlens imaging component relative to the display screen component to vary the distance, Z) chorus and thus control
- the embodied method may include some or all of the following steps, features, apparatus, components, limitations, and/or characteristics alone or in combination as one skilled in the art would understand: -further comprising time modulating the movement of microlens imaging component along the optical, z, axis (z(t)) in the manner so that the corresponding final virtual images from the viewing lens form a 3D image viewable by the viewer.
- Fig. 1 shows an example of a Google Cardboard VR viewer wherein a smartphone screen is split into two sections to provide two offset images, as known in the art.
- Figs. 2a and 2b respectively, illustrate conventional near-eye displays for one eye of a viewer for virtual reality (VR) and augmented reality (AR).
- VR virtual reality
- AR augmented reality
- Figs. 3a and 3b respectively, represent differences between natural viewing and stereo viewing.
- Figs. 4 schematically illustrates a 3D display apparatus according to an exemplary embodiment of the invention.
- Fig. 5a schematically shows the position of device components in a 3D near eye display VR system
- Fig. 5b schematically shows the position of device components in a 3D near eye display AR system, according to exemplary embodiments of the invention.
- Fig. 6 schematically illustrates a verification experiment set-up in accordance with embodiments of the invention.
- Fig. 7a shows a picture of a DSLR camera that was used to record virtual images from a display for a lens array at different distances to the display;
- Fig. 7b shows pictures taken with the camera at the best focus of the camera where the pixels (graininess) of the cubes in the pictures are resolvable when the pictures were zoomed in;
- Fig. 7c shows pictures that were taken with the camera at out of focus of the camera, where the pixels of the cubes in the pictures were blurred, according to illustrative embodiments of the invention.
- Fig. 8 shows a picture of an experimental setup of a near-eye display for one eye, in accordance with embodiments of the invention.
- Fig. 9a shows a picture of the camera at focus distance of 0.75 m
- Fig. 9b shows the best focused pictures (taken sequentially) of two cube targets - (left) and (right), through the Google Cardboard lens and the micro-lens array, in accordance with embodiments of the invention.
- Fig. 10a shows a picture of the camera at ⁇ focus
- Fig. 10(b) shows that the pixels of the pictures are blurred with the camera lens focusing at infinity, which indicates that the camera lens was defocused and the virtual images of the cubes were not at infinity in accordance with embodiments of the invention.
- Fig. 11a shows a picture of the camera at best focus with the microlens array 0.5 mm from the display screen
- Fig. 11(b) shows pictures of the cubes through the Goggle cardboard lens and the micro-lens array, indicating that the virtual images of the targets were re-located to 1.5 m away from the camera (or viewer's eye) by moving the micro-lens array from the screen 0.5 mm.
- Fig. 12a shows a picture of the camera at best focus with the microlens array 1.0 mm from the display screen
- Fig. 12(b) shows pictures of the cubes through the Goggle cardboard lens and the micro-lens array, indicating that the virtual images of the targets were re-located to 4 m away from the camera (or viewer's eye) by moving the micro-lens array from the screen 1 mm.
- Fig. 13a shows a picture of the camera at best focus at infinity with the microlens array 1.2 mm from the display screen;
- Fig. 13b shows pictures of the cubes through the Goggle cardboard lens and the micro-lens array, indicating that the virtual images of the targets were re-located to infinity by moving the micro-lens array from the screen 1.2 mm.
- Fig. 14 shows an alternative experimental set-up with a Fresnel lens used as the VR lens in this experiment.
- the same Nikon DSL camera was used for recording the results.
- Fig. 15a shows a picture of the camera at best focus at 1 m
- Fig. 15b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located at 1 m away from the camera (or viewer's eye).
- Fig. 16a shows a picture of the camera at best focus at 1.5 m when the micro-lens array was positioned 0.5 mm away from the display screen;
- Fig. 16b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located at 1.5 m away from the camera (or viewer's eye).
- Fig. 17a shows a picture of the camera at best focus at 3 m when the micro-lens array was positioned 1.0 mm away from the display screen;
- Fig. 17b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located at 3 m away from the camera (or viewer's eye).
- Fig. 18a shows a picture of the camera at best focus at close to infinity when the micro-lens array was positioned 1.2 mm away from the display screen;
- Fig. 18b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located essentially at infinity from the camera (or viewer's eye).
- a micro-lens array 402 comprising micro-lens elements (or lenslets) 402 n is located in front of (to the left of) a display 404 including display pixels 404 n .
- the micro-lens elements may all be the same or not.
- a precision actuator 406 is provided to move the micro-lens array 402 relative to the display 404.
- the actuator may be, e.g., one or more MEMS or silicon-based devices, piezo-based devices, memory alloy-based devices, VCM (Voice Coil Motor) or VCA (Voice Coil Actuator)-based devices, stepper motor-based devices, and others known in the art.
- the lateral dimensions of each micro-lens element, as well as the array pitch, are advantageously the same (e.g., 1 pixel) or may be a bigger (e.g., up to 2 pixels) or smaller (e.g., down to 0.5 pixel) than the lateral dimensions of each display pixel.
- each pixel's lateral (x by y) dimensions in an illustrative OLED micro-display are 10 x 10 ⁇
- each micro- lens element's dimensions could also be 10 x 10 ⁇ (or up to 20 x 20 ⁇ , or down to 5 x 5 ⁇ ).
- the micro-lens array is mounted on the actuator (or other appropriate movement device) to enable precise movement along the optical path (z) direction. Assuming the effective focal length of each lenslet is f a , F a is the focal point of a lenslet along its own optical axis.
- the distance, d 0 between the display and micro-lens array may be larger than the focal length, f, of each lenslet for a real image, or smaller than f for a virtual image of each display pixel that is formed by the lenslet in front of it.
- f focal length
- f a virtual image of each display pixel that is formed by the lenslet in front of it.
- the virtual image 408 of the entire display pixel array is also formed at a distance, d t , from the display through the lens array.
- d 0 changes in the z-direction and the virtual image of the display pixel array also moves (dj changes).
- the relationships of location of each component in Fig. 4 are: 1 1
- FIG. 5a shows the position of each device/component in a VR system 500-1 that follows the relationships below, assuming the VR lens has an effective focal length of fi, and the focal point is at F b , its distance to the display is D 0 , and the final virtual image of display is at distance Z), from the VR lens.
- Equation (2) shows that when the micro-lens array is moved (d 0 changes), d t also changes, and Z), also changes (Equation (4)), where Z), is the accommodation distance of the viewer's eye if we ignore the distance between the viewer's eye to the VR lens since they are very close.
- Optical imaging systems of some current commercial VR systems are very simple; they comprise just two identical positive lenses as VR lenses, one for each eye of the viewer, in front of a micro-display or smart phone screen that functions as the image source display (e.g., Google Cardboard VR system shown in Fig. 1).
- a lens from an off-the-shelf Google Cardboard VR system was chosen as the VR lens for the first verification experiment, whose results are shown below in Experiment I.
- an off-the-shelf Fresnel lens was used for the second verification experiment, whose results are presented in Experiment II below.
- a schematic diagram of the verification experiment set-up is shown in Fig.
- each lenslet in the lens array 604 is 0.1 x 0.1 mm, while the size of the display pixel 607 is about 0.08 x 0.08 mm; thus each lenslet is slightly larger than each pixel on the display.
- the focal length, ,, of each lenslet is 2.8 mm.
- Each lenslet has a single side spherical convex surface (as illustrated) and is advantageously made of polymer having a thickness of 0.02 to 0.2 mm on a glass substrate with thickness of 2 mm.
- the lenslet side of the array was facing the display as illustrated.
- the overall size of the array is 50.8 x 50.8 mm.
- the lens array was mounted on a precision translation stage so that its distance to the screen could be adjusted. Because there is a touch screen glass in front of the display pixels in an iPhone 6, we can assume the optical equivalent distance from the front end of the touch screen to the display pixels is d e , which was estimated to be ⁇ 1 mm.
- a DSLR camera Nakon D610 with a Nikon Zoom lens (18-200 mm) shown in Fig. 7a was used to record the virtual images from the display for the lens array at different distances to the display.
- FIG. 7b shows that the pictures were taken at the best focus of the camera where the pixels (graininess) of the cubes in the pictures are resolvable when the pictures were zoomed in.
- Fig. 7c shows that the pictures were taken at out of focus of the camera, where the pixels of the cubes in the pictures were blurred.
- Figure 8 shows a picture of an experimental setup of a near-eye display for one eye.
- the Google Cardboard lens was from BrizTech Ltd and purchased from Amazon.com.
- the lens was mounted on a multi-axis precision micro-stage for alignment with the cube target on the display and distance adjustment to the display screen.
- the lens has a focal length, f b , of 45 mm and a diameter of 25 mm.
- the entire setup follows the schematic diagram shown in Fig. 6 and represents a conventional VR near eye display if without the micro-lens array that is shown in Fig. 2a.
- Fig. 9(a) shows the lens window of the DSLR camera for the focusing distance, which is at about 0.75 m
- Fig. 9(b) shows two pictures taken one at a time of two cubes with the camera at the best focus.
- the pictures were zoomed in and cropped so that the pixels could be seen on the pictures. This indicates that the virtual images of the cubes on the display screen were located at about 0.75 m away from the camera (or viewer's eye) when they were seen through the Google Cardboard lens and the micro-lens array.
- Figs. 10(a) and (b) show that the pixels of the pictures are blurred with the camera lens focusing at infinity, which indicates that the camera lens was defocused and the virtual images of the cubes were not at ⁇ .
- Figs. 1 1a and 1 lb indicate that the virtual images of the targets were re-located to 1.5 m away from the camera (or viewer's eye) by simply moving the micro-lens array 0.5 mm away from the display screen.
- Figs. 12a and 12b The pictures indicate that the positions of the virtual images of targets on the display screen were changed to 4 m away from the camera (or viewer's eye) by moving the micro-lens array from the screen 1 mm.
- Fig. 13a shows that the position of the virtual images of the targets on the display screen has changed to ⁇ .
- This experiment shows that the positions of the virtual images of targets on the display screen can be changed from 0.75 m to infinity by just moving the micro-lens array 1.2 mm away from the screen using this experimental setup.
- the movement ranges of the virtual images and the micro-lens array depend on the optical power of the VR lens and the lenslets of the micro-lens array that follow Equation (1) - (4). Generally, the higher the optical power of the VR lens and/or the lenslets, the shorter the movement of the micro-lens array for the same movement range of the virtual images.
- Figs. 18a and 18b show the results, which indicate that the virtual images were now located at close to ⁇ .
- accommodation distances (Z ) - locations of the virtual images that the viewer will see
- Z accommodation distances
- An ima display can be expressed as where /, ⁇ ,, y t) represents the light intensity on the i pixel at location (xicide y,) and time t. (R, G, B) represent red, green and blue colors respectively.
- the image on the display is represented in 2D (x and y) dimensions (perpendicular to the optical, z, axis).
- I j(Xj,yj,z(t), t) represents the light intensity of the virtual image of the i th pixel on the display at location (x h y h z(t)) and time t.
- the image is now represented in the z dimension (also the optical axis) as well as the x and y dimensions. It is possible to time-modulate the micro-lens array movement so that z(t) can be changed in a manner to make a true 3D virtual image of display expressed in Equation (6).
- the lens elements will collect more light from the display pixels, so the brightness of the near-eye display will be improved, which is very helpful for AR applications.
- SDE screen-door effect
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Devices For Indicating Variable Information By Combining Individual Elements (AREA)
Abstract
A 3D display apparatus and method that address the vergence-accommodation conflict. A display screen component includes a display screen pixel array adapted to display a display screen image, a microlens imaging component including an array of microlenses corresponding to the display screen pixel array that can form a virtual or a real image of the display screen image, and a controllable movement component coupled to the imaging component or the display screen, wherein the imaging component and the display screen are controllably movable relative to each other, further wherein upon a controlled movement of the imaging component relative to the display screen, a location of the virtual or the real image along an optical axis is controllably changed.
Description
3D DISPLAY APPARATUS, METHOD, AND APPLICATIONS
Related Application data
The instant application claims priority to US provisional application No. 62/308,308 filed on March 15, 2016 and US provisional application No. 62/333,420 filed on May 9, 2016, the subject matters of which are incorporated by reference herein in their entireties.
Background
Aspects and embodiments of the invention most generally pertain to visual display apparatus and systems, image display methods, and applications. More particularly, aspects and embodiments are directed to 3D near-eye display systems for Virtual Reality (VR) and Augmented Reality (AR) systems, methods pertaining thereto, and applications thereof.
Conventional 3D near-eye displays for VR and/or AR applications use stereoscopic methods to realize 3D imaging effects by separately presenting two offset images to the left and right eyes of the viewer. Typically, one micro-display (e.g., OLED, LCOS, etc.) or portable display (such as a smartphone screen that is split into two sections, etc.) (hereinafter•display') is used as an image source along with suitable optics to form a virtual image viewable by each eye. Fig. 1 shows an example of a Google Cardboard VR viewer wherein a smartphone screen is split into two sections to provide two offset images.
An image display is generally composed of an array of pixels (e.g., 1080 x 1200) as minimal display- source elements. For VR applications, the display and optics are located in front of each eye to form the near-eye display. For AR applications, the display and optics are typically not directly located in front of each eye; rather, the virtual image is re-directed to the eye through a partial reflector (or diffractive element functioning as a partial reflector) so that the eye can also see the real world. Figs. 2a and 2b, respectively, illustrate near-eye displays for one eye of a viewer for VR and AR applications.
During stereoscopic viewing the viewer is presented with two near-eye displays; one for the left eye and another for the right eye. Both eyes focus on each virtual screen, respectively, to see two offset images whose optical paths cross to form a 3D image. Figs. 3a and 3b, respectively, illustrate differences between natural viewing and stereo viewing. In natural viewing, vergence and focal distance are equal. The viewer adjusts the vergence of the eyes to look at an object, and the eyes focus to sharpen the retinal image. Because of the tight correlation in natural viewing, vergence and accommodation are neurally coupled. Specifically, accommodative changes evoke changes in vergence (accommodative vergence), and vergence changes evoke changes in accommodation
(vergence accommodation). In stereo viewing on a conventional stereo display, the focal distance is fixed at the distance from the eyes to the display screen, while vergence distance varies depending on the distance being simulated on the display. Thus a vergence-accommodation conflict is created when viewing a stereo display. To see the object clearly and without double vision the viewer must counteract the neural coupling between vergence and accommodation to accommodate to a different distance than the distance at which the eyes must converge. Visual fatigue and discomfort occur as the viewer attempts to adjust vergence and accommodation appropriately.
The literature describes a Zone of Comfort (ZoC) as a relationship between distance of vergence and distance to the screen (accommodation distance). Certain VR/AR applications such as, e.g., a computing platform suggest that the vergence distance of the 3D image should be less than 0.5 meter so that the viewer's hand can reach it easily. The ZoC suggests that the accommodation distance of the virtual screen should be less than one (1) meter for 0.5 meter comfort vergence distance.
Currently available VR/AR devices either do not or cannot meet this requirement. The inventor has recognized the advantages and benefits provided by a solution to the vergence-accommodation problem, said solution being enabled by the invention claimed herein.
Summary
An aspect of the invention is a 3D display apparatus. In a non-limiting, exemplary embodiment the 3D display apparatus includes a display screen component comprising a display screen pixel array adapted to display a display screen image; a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and a controllable movement component coupled to at least one of the imaging component and the display screen, wherein the imaging component and the display screen are controllably movable relative to each other, further wherein upon a controlled movement of the imaging component relative to the display screen, a location of the virtual or the real image along an optical axis is controllably changed. In various non-limiting, exemplary aspects the embodied apparatus may include some or all of the following features, limitations, and/or characteristics alone or in combination as one skilled in the art would understand:
-wherein the display screen is in a fixed location and the microlens imaging component is movable;
-wherein the microlens imaging component is movable along an optical, z, axis in front of the display screen;
-wherein the microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen;
-wherein the microlens imaging component is in a fixed location and the display screen is movable;
-wherein the display screen is movable along an optical, z, axis behind the microlens imaging component;
-wherein the display screen is movable in a plane, x-y, perpendicular to an optical, z, axis behind the microlens imaging component;
-wherein both the microlens imaging component and the display screen are movable;
-wherein the microlens imaging component is movable along an optical, z, axis in front of the display screen, and the display screen is movable in a plane, x-y, perpendicular to an optical, z, axis behind the microlens imaging component;
-wherein the microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen, the display screen is movable along an optical, z, axis behind the microlens imaging component;
-wherein each of the microlenses has a lateral dimension between 0.5x and 2x a lateral dimension of a single display pixel;
-wherein each of the microlenses has a lateral dimension that is equal to the lateral dimension of a single corresponding display pixel;
-wherein the pitch of the microlenses is equal to or larger than the lateral dimension of each of the microlenses.
An aspect of the invention is a near-eye display system. In a non-limiting, exemplary embodiment the near-eye display system includes a 3D display apparatus, comprising a display screen component comprising a display screen pixel array adapted to display a display screen image; a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and a controllable movement component coupled to at least one of the imaging component and the display screen, wherein the imaging component and the display screen are controllably movable relative to each other; and a viewing lens fixedly disposed adjacent the microlens imaging component, wherein a final virtual image of the display that is viewable by a viewer will be formed by the viewing lens at a controllably variable distance, Z)„ from the viewing lens relative to a movable position of the microlens imaging component, where Z), is the accommodation distance of the viewer's eye. In various non-limiting, exemplary aspects the embodied system may include some or all of the following features, apparatus, components, limitations, and/or characteristics alone or in combination as one skilled in the art would understand:
-wherein the viewing lens is a positive lens system comprising at least a piece of positive lens including a Fresnel lens or a group of lenses to form a positive lens group;
-wherein the movement of microlens imaging component along the z-axis is time modulated (z(t)) in the manner so that the corresponding final virtual images from the viewing lens can form a 3D image viewable by the viewer.
An aspect of the invention is a method for displaying an image. In a non-limiting, exemplary embodiment the method includes the steps of providing a near-eye display system, comprising a near- eye display apparatus, further comprising a display screen component comprising a display screen pixel array adapted to display a display screen image; a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and a controllable movement component coupled to at least one of the imaging component and the display screen, wherein the microlens imaging component and the display screen are controllably movable relative to each other; and a viewing lens fixedly disposed adjacent the microlens imaging component, so as to form a final virtual image of the display at a controllably variable distance, Z)„ from the viewing lens; and moving the microlens imaging component relative to the display screen component to vary the distance, Z)„ and thus controllably change the
accommodation distance of the viewer's eye. In various non-limiting, exemplary aspects the embodied method may include some or all of the following steps, features, apparatus, components, limitations, and/or characteristics alone or in combination as one skilled in the art would understand: -further comprising time modulating the movement of microlens imaging component along the optical, z, axis (z(t)) in the manner so that the corresponding final virtual images from the viewing lens form a 3D image viewable by the viewer.
Brief Description of the Drawings
Fig. 1 shows an example of a Google Cardboard VR viewer wherein a smartphone screen is split into two sections to provide two offset images, as known in the art.
Figs. 2a and 2b, respectively, illustrate conventional near-eye displays for one eye of a viewer for virtual reality (VR) and augmented reality (AR).
Figs. 3a and 3b, respectively, represent differences between natural viewing and stereo viewing.
Figs. 4 schematically illustrates a 3D display apparatus according to an exemplary embodiment of the invention.
Fig. 5a schematically shows the position of device components in a 3D near eye display VR system; Fig. 5b schematically shows the position of device components in a 3D near eye display AR system, according to exemplary embodiments of the invention.
Fig. 6 schematically illustrates a verification experiment set-up in accordance with embodiments of the invention.
Fig. 7a shows a picture of a DSLR camera that was used to record virtual images from a display for a lens array at different distances to the display; Fig. 7b shows pictures taken with the camera at the best focus of the camera where the pixels (graininess) of the cubes in the pictures are resolvable when the pictures were zoomed in; Fig. 7c shows pictures that were taken with the camera at out of focus of the camera, where the pixels of the cubes in the pictures were blurred, according to illustrative embodiments of the invention.
Fig. 8 shows a picture of an experimental setup of a near-eye display for one eye, in accordance with embodiments of the invention.
Fig. 9a shows a picture of the camera at focus distance of 0.75 m; Fig. 9b shows the best focused pictures (taken sequentially) of two cube targets - (left) and (right), through the Google Cardboard lens and the micro-lens array, in accordance with embodiments of the invention.
Fig. 10a shows a picture of the camera at∞ focus; Fig. 10(b) shows that the pixels of the pictures are blurred with the camera lens focusing at infinity, which indicates that the camera lens was defocused and the virtual images of the cubes were not at infinity in accordance with embodiments of the invention.
Fig. 11a shows a picture of the camera at best focus with the microlens array 0.5 mm from the display screen; Fig. 11(b) shows pictures of the cubes through the Goggle cardboard lens and the micro-lens array, indicating that the virtual images of the targets were re-located to 1.5 m away from the camera (or viewer's eye) by moving the micro-lens array from the screen 0.5 mm.
Fig. 12a shows a picture of the camera at best focus with the microlens array 1.0 mm from the display screen; Fig. 12(b) shows pictures of the cubes through the Goggle cardboard lens and the micro-lens array, indicating that the virtual images of the targets were re-located to 4 m away from the camera (or viewer's eye) by moving the micro-lens array from the screen 1 mm.
Fig. 13a shows a picture of the camera at best focus at infinity with the microlens array 1.2 mm from the display screen; Fig. 13b shows pictures of the cubes through the Goggle cardboard lens and the micro-lens array, indicating that the virtual images of the targets were re-located to infinity by moving the micro-lens array from the screen 1.2 mm.
Fig. 14 shows an alternative experimental set-up with a Fresnel lens used as the VR lens in this experiment. The same Nikon DSL camera was used for recording the results.
Fig. 15a shows a picture of the camera at best focus at 1 m; Fig. 15b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located at 1 m away from the camera (or viewer's eye).
Fig. 16a shows a picture of the camera at best focus at 1.5 m when the micro-lens array was positioned 0.5 mm away from the display screen; Fig. 16b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located at 1.5 m away from the camera (or viewer's eye).
Fig. 17a shows a picture of the camera at best focus at 3 m when the micro-lens array was positioned 1.0 mm away from the display screen; Fig. 17b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located at 3 m away from the camera (or viewer's eye).
Fig. 18a shows a picture of the camera at best focus at close to infinity when the micro-lens array was positioned 1.2 mm away from the display screen; Fig. 18b shows pictures of the cubes through the Fresnel lens and the micro-lens array, indicating that the virtual image of the display screen was located essentially at infinity from the camera (or viewer's eye).
Detailed Description of Non-limiting, Exemplary Embodiments
Aspects and embodiments of the invention describe 3D near-eye display apparatus for VR and/or AR applications and methods that eliminate or mitigate accommodation and vergence conflict, and which may be used to generate true 3D images. Referring to Fig. 4, a micro-lens array 402 comprising micro-lens elements (or lenslets) 402n is located in front of (to the left of) a display 404 including display pixels 404n. The micro-lens elements may all be the same or not. A precision actuator 406 is provided to move the micro-lens array 402 relative to the display 404. The actuator may be, e.g., one or more MEMS or silicon-based devices, piezo-based devices, memory alloy-based devices, VCM (Voice Coil Motor) or VCA (Voice Coil Actuator)-based devices, stepper motor-based devices, and others known in the art. The lateral dimensions of each micro-lens element, as well as the array pitch, are advantageously the same (e.g., 1 pixel) or may be a bigger (e.g., up to 2 pixels) or smaller (e.g., down to 0.5 pixel) than the lateral dimensions of each display pixel. For example, if each pixel's lateral (x by y) dimensions in an illustrative OLED micro-display are 10 x 10 μπι, then each micro- lens element's dimensions could also be 10 x 10 μπι (or up to 20 x 20 μπι, or down to 5 x 5 μπι). The micro-lens array is mounted on the actuator (or other appropriate movement device) to enable precise movement along the optical path (z) direction. Assuming the effective focal length of each lenslet is fa, Fa is the focal point of a lenslet along its own optical axis. The distance, d0, between the display and micro-lens array may be larger than the focal length, f, of each lenslet for a real image, or smaller than f for a virtual image of each display pixel that is formed by the lenslet in front of it. Here we only consider the case of a virtual image when d0 < f; then the virtual image 408 of the entire display pixel array is also formed at a distance, dt, from the display through the lens array. With movement of the lens array, d0 changes in the z-direction and the virtual image of the display pixel array also moves (dj changes). The relationships of location of each component in Fig. 4 are:
1 1
L + d ~ ! ,
(2)
When the display 404 is placed in the near-eye display system 500-1, 500-2, shown in Fig. 5 for (a) VR and (b) AR, with the movement of the micro-lens array 402, the location of the final virtual image that will be seen by the viewer also changes along the optical path direction, such that the
accommodation distance of the viewer's eye also changes. Fig. 5a shows the position of each device/component in a VR system 500-1 that follows the relationships below, assuming the VR lens has an effective focal length of fi, and the focal point is at Fb, its distance to the display is D0, and the final virtual image of display is at distance Z), from the VR lens.
D + d ≤ f
0 i ' b (3)
D0 + dj fb
(4)
Equation (2) shows that when the micro-lens array is moved (d0 changes), dt also changes, and Z), also changes (Equation (4)), where Z), is the accommodation distance of the viewer's eye if we ignore the distance between the viewer's eye to the VR lens since they are very close. In addition, due to the reciprocal relationship in Equations (2) and (4), if d0 was set to be close to fa and (D0 + d,) close to fi,, then a small movement (e.g., two to several microns to several mm depending on fa or /and ¾,) of the micro-lens array (small change of d0) can cause a large change (e.g., fractions of up to 10s of m) in Z),. Fig. 5b shows the position of each device/component in an AR system 500-2 that will also follow the same relationships in Equations (1) to (4) with D0 = Dj + D2.
Two experimental examples are presented below.
Experimental verifications
Optical imaging systems of some current commercial VR systems are very simple; they comprise just two identical positive lenses as VR lenses, one for each eye of the viewer, in front of a micro-display or smart phone screen that functions as the image source display (e.g., Google Cardboard VR system shown in Fig. 1). A lens from an off-the-shelf Google Cardboard VR system was chosen as the VR lens for the first verification experiment, whose results are shown below in Experiment I. Because some current VR systems also use Fresnel lenses as VR lenses for the viewer, an off-the-shelf Fresnel lens was used for the second verification experiment, whose results are presented in Experiment II below.
A schematic diagram of the verification experiment set-up is shown in Fig. 6, where the VR lens 602 was an off-the-shelf Google Cardboard lens in Experiment I and an off-the -shelf Fresnel lens in Experiment II. An off-the-shelf micro-lens array 604 (produced by RPC Photonics, part number: MLA-S 100-f28-A-2S) was placed in front of (to the left of) an Apple iPhone 6, which was used as the display 606. The size of each lenslet in the lens array 604 is 0.1 x 0.1 mm, while the size of the display pixel 607 is about 0.08 x 0.08 mm; thus each lenslet is slightly larger than each pixel on the display. The focal length, ,, of each lenslet is 2.8 mm. Each lenslet has a single side spherical convex surface (as illustrated) and is advantageously made of polymer having a thickness of 0.02 to 0.2 mm on a glass substrate with thickness of 2 mm. The lenslet side of the array was facing the display as illustrated. The overall size of the array is 50.8 x 50.8 mm. The lens array was mounted on a precision translation stage so that its distance to the screen could be adjusted. Because there is a touch screen glass in front of the display pixels in an iPhone 6, we can assume the optical equivalent distance from the front end of the touch screen to the display pixels is de , which was estimated to be < 1 mm. Because each lenslet in the lens array has a long focal length (fa = 2.8 mm), in both experiments the distance, d0 , between the lens array and the screen of the display was changed from 0 to 1.2 mm to ensure d0 <fa in Equation (1) for the virtual image case; here d0 = d0 + de , while the distance, D0 , between the VR lens and the display screen was kept unchanged. A DSLR camera (Nikon D610 with a Nikon Zoom lens (18-200 mm)) shown in Fig. 7a was used to record the virtual images from the display for the lens array at different distances to the display. The distances, Z)„ of the virtual images to the VR lens were reflected by the best focal distances, Df, of the DSLR camera to take the pictures. Here Df~ Z), since the camera was placed very close (almost touching) to the VR lens. Df is also the accommodation (or focal) distance of a viewer's eye when the viewer is seeing the virtual image of the display through the VR lens. In the experiments two cube images with different colors were generated, one at a time, on the display for the camera to take their virtual images (one at a time) through the VR lens and lens array. Figs. 7b and 7c show two examples of pictures acquired by the DSLR camera. Fig. 7b shows that the pictures were taken at the best focus of the camera where the pixels (graininess) of the cubes in the pictures are resolvable when the pictures were zoomed in. Fig. 7c shows that the pictures were taken at out of focus of the camera, where the pixels of the cubes in the pictures were blurred.
Experiment I - Google Cardboard lens as VR lens
Figure 8 shows a picture of an experimental setup of a near-eye display for one eye. The Google Cardboard lens was from BrizTech Ltd and purchased from Amazon.com. The lens was mounted on a multi-axis precision micro-stage for alignment with the cube target on the display and distance adjustment to the display screen. The lens has a focal length, fb, of 45 mm and a diameter of 25 mm.
The entire setup follows the schematic diagram shown in Fig. 6 and represents a conventional VR near eye display if without the micro-lens array that is shown in Fig. 2a.
As described above, in the experiments the distance between the micro-lens array and the display screen, d0 , was changed while keeping the distance, D0 , between the Google Cardboard lens and the display screen unchanged. Due to the focal length of the VR lens, fi, = 45 mm, D0 was set at 42 mm to ensure it met the requirement of Equation (3). Using the Nikon DSL camera to view the cube image on the screen (one cube at a time) through the Google Cardboard lens and the micro-lens array, the camera was adjusted to best focus to take the pictures of the cubes (one at a time). The best focus distance of the camera was recorded. Following are the details of the experiments.
1. Move the micro-lens array so that it touches the display screen (d0 = 0). Take pictures of the cubes (one at a time) with the best focus and record the focusing distance. Fig. 9(a) shows the lens window of the DSLR camera for the focusing distance, which is at about 0.75 m, and Fig. 9(b) shows two pictures taken one at a time of two cubes with the camera at the best focus.
To show the details, the pictures were zoomed in and cropped so that the pixels could be seen on the pictures. This indicates that the virtual images of the cubes on the display screen were located at about 0.75 m away from the camera (or viewer's eye) when they were seen through the Google Cardboard lens and the micro-lens array.
For comparison, the pictures of the same cubes were also taken using the camera with the lens focused at infinity (∞, not the best focus), which are shown in Figs. 10(a) and (b). Fig. 10(b) shows that the pixels of the pictures are blurred with the camera lens focusing at infinity, which indicates that the camera lens was defocused and the virtual images of the cubes were not at∞ .
2. Move the micro-lens array 0.5 mm away from the iPhone screen (d0 = 0.5 mm) and keep all other parameters the same. Adjust the camera to the best focus and take pictures of the cubes through the Goggle cardboard lens and the micro-lens array, whose results are shown in Figs. 1 1a and 1 lb. Figs. 1 la and 1 lb indicate that the virtual images of the targets were re-located to 1.5 m away from the camera (or viewer's eye) by simply moving the micro-lens array 0.5 mm away from the display screen.
3. Move the micro-lens array another 0.5 mm away from the display screen (d0 = 1 mm) and keep everything else the same. Using the camera with the best focus to take pictures of the cubes through the Goggle cardboard lens and the micro-lens array, the best focus was at about
4 m (a bit less than 15 feet), as shown in Figs. 12a and 12b. The pictures indicate that the
positions of the virtual images of targets on the display screen were changed to 4 m away from the camera (or viewer's eye) by moving the micro-lens array from the screen 1 mm.
4. Move the micro-lens array another 0.2 mm away from the display screen (d0 =1.2 mm) and keep everything else the same. Adjust the camera to the best focus to take pictures of the targets through the Goggle cardboard lens and the micro-lens array, whose results are shown in Figs. 13a and 13b. Fig. 13a shows that the position of the virtual images of the targets on the display screen has changed to∞.
This experiment shows that the positions of the virtual images of targets on the display screen can be changed from 0.75 m to infinity by just moving the micro-lens array 1.2 mm away from the screen using this experimental setup. The movement ranges of the virtual images and the micro-lens array depend on the optical power of the VR lens and the lenslets of the micro-lens array that follow Equation (1) - (4). Generally, the higher the optical power of the VR lens and/or the lenslets, the shorter the movement of the micro-lens array for the same movement range of the virtual images.
Experiment II - Fresnel lens as VR lens
An off-the-shelf Fresnel lens (Part #3.3 from Fresnel Technologies, Inc.) with focal length fb = 51 mm was used as the VR lens in this experiment. The lens was made of Acrylic with the lens diameter of 50 mm and overall size of 64 x 64 x 1.5 mm. Everything else was kept the same as in Experiment I. Fig. 14 shows the experimental set-up. The same Nikon DSL camera was used for recording the results.
Following are the details of the experiment.
1. Move the micro-lens array until it touches the display screen (d0 = 0). Set up the distance between the Fresnel lens and the display screen at about 47 mm (D0 = 47 mm) to satisfy Equation (3). Use the camera to take pictures of the targets with the best focus through the Fresnel lens and the micro-lens array, as shown in Figs. 15a and 15b. The best focus distance of camera indicates that the virtual image of the display screen was located at 1 m away from the camera (or viewer's eye).
2. Move the micro-lens array 0.5 mm away from the display screen (d0 = 0.5 mm) and keep everything else the same. Use the camera with the best focus to take pictures of the targets through the Fresnel lens and the micro-lens array. The best focus is at about 1.5 m (5 feet), as shown in Figs. 16a and 16b.
3. Move the micro-lens array another 0.5 mm away from the display screen (d0 = 1 mm) and keep everything else the same. Use the camera with the best focus to take pictures of the targets through the Fresnel lens and the micro-lens array, whose results are shown in Figs. 17a and 17b. Fig. 17a indicates the location of virtual images of targets were at greater than 3 m away from the camera (or viewer).
4. Move the micro-lens array another 0.2 mm away from the display screen (d0 = 1.2 mm) and keep everything else the same. Use the camera with the best focus to take pictures of the targets through the Fresnel lens and the micro-lens array. Figs. 18a and 18b show the results, which indicate that the virtual images were now located at close to∞ .
This experiment shows that the positions of the virtual images of the targets on the display screen can be changed from 1 m to (close to) infinity by moving the micro-lens array just 1.2 mm away from the screen. We also see that due to the weaker optical power of the Fresnel lens compared to the Google Cardboard lens, the movement range of the virtual images of the targets is smaller than that in Experiment I with the same movement of the micro-lens array.
Both experiments verified that by placing a movable micro-lens array in front of the display screen and effecting a small movement, a relatively large positional change of the virtual image can be realized in the optical path (z) direction that will be seen by the viewer in a VR (or AR) system. Therefore, the accommodation (or focal) distance is adjustable for both systems. Here, all the components and devices used were available off-the-shelf; they were not optimized for the best image quality. Both the Google Cardboard lens and the Fresnel lens used here had large field curvature and distortion, the Google Cardboard lens being worse. The image quality could be further improved by optimizing the micro-lens array and VR (or AR) optical system to the display screen.
There are two methods to eliminate or mitigate accommodation- vergence conflict:
1. With the stereoscopic method for 3D imaging, one could set up different accommodation distances (Z), - locations of the virtual images that the viewer will see) to reach comfortable vergence distances by moving the micro-lens array based on the ZoC. For example, we can move the micro-lens array to make the virtual image (accommodation) distance (D,) 1 m for comfort vergence distances of the 3D image from 0.5 m to 1.5 m and, then, at 10 m for comfort vergence distances from 1.5 m to infinity based on the ZoC.
2. An ima display can be expressed as
where /, χ,, y t) represents the light intensity on the i pixel at location (x„ y,) and time t. (R, G, B) represent red, green and blue colors respectively. The image on the display is represented in 2D (x and y) dimensions (perpendicular to the optical, z, axis). When the display has a movable micro-lens array in front of it, the virtual image of the display through the micro-lens array can be expressed as:
/fa. yitz(t),t) = ({χ,, γ {ίΙή + /"£(*,, yf,z(r), ή + l'!; (xt, γ ΙΟ, t)} (g)
Where I j(Xj,yj,z(t), t) represents the light intensity of the virtual image of the ith pixel on the display at location (xhyhz(t)) and time t. The image is now represented in the z dimension (also the optical axis) as well as the x and y dimensions. It is possible to time-modulate the micro-lens array movement so that z(t) can be changed in a manner to make a true 3D virtual image of display expressed in Equation (6).
Other benefits of placing the micro-lens array in front of the micro-display include:
1. The lens elements will collect more light from the display pixels, so the brightness of the near-eye display will be improved, which is very helpful for AR applications.
2. If adding lateral movement of the lens array in the x and y directions, it may reduce the screen-door effect (SDE) that happens in almost all the current versions of VR systems. The SDE is a grid of fine lines one may see when you have your face in VR, as if looking out of a screen door. That grid is actually the space between pixels. If the lens array is vibrated along the x and y directions with an amplitude of about the spacing between pixels and with a suitable frequency (e.g., > 30 Hz), such movement could blur the grid lines to reduce the SDE without degradation of image quality.
Claims
1. A 3D display apparatus, comprising:
a display screen component comprising a display screen pixel array adapted to display a display screen image;
a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and
a controllable movement component coupled to at least one of the imaging component and the display screen,
wherein the imaging component and the display screen are controllably movable relative to each other,
further wherein upon a controlled movement of the imaging component relative to the display screen, a location of the virtual or real image along an optical axis is controllably changed.
2. The 3D display apparatus of claim 1, wherein the display screen is in a fixed location and the microlens imaging component is movable.
3. The 3D display apparatus of claim 2, wherein the microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen.
4. The 3D display apparatus of claim 2, wherein the microlens imaging component is movable along an optical, z, axis in front of the display screen.
5. The 3D display apparatus of claim 4, wherein the microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen.
6. The 3D display apparatus of claim 1, wherein the microlens imaging component is in a fixed location and the display screen is movable.
7. The 3D display apparatus of claim 6, wherein the display screen is movable in a plane, x-y, perpendicular to an optical, z, axis behind the microlens imaging component.
8. The 3D display apparatus of claim 6, wherein the display screen is movable along an optical, z, axis behind the microlens imaging component.
9. The 3D display apparatus of claim 8, wherein the display screen is movable in a plane, x-y, perpendicular to an optical, z, axis behind the microlens imaging component.
10. The 3D display apparatus of claim 1, wherein both the microlens imaging component and the display screen are movable.
11. The 3D display apparatus of claim 10, wherein the microlens imaging component is movable along an optical, z, axis in front of the display screen, and the display screen is movable in a plane, x- y, perpendicular to an optical, z, axis behind the microlens imaging component.
12. The 3D display apparatus of claim 10, wherein the microlens imaging component is movable in a plane, x-y, perpendicular to an optical, z, axis in front of the display screen, the display screen is movable along an optical, z, axis behind the microlens imaging component.
13. The near-eye display apparatus of claim 1, wherein each of the microlenses has a lateral dimension between 0.5x and 2x a lateral dimension of a single display pixel.
14. The near-eye display apparatus of claim 1, wherein each of the microlenses has a lateral dimension that is equal to the lateral dimension of a single corresponding display pixel.
15. The near-eye display apparatus of claim 1, wherein the pitch of the microlenses is equal to or larger than the lateral dimension of each of the microlenses.
16. A near-eye display system, comprising:
a 3D display apparatus, comprising:
a display screen component comprising a display screen pixel array adapted to display a display screen image;
a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and
a controllable movement component coupled to at least one of the imaging component and the display screen,
wherein the imaging component and the display screen are controllably movable relative to each other; and
a viewing lens fixedly disposed adjacent the microlens imaging component, wherein a final virtual image of the display that is viewable by a viewer will be formed by the viewing lens at a controllably variable distance, Z)„ from the viewing lens relative to a movable position of the microlens imaging component, where Z), is the accommodation distance of the viewer's eye.
17. The near-eye display system of claim 16, wherein the viewing lens is a positive lens system comprising at least a piece of positive lens including a Fresnel lens or a group of lenses to form a positive lens group.
18. The near-eye display system of claim 16, wherein the movement of microlens imaging component along the z-axis is time modulated (z(t)) in the manner so that the corresponding final virtual images from the viewing lens can form a 3D image viewable by the viewer.
19. A method for displaying an image, comprising:
providing a near-eye display system, comprising:
a near-eye display apparatus, further comprising:
a display screen component comprising a display screen pixel array adapted to display a display screen image;
a microlens imaging component comprising an array of microlenses corresponding to the display screen pixel array disposed adjacent the display screen component such that the microlens imaging component can form a virtual or a real image of the display screen image; and
a controllable movement component coupled to at least one of the imaging component and the display screen,
wherein the microlens imaging component and the display screen are
controllably movable relative to each other; and
a viewing lens fixedly disposed adjacent the microlens imaging component, so as to form a final virtual image of the display at a controllably variable distance, Z)„ from the viewing lens; and
moving the microlens imaging component relative to the display screen component to vary the distance, Z)„ and thus controllably change the accommodation distance of the viewer's eye.
20. The method of claim 19, further comprising:
time modulating the movement of microlens imaging component along the optical, z, axis (z(t)) in the manner so that the corresponding final virtual images from the viewing lens form a 3D image viewable by the viewer.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201780018798.4A CN108886610A (en) | 2016-03-15 | 2017-02-23 | 3D display device, methods and applications |
EP17767148.4A EP3430804A4 (en) | 2016-03-15 | 2017-02-23 | 3d display apparatus, method, and applications |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662308308P | 2016-03-15 | 2016-03-15 | |
US62/308,308 | 2016-03-15 | ||
US201662333420P | 2016-05-09 | 2016-05-09 | |
US62/333,420 | 2016-05-09 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017160484A1 true WO2017160484A1 (en) | 2017-09-21 |
Family
ID=59847720
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2017/019064 WO2017160484A1 (en) | 2016-03-15 | 2017-02-23 | 3d display apparatus, method, and applications |
Country Status (4)
Country | Link |
---|---|
US (1) | US10088673B2 (en) |
EP (1) | EP3430804A4 (en) |
CN (1) | CN108886610A (en) |
WO (1) | WO2017160484A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110505385A (en) * | 2019-08-29 | 2019-11-26 | Oppo广东移动通信有限公司 | Imaging system, terminal and image acquiring method |
DE102018115785A1 (en) | 2018-06-29 | 2020-01-02 | BIAS Bremer Institut für angewandte Strahltechnik GmbH | Display unit and method for three-dimensional rendering of an object |
CN110824724A (en) * | 2019-10-25 | 2020-02-21 | 深圳市华星光电半导体显示技术有限公司 | 3D display optical system and device |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6557868B2 (en) * | 2014-03-27 | 2019-08-14 | パナソニックIpマネジメント株式会社 | Virtual image display device, head-up display system, and vehicle |
WO2018074528A1 (en) * | 2016-10-20 | 2018-04-26 | 株式会社ニコン・エシロール | Image preparation device, image preparation method, image preparation program, method for designing eyeglass lens, and method for manufacturing eyeglass lens |
CN107102446B (en) * | 2017-06-27 | 2019-07-09 | 京东方科技集团股份有限公司 | A kind of 3 D stereo display panel, its display methods and display device |
CN109581678A (en) * | 2017-09-29 | 2019-04-05 | 中强光电股份有限公司 | Nearly eye field display device and nearly eye display device |
CN107911691A (en) * | 2017-10-20 | 2018-04-13 | 深圳市虚拟现实科技有限公司 | Mobile terminal virtual reality head based on scale identification shows correction system |
US11435583B1 (en) * | 2018-01-17 | 2022-09-06 | Apple Inc. | Electronic device with back-to-back displays |
CN110376734B (en) | 2018-04-12 | 2021-11-19 | 肥鲨技术 | Single-panel head-mounted display |
DE102018126065A1 (en) | 2018-10-19 | 2020-04-23 | Osram Opto Semiconductors Gmbh | OPTICAL DISPLAY DEVICE AND METHOD FOR OPERATING AN OPTICAL DISPLAY DEVICE |
US11454779B1 (en) | 2018-12-12 | 2022-09-27 | Meta Platforms Technologies, Llc | Head-mounted display device with stepper motors for moving displays |
US10871627B1 (en) | 2018-12-12 | 2020-12-22 | Facebook Technologies, Llc | Head-mounted display device with direct-current (DC) motors for moving displays |
US11042187B1 (en) * | 2018-12-12 | 2021-06-22 | Facebook Technologies, Llc | Head-mounted display device with voice coil motors for moving displays |
CN110161712A (en) * | 2019-05-29 | 2019-08-23 | 广州易视光电科技有限公司 | Display system and head-wearing display device |
CN112241068A (en) * | 2019-07-18 | 2021-01-19 | 三星电子株式会社 | Image display device capable of multi-depth expression |
US20220326522A1 (en) * | 2019-07-18 | 2022-10-13 | Samsung Electronics Co., Ltd. | Image display device capable of expressing multiple depth |
TWI811416B (en) * | 2019-07-31 | 2023-08-11 | 友達光電股份有限公司 | Near-eye display device |
WO2021256313A1 (en) * | 2020-06-15 | 2021-12-23 | ソニーセミコンダクタソリューションズ株式会社 | Display device |
CN111679445A (en) * | 2020-06-18 | 2020-09-18 | 深圳市洲明科技股份有限公司 | Light field display device and stereoscopic display method |
CN112859349A (en) * | 2021-03-02 | 2021-05-28 | 西安梦幻世界信息科技有限公司 | Augmented reality display device based on micro LED micro display |
CN114326129A (en) * | 2022-02-22 | 2022-04-12 | 亿信科技发展有限公司 | Virtual reality glasses |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20080062912A (en) * | 2006-12-29 | 2008-07-03 | 엘지디스플레이 주식회사 | Three dimensional image display device |
US20130021226A1 (en) * | 2011-07-21 | 2013-01-24 | Jonathan Arnold Bell | Wearable display devices |
US20130285885A1 (en) * | 2012-04-25 | 2013-10-31 | Andreas G. Nowatzyk | Head-mounted light-field display |
US20140168783A1 (en) * | 2012-07-02 | 2014-06-19 | Nvidia Corporation | Near-eye microlens array displays |
US20140340390A1 (en) * | 2013-05-17 | 2014-11-20 | Nvidia Corporation | System, method, and computer program product to produce images for a near-eye light field display having a defect |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8730129B2 (en) * | 1990-12-07 | 2014-05-20 | Dennis J Solomon | Advanced immersive visual display system |
US5303085A (en) | 1992-02-07 | 1994-04-12 | Rallison Richard D | Optically corrected helmet mounted display |
AU6276496A (en) * | 1995-06-07 | 1996-12-30 | Jacob N. Wohlstadter | Three-dimensional imaging system |
JPH10319342A (en) * | 1997-05-15 | 1998-12-04 | Olympus Optical Co Ltd | Eye ball projection type video display device |
US20110075257A1 (en) | 2009-09-14 | 2011-03-31 | The Arizona Board Of Regents On Behalf Of The University Of Arizona | 3-Dimensional electro-optical see-through displays |
US8508851B2 (en) | 2011-07-20 | 2013-08-13 | Google Inc. | Compact see-through display system |
US8705177B1 (en) * | 2011-12-05 | 2014-04-22 | Google Inc. | Integrated near-to-eye display module |
US20150262424A1 (en) * | 2013-01-31 | 2015-09-17 | Google Inc. | Depth and Focus Discrimination for a Head-mountable device using a Light-Field Display System |
US20160097930A1 (en) * | 2014-10-06 | 2016-04-07 | Steven John Robbins | Microdisplay optical system having two microlens arrays |
-
2017
- 2017-02-23 EP EP17767148.4A patent/EP3430804A4/en not_active Withdrawn
- 2017-02-23 US US15/440,487 patent/US10088673B2/en active Active
- 2017-02-23 CN CN201780018798.4A patent/CN108886610A/en active Pending
- 2017-02-23 WO PCT/US2017/019064 patent/WO2017160484A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20080062912A (en) * | 2006-12-29 | 2008-07-03 | 엘지디스플레이 주식회사 | Three dimensional image display device |
US20130021226A1 (en) * | 2011-07-21 | 2013-01-24 | Jonathan Arnold Bell | Wearable display devices |
US20130285885A1 (en) * | 2012-04-25 | 2013-10-31 | Andreas G. Nowatzyk | Head-mounted light-field display |
US20140168783A1 (en) * | 2012-07-02 | 2014-06-19 | Nvidia Corporation | Near-eye microlens array displays |
US20140340390A1 (en) * | 2013-05-17 | 2014-11-20 | Nvidia Corporation | System, method, and computer program product to produce images for a near-eye light field display having a defect |
Non-Patent Citations (1)
Title |
---|
See also references of EP3430804A4 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102018115785A1 (en) | 2018-06-29 | 2020-01-02 | BIAS Bremer Institut für angewandte Strahltechnik GmbH | Display unit and method for three-dimensional rendering of an object |
DE102018115785B4 (en) | 2018-06-29 | 2023-09-07 | BIAS Bremer Institut für angewandte Strahltechnik GmbH | Display unit for three-dimensional reproduction, method for holographic reproduction and hologram of a moving object |
CN110505385A (en) * | 2019-08-29 | 2019-11-26 | Oppo广东移动通信有限公司 | Imaging system, terminal and image acquiring method |
CN110505385B (en) * | 2019-08-29 | 2021-06-11 | Oppo广东移动通信有限公司 | Imaging system, terminal and image acquisition method |
CN110824724A (en) * | 2019-10-25 | 2020-02-21 | 深圳市华星光电半导体显示技术有限公司 | 3D display optical system and device |
Also Published As
Publication number | Publication date |
---|---|
US10088673B2 (en) | 2018-10-02 |
US20170269353A1 (en) | 2017-09-21 |
CN108886610A (en) | 2018-11-23 |
EP3430804A1 (en) | 2019-01-23 |
EP3430804A4 (en) | 2019-11-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10088673B2 (en) | 3D display apparatus, method, and applications | |
JP7369507B2 (en) | Wearable 3D augmented reality display with variable focus and/or object recognition | |
JP6796368B2 (en) | Stereoscopic display with addressable focus cues | |
JP7185331B2 (en) | How to render light field images for integral imaging light field displays | |
US9841537B2 (en) | Near-eye microlens array displays | |
CN109997070B (en) | Near-to-eye display system including modulation stack | |
US9905143B1 (en) | Display apparatus and method of displaying using image renderers and optical combiners | |
CN109683335A (en) | Use showing without 3D glasses light field for eye position | |
CN110582718A (en) | zoom aberration compensation for near-eye displays | |
WO2018100239A1 (en) | Imaging system and method of producing images for display apparatus | |
CN102566248B (en) | Stereoscopic imaging apparatus | |
JP2001238229A (en) | Stereoscopic image photographing device and stereoscopic image display device, and stereoscopic image photographing and display system | |
CN110879478B (en) | Integrated imaging 3D display device based on compound lens array | |
KR101322187B1 (en) | 3-dimensional moving picture photographing apparatus and camera | |
JP5793099B2 (en) | Display device, electronic device and control circuit | |
US11463674B1 (en) | Imaging system and display apparatus incorporating super resolution using fixed focus cameras | |
KR102026361B1 (en) | Display device | |
JP2001197522A (en) | Image display device | |
KR101578890B1 (en) | 3-dimensional moving picture photographing apparatus and camera | |
CN112526801B (en) | Double-lens imaging module and extraction method thereof | |
KR102135888B1 (en) | Projection device | |
JP6541005B2 (en) | Distant view imaging device | |
KR101746717B1 (en) | Output method of view images in three-dimensional display with a different focus lens array | |
Fujikawa et al. | Retinal projection type super multi-view stereoscopic head-mounted display | |
KR101433645B1 (en) | 3-dimensional moving picture photographing apparatus and camera |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2017767148 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2017767148 Country of ref document: EP Effective date: 20181015 |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17767148 Country of ref document: EP Kind code of ref document: A1 |