TW202332941A - Wavelength multiplexed waveguide system for eye tracking - Google Patents

Abstract

An apparatus, system, and method for a waveguide system may be used to support eye tracking in a head mounted display (HMD). The waveguide system may be positioned in a user's field of view and within a lens assembly of the HMD to capture light that is reflected from an eye. The waveguide system may include a waveguide, a first diffraction grating, and a second diffraction grating. The first diffraction grating may be configured to in-couple light of a first wavelength into the waveguide, and the second diffraction grating may be configured to in-couple light of a second wavelength. The first and second diffraction gratings operate together to detect light reflections from an eyebox region.

Description

Wavelength multiplexed waveguide system for eye tracking

The present invention relates generally to optics and, in particular, to eye tracking technology. Cross-references to related applications

This application claims priority to U.S. Provisional Application No. 63/294,341 filed on December 28, 2021 and U.S. Non-Provisional Application No. 17/830339 filed on June 2, 2022, which are hereby incorporated by reference. way to incorporate.

Eye tracking technology enables head mounted displays (HMDs) to interact with users based on their eye movement or eye orientation. Existing eye tracking systems are technically limited by natural obstacles. For example, eyelashes and eyelids can block images of the eyes, which can degrade the quality of the eye-tracking operation.

One aspect of the invention is a lens component comprising: a waveguide; a first diffraction grating disposed in the waveguide and configured to couple first light from within the orbital region into the waveguide, the third a light having a first wavelength; and a second diffraction grating disposed in the waveguide and configured to couple second light from within the orbital region into the waveguide, the second light having a second wavelength.

Another aspect of the invention is an eye tracking system including: a controller configured to determine eye orientation based on image data; a waveguide system including: a waveguide; and a first diffraction grating disposed in the waveguide. and configured to couple first light from within the orbital region into the waveguide, the first light having a first wavelength; and a second diffraction grating disposed in the waveguide and configured to couple the second light The second light has a second wavelength coupled from within the orbital region into the waveguide; and an image sensor optically coupled to the waveguide system to receive the first light and the second light from the waveguide, The image sensor is configured to generate the image data.

Another aspect of the invention is a head mounted device including: a frame; a lens member coupled to the frame and configured to transmit scene light to the eye orbit region; and a waveguide system coupled to the a lens member and coupled to the frame, wherein the waveguide system includes: a waveguide; a first diffraction grating disposed in the waveguide and configured to couple first light from within the orbital region into the waveguide, The first light has a first wavelength; and a second diffraction grating disposed in the waveguide and configured to couple a second light from within the orbital region into the waveguide, the second light having a second a wavelength; and an image sensor positioned in the frame to generate image data using the first light and the second light received from the waveguide.

This article describes a specific example of a wavelength-multiplexed waveguide imaging system that supports in-field eye tracking. In the following description, numerous specific details are set forth in order to provide a thorough understanding of specific examples. However, one skilled in the relevant art will recognize that the techniques described herein may be practiced without one or more of the specific details or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. In the instance. Therefore, the appearances of the phrases "in a specific example" or "in a specific example" throughout this specification do not necessarily refer to the same specific example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

In aspects of the invention, visible light may be defined as having a wavelength range of approximately 380 nm to 700 nm. Non-visible light can be defined as light having wavelengths outside the visible range, such as ultraviolet light and infrared light. In aspects of the invention, red light may be defined as having a wavelength range of approximately 620 to 750 nm, green light may be defined as having a wavelength range of approximately 495 to 570 nm, and blue light may be defined as having a wavelength range of approximately 450 to 495 nm. wavelength range.

As used herein, the diffraction angle of light from an optical element (e.g., optical coupler, diffraction grating, holographic optical element, etc.) is the normal (i.e., 90 degrees) of the emergent ray with respect to the exit surface of the optical element the displacement angle.

As used herein, diffraction gratings may include prescribed gratings or holographic gratings. The holographic grating may include a substrate having a photosensitive material on which the grating is recorded (eg, within the substrate). Holographic gratings can also be called holographic optical elements (HOE).

Eye-tracking functionality extends head-mounted displays (HMDs) to provide interactive services and quality to users. When imaging is performed from the periphery of the eye, eyelashes and eyelids can block and inhibit the quality of signals (eg, images) available from the eye. The obviously preferred location for imaging light reflections from the eye is directly in front of the eye. However, placing the camera directly in front of the eyes may hinder the user's vision and may reduce the quality of the user's HMD experience. Disclosed herein are technologies for waveguide systems that capture light from the eye from directly in front of the eye and within the eye's field of view (field). The waveguide system guides light from the in-field portion of the lens member to an image sensor that may be positioned on or in the frame of the HMD. In addition, waveguide systems can use wavelength multiplexed diffraction gratings to encode the spatial location of reflections into angles, wavelength-encode portions of the eyebox region, and expand the eyebox region where reflections can be detected.

The HMD may include a (eg, wavelength multiplexed) waveguide system disposed at least partially in the lens member and in the frame of the HMD to receive light reflections from the user's eyes. The waveguide system can guide light reflection (eg, infrared rays) from the user's eyes to the image sensor to achieve non-dispersion and in-field imaging of the user's eyes. The waveguide system may include two or more in-coupling diffraction gratings, waveguides and out-coupling diffraction gratings. Incoupling diffraction gratings may be configured to incouple reflected light from the eye (or orbital region) into the waveguide. The diffracted light reflection may include first wavelength light and second wavelength light. The waveguide can guide light from the in-coupling diffraction grating (eg, via total internal reflection (TIR)) to the out-coupling diffraction grating. The outcoupling diffraction grating may be configured to outcouple the first wavelength light and the second wavelength light from the waveguide to the image sensor (eg, via a lens).

Each of the in-coupling diffraction gratings may be a holo-optical element (HOE) having a plurality of tilted grating planes configured to map (or encode) the incident location of each ray to a TIR angle, The incident position is relative to the surface of the in-coupling diffraction grating. In other words, the TIR angle of a particular light ray indicates the location on the incoupling diffraction grating where the light ray is received. The outcoupling diffraction grating may then be configured to decode the incident location of each ray based on the diffraction angle of the particular ray. In a specific example, the exit angle or exit position of the light ray from the outcoupling diffraction grating is proportional or related to the incident position of the specific light ray. The outcoupling diffraction grating may be configured to diffract a wavelength band including the first wavelength of light and the second wavelength of light.

The controller can be communicatively coupled to the image sensor to receive image data from the image sensor. The controller may use the image data to determine eye orientation and/or perform one or more eye tracking operations. According to aspects of the invention, based on eye orientation and/or eye tracking data, the HMD may be configured to selectively display information and/or provide or adjust the number of user interface elements in the lens member of the HMD.

The in-coupling and out-coupling diffraction gratings can be implemented as transmissive or reflective diffraction gratings. Transmissive diffraction gratings transmit specific light wavelengths (for example, in the infrared range) and only pass or transmit other wavelengths without diffracting. Reflective diffraction gratings reflect certain wavelengths of light (for example, in the infrared range) and pass or transmit other wavelengths without diffraction. The surface area and/or volume of the in-coupling diffraction grating may be larger than the surface area and/or volume of the out-coupling diffraction grating to facilitate capturing light reflections from the eye orbit and to facilitate focusing light into the frame from the HMD for image sensing on the device. The combined surface area of multiple (eg, two) incoupling diffraction gratings is operable to expand the effective area of the orbital region from which light can be reflected and diffracted into the waveguide.

Each of the in-coupling diffraction gratings (and/or the out-coupling diffraction gratings) may be a rolled diffraction grating with several tilted diffraction gratings. A tilted diffraction grating diffracts light into a waveguide. The tilted diffraction grating may diffract light at a different diffraction angle on a first side of the incoupling diffraction grating than on a second side of the incoupling diffraction grating. According to aspects of the invention, a tilted diffraction grating may have a tilt angle that changes (eg, increases or decreases) from a first side of the incoupling diffraction grating to a second side of the incoupling diffraction grating. Tilted diffraction gratings may be designed or configured to operate at a specific wavelength range (eg, specific near-infrared or infrared wavelengths). According to specific examples of the present invention, the tilted diffraction grating may have a tilt angle, a grating line, and a grating period defined based on the diffraction angle and angular bandwidth of the tilted diffraction grating.

The apparatus, systems and methods described in this disclosure for wavelength multiplexed waveguide systems may enable improved eye tracking technology, such as supporting the operation of HMDs. These and other specific examples are described in greater detail in conjunction with Figures 1-11.

Figure 1 illustrates an example head-mounted display (HMD) 100 that supports eye tracking from within the user's field of view (in-field), in accordance with an embodiment of the present invention. According to one specific example, HMD 100 includes a waveguide system 102 configured to in-couple light from the eye orbital region and to out-couple light from the eye orbit region to an image sensor positioned in or on frame 106 104. According to one embodiment, waveguide system 102 is disposed partially within lens member 108 and partially within frame 106 to support in-field reception of light reflected from the orbital region of the eye. An advantage of in-field imaging of the orbital region and the user's eyes is that positioning the waveguide system 102 in front of the user's eyes reduces obstructions such as eyelids and eyelashes that can reduce capture from the user's eyes. The quality of the image. According to aspects of the present invention, another benefit of in-field imaging of the orbital region may be improved reflection reception from the user's eyes. Waveguide system 102 may be used to support eye tracking, user experience (UX), and other features of HMD 100. An HMD, such as HMD 100, is a type of head-mounted display that is typically worn on a user's head to provide artificial reality content to the user. Artificial reality is a form of reality that has been adjusted in some way before being presented to the user. It can include, for example, virtual reality (VR), augmented reality (AR), mixed reality ( mixed reality (MR), mixed reality or a combination and/or derivative thereof.

The HMD 100 utilizes the frame 106 to carry the waveguide system 102 and the image sensor 104 . Frame 106 is coupled to arms 110A and 110B. Lens member 108 is mounted to frame 106 . Lens member 108 may include prescription optical layers matched to the particular user of HMD 100, or may be a non-prescription lens. The illustrated HMD 100 is configured to be worn on or around the head of a wearer of the HMD 100 .

Lens member 108 may appear transparent to the user to facilitate augmented reality or mixed reality so that the user can view scene light from her surrounding environment while also receiving image light directed to her eyes. Accordingly, lens member 108 may be considered (or includes) an optical combiner. In a specific example, lens member 108 may include two or more optical layers carrying portions of waveguide system 102 . In some embodiments, display light from one or more integrated displays is directed to one or both eyes of the wearer of HMD 100.

According to one specific example, the waveguide system 102 and the image sensor 104 may be configured to capture reflected images leaving the user's eyes. To create reflections of light away from the user's eyes, HMD 100 may include multiple light sources 112 positioned at one or more locations around frame 106 . The light source 112 is directed to direct light toward the orbital region to illuminate at least one user's eye. Light source 112 may emit light in the non-visible spectrum. For example, according to one specific example, light source 112 is configured to emit infrared light, for example, having a wavelength in the range of 750 nm to 1500 nm. Some of the light sources 112 may be configured to emit light at a first wavelength, which is light having a first wavelength (eg, 1300 nm), and others of the light sources 112 may be configured to emit light at a second wavelength, which is Light with a second wavelength (for example, 940 nm). The light source 112 may be a light emitting diode (light emitting diode; LED), a vertical-cavity surface-emitting laser (VCSEL), a micro-light emitting diode (micro LED), an edge-emitting LED, Superluminescent diode (SLED) or another type of light source. In one specific example, the light emitted from light source 112 is infrared light centered at about 850 nm. Infrared light from other sources can also illuminate the eyes. According to one specific example, HMD 100 may be configured to use images reflected off the user's eyes to determine the orientation of the user's eyes and/or perform eye tracking operations.

According to one specific example, HMD 100 includes controller 118 communicatively coupled to image sensor 104 . According to one embodiment, controller 118 is coupled to image sensor 104 to receive images captured by image sensor 104 using waveguide system 102 . According to a specific example, the controller 118 may include processing logic 120 and one or more memories 122 to analyze image data received from the image sensor 104, determine the orientation of one or more of the user's eyes, and execute One or more eye tracking operations and/or user interface elements are displayed or provided in lens member 108 . Controller 118 may include wired and/or wireless data interfaces for sending and receiving data, a graphics processor, and one or more memories 122 for storing data and computer-executable instructions. Controller 118 and/or processing logic 120 may include circuitry, logic, instructions stored in a machine-readable storage medium, ASIC circuitry, FPGA circuitry, and/or one or more processors. In one specific example, HMD 100 may be configured to receive wired power. In one specific example, HMD 100 is configured to be powered by one or more batteries. In one specific example, HMD 100 may be configured to receive wired data including video data via a wired communication channel. In one specific example, HMD 100 is configured to receive wireless data including video data via a wireless communication channel.

HMD 100 may include waveguide system 114 and image sensor 116 positioned on or around lens member 124, such as on the left side of frame 106. According to a specific example, waveguide system 114 may include similar features as waveguide system 102 . According to one specific example, image sensor 116 may be configured to operate similarly to image sensor 104 . Lens member 124 may include similar features and/or layers as lens member 108 .

Waveguide system 102 may be configured to pass or transmit scene light from the scene side of HMD 100 such that waveguide system 102 appears transparent to a user of HMD 100 . According to various aspects of the invention, waveguide system 102 is also configured to selectively direct light from, for example, central region 126 of lens member 108 to image sensor 104 .

Figure 2 illustrates an example top view of an eye environment 200 in accordance with various embodiments of the present invention. According to a specific example, eye environment 200 includes HMD 202 and eyes 204 . HMD 202 is an example implementation of HMD 100. As illustrated, HMD 202 is a partial cross-sectional view of what may be a head-mounted display, according to one specific example. The eye 204 is positioned on the orbital side 206 of the HMD 202 . The eye 204 is positioned in the orbital region 208 on the orbital side 206 and is positioned to receive scene light 210 from the scene side 212 . According to a specific example, scene light 210 passes through lens member 214 to orbital region 208 and to eye 204 . The scene light 210 passes through the lens member 214 and the waveguide system 216 from the scene side 212 to the eye orbit side 206 .

According to a specific example, waveguide system 216 is an example implementation of waveguide systems 102 and/or 114 . According to one specific example, waveguide system 216 is configured to receive reflections from eye 204 and/or orbital region 208 of non-visible light 218 that becomes incident on surface 220 . According to a specific example, the waveguide system 216 includes a waveguide 222, an in-coupling diffraction grating 224, and an out-coupling diffraction grating 226.

According to one specific example, waveguide system 216 is configured to receive reflections of non-visible light 218 using incoupling diffraction grating 224 . According to a specific example, in-coupling diffraction grating 224 in-couples reflected light into waveguide 222. In-coupled diffraction grating 224 may represent two or more in-coupled diffraction gratings (eg, a first diffraction grating configured to diffract light of a first wavelength and a second diffraction grating configured to diffract light of a second wavelength). diffraction grating). According to one embodiment, in-coupling diffraction grating 224 guides the reflected light to out-coupling diffraction grating 226 by co-coupling the reflected light into waveguide 222 . According to a specific example, after the reflected light has propagated from the in-coupling diffraction grating 224 to the out-coupling diffraction grating 226 by total internal reflection (TIR) within the waveguide 222 , the out-coupling diffraction grating 226 passes from the in-coupling diffraction grating 224 Receive reflected light.

According to one specific example, outcoupling diffraction grating 226 is configured to receive reflected light from waveguide 222 and outcouple the reflected light. According to one specific example, outcoupling diffraction grating 226 is configured to provide received reflected light to image sensor 104 . As illustrated, according to one embodiment, outcoupling diffraction grating 226 and image sensor 104 may be positioned within (or on) a portion of frame 106 (eg, outside the field of view of eye 204). According to one embodiment, outcoupling diffraction grating 226 and a portion of waveguide 222 may be positioned within a portion of frame 106 to facilitate outcoupling reflected light from outcoupling diffraction grating 226 to image sensor 104 . According to a specific example, the out-coupling diffraction grating 226 may be configured to diffract a range or frequency band of wavelengths including the first wavelength of light and the second wavelength of light.

Image sensor 104 is configured to convert received reflected light into electrical signals. According to a specific example, the electrical signal may represent reflected light received by the in-coupled diffraction grating 224 . According to a specific example, the image sensor 104 converts the received reflected light into image data 228 and provides the image data 228 to the controller 118 via the communication channel 230 . In other words, the controller 118 is communicatively coupled to receive image data 228 from the image sensor 104 . According to one embodiment, controller 118 may use one or more of a variety of techniques to determine the orientation of eye 204 and perform one or more eye tracking operations based on image data 228 .

According to one specific example, the HMD 202 may include a projector 232 and a display 234 configured to provide information and/or user interface elements to the orbital region 208 for viewing by a user of the HMD 202 . The display 234 may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a microLED display, a quantum dot display, a pico projector, or a liquid crystal on silicon (liquid crystal on silicon; LCOS) display for directing image light to the wearer of HMD 202. According to a specific example, projector 232 may be positioned in or on frame 106 and display 234 may be positioned at least partially within lens member 214 . According to a specific example, display 234 may be transparent and may be configured to allow scene light 210 to pass through lens member 214 to eye orbit region 208 . According to one specific example, projector 232 and display 234 may be communicatively coupled to receive instructions and/or information from controller 118 and may be configured to project information based at least in part on the orientation of eye 204 .

For purposes of illustration, lens member 214 is illustrated as a single optical layer. According to a specific example, lens member 214 may be implemented as a single optical layer, as illustrated, or may be implemented as two or more optical layers coupled together to include waveguide system 216 and display 234 .

According to a specific example, FIG. 3 illustrates a top view of HMD 300. According to a specific example, HMD 300 includes a lens member 302 that includes a plurality of optical layers. According to a specific example, lens member 302 is an example implementation of lens member 214 . According to a specific example, lens member 302 includes a waveguide optical layer 304 and a display optical layer 306. According to one specific example, waveguide optical layer 304 is coupled to display optical layer 306 to transmit scene light 210 to eye orbit region 208 . According to a specific example, lens member 302 may include one or more additional layers such as optical layer 308 and optical layer 310 that provide optical power, spacing, and one or more additional features or characteristics that support the operation of HMD 300 .

4A and 4B illustrate example embodiments of a waveguide imaging system that may be implemented into one or more of the disclosed HMDs, in accordance with aspects of the disclosure.

Figure 4A illustrates a waveguide imaging system 400 according to one specific example. According to one specific example, waveguide imaging system 400 includes waveguide system 402 configured to receive light (eg, reflect infrared light) from orbital region 208 and provide the light to image sensor 104 . Waveguide system 402 is an example implementation of waveguide system 102 (shown in FIG. 1 ) and/or waveguide system 216 (shown in FIG. 2 ). According to one specific example, waveguide system 402 uses diffraction grating 404 to incouple light into waveguide 406 and diffraction grating 408 to outcouple light from waveguide 406 to image sensor 104 . According to one specific example, diffraction grating 404 , waveguide 406 , and diffraction grating 408 are optical elements that at least partially define waveguide system 402 and operate together to direct light from orbital region 208 to image sensor 104 .

Diffraction grating 404 is a transmissive grating configured in transmissive operation to diffract some wavelengths of light while passing (without diffracting) other wavelengths of light. Diffraction grating 404 may be configured to diffract light having wavelengths in the infrared wavelength range while passing other wavelengths of light (eg, the visible wavelength band) without diffracting. Diffraction grating 404 incouples light 410 from orbital region 208 (eg, from eye 204) into waveguide 406 such that waveguide 406 reflects light 410 (eg, using TIR) to diffraction grating 408.

According to a specific example, the diffraction grating 404 includes a first end 412 and a second end 414 and is configured to diffract light from the first end 412 differently than the second end 414 . For example, the diffraction grating 404 may be configured to diffract light 416 on the first end 412 at a first diffraction angle θ _D1 , and may be configured to diffract light 416 on the second end 414 at a second diffraction angle θ _D2 Ray 418. In a specific example, the first diffraction angle θ _D1 is an angle greater than the second diffraction angle θ _D2 , such that the diffraction grating 404 diffracts the light 410 more strongly from the first end 412 and less strongly from the second end. 414 diffracts light 410 to reduce the likelihood that light reflected within waveguide 406 will be reflected back onto diffraction grating 404. According to a specific example, the diffraction grating 404 is configured to diffract light at a gradually smaller diffraction angle from the first end 412 to the second end 414 . Alternatively, according to one specific example, diffraction grating 404 is configured to diffract light at progressively larger diffraction angles from first end 412 to second end 414 . According to various aspects of the invention, light rays 416 and 418 represent a plurality of light rays (eg, light 410 ) that are received by the incident surface 420 and refract at the exit surface with varying diffraction angles from the first end 412 to the second end 414 422 external diffraction.

Diffraction grating 408 is configured to receive light rays 416 and 418 using incident surface 424 and is configured to direct light rays 416 and 418 to image sensor 104 . Diffraction grating 408 is a transmissive grating configured in transmissive operation to diffract some wavelengths of light while transmitting other wavelengths of light. According to one specific example, diffraction grating 408 is a transmissive diffraction grating that outcouples light rays 416 and 418 from waveguide 406 to image sensor 104 . According to one specific example, diffraction grating 408 may be configured to diffractively outcouple light rays 416 and 418 from waveguide 406 to image sensor 104 . Similar to diffraction grating 404, diffraction grating 408 may be configured to diffract light from exit surface 426 from first side 428 at a different angle than the angle of second side 430. Diffraction grating 408 may be configured to diffract light from first side 428 at a diffraction angle less than the diffraction angle from second side 430 . Diffraction grating 408 may be configured to emit light at a diffraction angle that changes gradually or incrementally from first side 428 to second side 430 . According to a specific example, the diffraction angle of light emitted from the exit surface 426 gradually increases from the first side 428 to the second side 430 . According to a specific example, the diffraction angle of the light emitted from the exit surface 426 gradually decreases from the first side 428 to the second side 430 .

According to one specific example, diffraction grating 404 is positioned within waveguide 406 proximate surface 432 of waveguide 406 such that diffraction grating 404 can incouple light 410 into waveguide 406 and that allows diffraction grating 404 to be directed toward diffraction grating 408 Light410. According to a specific example, the incident surface 420 of the diffraction grating 404 defines or constitutes at least a portion of the surface 432 of the waveguide 406 such that the portion of the incident surface 420 and the surface 432 are the same surface. According to one specific example, diffraction grating 404 is positioned in waveguide 406 on the lens member side 436 of waveguide 406 . According to one specific example, the lens member side 436 of the waveguide 406 represents the portion of the waveguide 406 that transmits scene light 210 to the eye orbit region 208 .

According to one specific example, the diffraction grating 408 is positioned within the waveguide 406 proximate the surface 434 of the waveguide 406 so that the diffraction grating 408 can couple light 410 out of the waveguide 406 and the diffraction grating 408 can be directed toward the image sensor 104 Light410. According to one embodiment, the exit surface 426 of the diffraction grating 408 defines or forms at least a portion of the surface 434 of the waveguide 406 such that the portion of the exit surface 426 and the surface 434 are the same surface. According to one specific example, diffraction grating 408 is positioned in waveguide 406 on frame side 438 of waveguide 406 . According to one specific example, the frame side 438 of the waveguide 406 represents a portion of the waveguide 406 that is positioned at least partially within or on the surface of the frame of the HMD to enable outcoupling of light to the image sensor 104 .

Waveguide imaging system 400 optionally includes lens 440 positioned between waveguide 406 and image sensor 104 . Lens 440 may be constructed from a single optical layer or may include multiple optical layers coupled together to focus light from exit surface 426 onto image sensor 104 . In one specific example, diffraction grating 408 and lens 440 are configured to focus light from first end 412 of diffraction grating 404 onto first end 442 of image sensor 104 and are configured to focus light from diffraction grating 404 to first end 442 of image sensor 104 . The light from the second end 414 of the grating 404 is focused onto the second end 444 of the image sensor 104, or vice versa.

Figure 4B illustrates a waveguide imaging system 450 according to one specific example. According to one specific example, waveguide imaging system 450 includes waveguide system 452 configured to receive light 410 from orbital region 208 and selectively provide light 410 to image sensor 104 . Waveguide system 452 is an example implementation of waveguide system 102 (shown in FIG. 1 ) and/or waveguide system 216 (shown in FIG. 2 ). According to one embodiment, waveguide system 452 uses one or more reflective volume Bragg gratings (VBG) to couple light 410 to image sensor 104 . By using VBG, the waveguide system 452 can advantageously operate with reduced or eliminated (eg, less than .01%) visible rainbow artifacts that can occur in in-field waveguide imaging systems. More specifically, according to some implementations, waveguide system 452 can operate with less than .01% of transmissive rainbow artifacts and can operate with almost no reflective rainbow artifacts. In one embodiment, waveguide system 452 uses reflective diffraction grating 454 to incouple light into waveguide 456 and uses reflective diffraction grating 458 to outcouple light from waveguide 456 to the image sensor, according to one embodiment. 104.

Diffraction grating 454 is a reflective diffraction grating (eg, a reflective VBG) configured in reflective operation to diffract some wavelengths of light while passing (without operating) other wavelengths of light. Diffraction grating 454 may be configured to diffract light having wavelengths in the infrared wavelength range (eg, 850 nm) while passing other wavelengths of light (eg, the visible wavelength band) without diffraction. Diffraction grating 454 incouples light 410 from orbital region 208 (eg, from eye 204) into waveguide 456 such that waveguide 456 reflects light 410 (eg, using TIR) to diffraction grating 458.

According to one specific example, diffraction grating 454 includes a first end 462 and a second end 464 and is configured to diffract light from first end 462 differently than light from second end 464 . For example, the diffraction grating 454 may be configured to diffract light 416 on the first end 462 at a first diffraction angle θ _D3 and may be configured to diffract light 416 on the second end 464 at a second diffraction angle θ _D4 Ray 418. In a specific example, the first diffraction angle θ _D3 is an angle greater than the second diffraction angle θ _D4 , such that the diffraction grating 454 diffracts the light 410 more strongly from the first end 462 and less strongly from the second end. 464 diffracts light 410 to reduce the likelihood of the light being reflected back onto diffraction grating 454. According to a specific example, the diffraction grating 454 is configured to diffract light at a gradually smaller diffraction angle from the first end 462 to the second end 464 . According to one specific example, diffraction grating 454 is configured to diffract light at progressively larger diffraction angles from first end 462 to second end 464 . According to various aspects of the invention, light rays 416 and 418 represent a plurality of light rays (eg, light 410 ) received by surface 470 and diffracted back out of surface 470 at diffraction angles that vary from first end 462 to second end 464 .

Diffraction grating 454 is a rolled diffraction grating having a plurality of inclined grating planes 472 that vary (eg, progressively increase or decrease) from first end 462 to second end 464 of diffraction grating 454 The diffraction angle of the outgoing ray. The tilted grating plane 472 changes the diffraction angle of the outgoing light based on the tilt angle of the tilted grating plane 472 . According to one specific example, diffraction grating 454 maps each location of incident light rays to one or more specific total internal reflection (TIR) angles within waveguide 456 . In other words, according to one embodiment, diffraction grating 454 encodes information onto the received light rays by correlating the incident position of the light rays (on diffraction grating 454) with the TIR angle within waveguide 456. According to one embodiment, the specific TIR angle through which light is received by diffraction grating 458 provides an indication of where the light is incident on diffraction grating 454 (eg, from orbital region 208 ). According to one specific example, diffraction grating 458 is configured to decode the incident location of a light ray based on its specific diffraction angle. According to one embodiment, the specific angle by which light exits waveguide 456 and/or is received by image sensor 104 provides an indication of the position and/or angle of incidence of light incident on diffraction grating 454 .

Tilted grating plane 472 (individually, tilted grating plane 472A, 472B, 472C, etc.) is associated with a tilt angle φ (individually, tilt angle φ _S1 , φ _S2 , φ _S3 ) that at least partially defines the angle of tilted grating plane 472 . For the sake of clarity of illustration, only a limited number of illustrated tilted grating planes are labeled. In practice, however, the number of grating planes with, for example, a few micron spacing between each other will be difficult to fully account for. According to a specific example, a tilt angle φ is defined with respect to surface 470 of diffraction grating 454 . According to a specific example, the tilt angle φ may also be defined with respect to the intersection of the surfaces 470 and with respect to the normal to each of the tilted grating planes 472 . According to specific examples of the present invention, the tilt angle φ and the tilt grating plane 472 are at least partially defined by the techniques described in connection with FIGS. 5 , 6 and 7 .

Diffraction grating 458 is configured to receive light rays 416 and 418 (eg, using surface 474 ) and is configured to direct light rays 416 and 418 to image sensor 104 . Diffraction grating 458 is a reflective diffraction grating configured to operate reflectively to diffract some wavelengths of light (eg, within infrared wavelengths) while transmitting other wavelengths of light (eg, visible wavelengths). According to one specific example, diffraction grating 458 is a reflective diffraction grating that outcouples light rays 416 and 418 from waveguide 456 to image sensor 104 . Similar to diffraction grating 454, diffraction grating 458 may be configured to diffract light from surface 474 from first side 478 at a different angle than the angle of second side 480. Diffraction grating 458 may be configured to diffract light from first side 478 at a diffraction angle that is less than the diffraction angle from second side 480 . Diffraction grating 458 may be configured to emit light at a diffraction angle that changes gradually or incrementally from first side 478 to second side 480 . According to a specific example, the diffraction angle of light emitted from surface 474 gradually increases from first side 478 to second side 480 . According to a specific example, the diffraction angle of light emitted from surface 474 gradually decreases from first side 478 to second side 480 .

According to one specific example, diffraction grating 454 is positioned within waveguide 456 proximate surface 482 of waveguide 456 such that diffraction grating 454 can incouple light 410 into waveguide 456 and that allows diffraction grating 454 to be directed toward diffraction grating 458 Light410. According to a specific example, at least one surface of diffraction grating 454 and surface 482 are on the same plane or at least partially define the same surface. According to one specific example, diffraction grating 454 is positioned in waveguide 456 on the lens member side 486 of waveguide 456 . According to one specific example, the lens member side 486 of the waveguide 456 represents the portion of the waveguide 456 that transmits the scene light 210 to the eye orbit region 208 .

According to one embodiment, diffraction grating 458 is positioned within waveguide 456 proximate surface 484 of waveguide 456 such that diffraction grating 458 can outcouple light 410 outside waveguide 456 and enable diffraction grating 458 to be directed toward image sensor 104 Light410. According to a specific example, at least one surface of diffraction grating 458 and surface 484 are on the same plane or at least partially define the same surface. According to one specific example, diffraction grating 458 is positioned in waveguide 456 on frame side 488 of waveguide 456 . According to one specific example, the frame side 488 of the waveguide 456 represents a portion of the waveguide 456 that is positioned at least partially within or on the surface of the frame of the HMD to enable outcoupling of light to the image sensor 104 .

Waveguide imaging system 450 optionally includes lens 440 positioned between waveguide system 452 and image sensor 104 . Lens 440 may be constructed from a single optical layer or may include multiple optical layers coupled together to focus light from diffraction grating 458 onto image sensor 104 . In one specific example, diffraction grating 458 and lens 440 are configured to focus light from first end 462 of diffraction grating 454 onto first end 442 of image sensor 104 and are configured to focus light from diffraction grating 454 to first end 442 of image sensor 104 . The light from the second end 464 of the grating 454 is focused onto the second end 444 of the image sensor 104 .

Figure 5 illustrates a diagram 500 for defining and constructing one or more features of waveguide systems 402 and/or 452 in accordance with an embodiment of the invention. In accordance with aspects of the present invention, diagram 500 illustrates incident on optical element 504 at multiple locations p (respectively, locations p ₁ , p ₂ , p ₃ , p _n ) to determine tilted grating plane 506 (respectively, locations p 1 , p 2 , p 3 , p n ). The diffraction angle, grating period and tilt angle of the ray 502 (individually, the rays 502A, 502B, 502C, 502D, 502E) of the tilted grating planes 506A, 506B, 506C). According to various aspects of the invention, optical element 504 may be a transmissive or reflective diffraction grating (eg, a holographic optical element).

To define the first tilt angle φ _p1 at the first point p ₁ , the diffraction angle θ ₁ of the first light ray 502A is defined as 80°. The first light 502A originates from the orbital region 208 at a distance d ₀ from the optical element 504 . The incident angle of the first ray 502A is 0°. The first tilt angle φ _p1 of the first tilt grating plane 506A can be adjusted until the diffraction angle θ ₁ of the first light ray 502A is 80°. The grating period Λ _p1 is the lateral distance between adjacent grating lines on the tilted grating plane 506 and is based on the wavelength of the selectively diffracted light (eg, 850 nm). Adjust the grating period Λ _p1 and the tilt angle φ _p1 at p ₁ until the diffraction angle θ ₁ of the first light 502A is 80°. Diffraction angle θ ₁ can be measured from the normal to the surface of optical element 504 (eg, the exit surface).

Once the grating period Λ _p1 and tilt angle φ _p1 of the first tilted grating plane 506A at the first point p ₁ have been determined to achieve a specific diffraction angle, the angular bandwidth θ _B,p1 at the first point p ₁ is determined. As an example, the angular bandwidth θ _B,p1 can be determined by directing various light rays at different incident angles at the first point p ₁ until the diffraction angle exceeds a predetermined threshold.

When the angular bandwidth θ _B,p1 has been determined, the second light 502B is emitted or guided to the first point p ₁ at the incident angle -θ _B,p1 /2 (negative θ divided by 2). Angle θ _2,3 is the resulting diffraction angle from the first point p ₁ of the second ray 502B. Angle θ _2,3 may be measured from the normal to the surface of optical element 504 (eg, the exit surface).

At the second point p ₂ , the grating period Λ _p2 and the tilt angle φ _p2 of the second inclined grating plane 506B are adjusted so that the third light ray 502C is also diffracted at the diffraction angle θ _2,3 . The third light ray 502C is emitted or directed to the second point p ₂ at an incident angle θ _B,p1 /2 (positive θ divided by 2). According to a specific example, the second point p ₂ is determined to be a distance w ₁₂ along the surface of the optical element 504 from the first point p ₁ . The distance w ₁₂ can be defined according to Equation 1, which is: w ₁₂ = 2*d ₀ * tan(θ _B,p1 /2).

In order to determine the grating period Λ _p3 and tilt angle φ _p3 at the third point p ₃ , the diffraction angle θ _4,5 is determined from the second point p ₂ . The diffraction angle θ _4,5 can be determined based on the angular bandwidth θ _B,p2 /2 of the second point p2. The angular bandwidth θ _B,p2 can be determined by directing various light rays from various incident angles to the second inclined grating plane 506B at the second point p ₂ until the diffraction angle exceeds a predetermined threshold value. The fourth ray 502D is emitted or guided toward the second point p2 at the incident angle -θ _B,p2 /2 (negative θ divided by 2), and the resulting diffraction angle of the fourth ray 502D is the diffraction angle θ _4,5 . Angle θ _4,5 can be measured from the normal to the surface of optical element 504 (eg, the exit surface).

The grating period Λ p3 and the tilt angle φ _p3 of the third _tilted grating plane 506C at the third point p3 are determined based at least in part on the diffraction angle θ _4,5 . According to one specific example, the third point p ₃ is determined as a distance w ₂₃ along the surface of the optical element 504 from the second point p ₂ . The distance w ₂₃ can be defined according to Equation 2, which is: w ₂₃ = 2*d ₀ * tan(θ _B,p2 /2).

The grating period Λ _p3 and the tilt angle φ _p3 are determined by adjusting the grating period Λ _p3 and the tilt angle φ _p3 until the fifth light ray 502E is diffracted from the tilted grating plane 506C at a diffraction angle θ _4,5 . According to a specific example, the fifth light 502E is emitted or guided toward the third point p ₃ at the incident angle θ _{B, p2} /2 (θ divided by 2), and the grating period Λ _p3 and the tilt angle φ _p3 are adjusted at the same time.

In accordance with specific examples of the present invention, the general sequence discussed for determining the characteristics of tilted grating plane 506 may be applied iteratively throughout the length of optical element 504 to produce an optical element having a tilted grating plane that operates as described herein. The described diffraction gratings (eg, diffraction gratings 404, 454) diffract light. This sequence can be repeated until a critical diffraction angle is reached where diffracted rays from the tilted grating plane no longer experience TIR within the waveguide.

In some embodiments, the process of identifying and defining characteristics of optical element 504 is performed by one or more processors configured to operate, for example, to record and/or test optics, according to various embodiments. Manufacturing or production equipment of components, diffraction gratings, waveguide systems, waveguide imaging systems and/or HMDs.

Figure 6 illustrates a graph 600 illustrating the optical properties of a waveguide system and a diffraction grating in accordance with aspects of the invention. According to one specific example, diagram 600 includes a top view of a waveguide system 602 operating at least partially at the rotation angle illustrated in diagram 604. According to a specific example, waveguide system 602 includes waveguide 606, in-coupling diffraction grating 608, and out-coupling diffraction grating 610. According to a specific example, waveguide system 602 is an example of a top view of waveguide systems 402 and/or 452 . According to a specific example, in-coupling diffraction grating 608 is an example top view of diffraction gratings 404 and/or 454 . According to a specific example, outcoupling diffraction grating 610 is a top view of an example of diffraction grating 408 and/or 458 .

According to one specific example, in-coupling diffraction grating 608 includes a tilted grating plane 612 that is arcuate and concavely curved with respect to the direction of out-coupling diffraction grating 610 . According to one specific example, the curvature of the tilted grating plane 612 directs the rays 614 (at various angles) toward the outcoupling diffraction grating 610 and enables the outcoupling diffraction grating 610 to have a smaller receiving surface area than the emitting surface area of the incoupling diffraction grating 608 surface area. According to one specific example, the smaller surface area of the outcoupling diffraction grating 610 makes it easier to conceal and place the outcoupling diffraction grating 610 within or on the frame of the HMD. The larger surface area of the incoupling diffraction grating 608 may enable the reception and incoupling of more light from the orbital region of the HMD or from the eyes of the user of the HMD. According to one specific example, graph 604 illustrates the degree of rotation experienced by a ray (eg, ray 614 ) based on positive and negative displacements of the ray along the x- and y-axes of the in-coupling diffraction grating 608 .

Figure 7 illustrates a process 700 for fabricating a rolled diffraction grating according to one specific example. According to a specific example, process 700 may be incorporated into one or more manufacturing systems including one or more processors and one or more laser controllers configured to convert the surrounding The radiation pattern is recorded in a recording medium to produce, for example, a volume grating. The order in which some or all of the process blocks appear in process 700 should not be considered limiting. Indeed, those of ordinary skill in the art having the benefit of this disclosure will understand that some of the process blocks may be executed in various orders not illustrated, or even in parallel.

According to one specific example, at process block 702 , the process 700 determines the operating distance to the optical element. The operating distance may be the distance between the optical element and the user's orbital area or eyes. The optical element can be a recording medium from which the holographic optical element can be manufactured. According to a specific example, process block 702 may proceed to process block 704.

According to one specific example, at process block 704, process 700 provides an initial ray with an initial angle of incidence to an initial point on the optical element. The initial angle of incidence can be 0°. According to a specific example, process block 704 may proceed to process block 706.

According to one specific example, at process block 706 , process 700 adjusts the initial grating period and/or tilt angle of the initial tilt grating plane at the initial point until the initial diffraction angle of the initial light ray reaches an initial threshold value. The initial threshold may be a predetermined threshold, such as 80°. According to a specific example, process block 706 may proceed to process block 708.

According to one specific example, at process block 708 , process 700 determines the angular bandwidth of the tilted grating plane. According to a specific example, process block 708 proceeds to process block 710.

According to one specific example, at process block 710 , process 700 provides the next ray at a point with an angle of incidence of -½ angular bandwidth and determines a diffraction angle for the next ray. According to a specific example, process block 710 proceeds to process block 712.

According to one specific example, at process block 712, the process 700 moves to a point below the optical element. According to a specific example, process block 712 proceeds to process block 714.

According to one specific example, at process block 714, process 700 provides the next ray at the next point with an angle of incidence of +½ angular bandwidth. According to a specific example, process block 714 proceeds to process block 716.

According to a specific example, at process block 716 , the process 700 adjusts the next tilted grating plane around the next point by the next grating period and/or the next tilt angle until the next ray is at the next diffraction angle determined at process block 710 Diffracted from the next tilted grating plane. According to a specific example, process block 716 proceeds to process block 708 until the next diffraction angle meets or exceeds the critical angle threshold.

Figure 8 illustrates a process 800 for eye tracking according to one specific example. Process 800 may be incorporated, at least in part, into one or more HMDs disclosed herein (eg, into controller 118). The order in which some or all of the process blocks appear in process 800 should not be considered limiting. Indeed, those of ordinary skill in the art having the benefit of this disclosure will understand that some of the process blocks may be executed in various orders not illustrated, or even in parallel.

According to one specific example, at process block 802 , process 800 directs light toward the orbital region to illuminate the eyes of the user of the HMD. Directing light toward the orbital region may include using one or more light sources (eg, LEDs) to emit infrared light toward the orbital region. According to a specific example, process block 802 may proceed to process block 804.

According to one specific example, at process block 804, process 800 utilizes a waveguide system to receive reflected light. The waveguide system may include any of the waveguide systems disclosed herein, and may include in-coupling and out-coupling diffraction gratings positioned on or within the waveguide. According to aspects of the invention, the in-coupling diffraction grating and/or the out-coupling diffraction grating may be rolled diffraction gratings. The waveguide system may be at least partially included in the lens member and may be at least partially positioned in the frame of the HMD. According to a specific example, process block 804 may proceed to process block 806.

According to one embodiment, at process block 806 , process 800 utilizes a waveguide system to guide the reflected light to an image sensor. Image sensors can be positioned in or on the frame of the HMD to receive reflected light from the waveguide system. According to a specific example, process block 806 may proceed to process block 808.

According to one embodiment, at process block 808 , process 800 utilizes an image sensor to receive reflected light from a waveguide system. The image sensor can convert the reflected light from an optical signal into an electrical signal, and save the electrical signal as image data or provide it to the controller. According to a specific example, process block 808 proceeds to process block 810.

According to one embodiment, at process block 810 , process 800 determines the orientation of the user's eyes based on image data representing reflected light.

9-13 illustrate specific examples of wavelength multiplexed waveguide systems and related processes in accordance with aspects of the present invention. Wavelength multiplexed waveguide systems can use multi-wavelength specific incoupling diffraction gratings to: i) encode the spatial location of reflections into angles, ii) wavelength encode portions of the orbital region, and iii) enlarge the area where reflections can be detected Orbital area of the eye.

Figure 9 illustrates a top view of a wavelength multiplexed waveguide system 900 in accordance with aspects of the invention. According to one specific example, wavelength multiplexed waveguide system 900 includes a plurality of in-coupled diffraction gratings that operate to expand the size or area of the orbital region in which reflections are detectable. A plurality of incoupling diffraction gratings are configured to operate at different wavelengths of light. The plurality of in-coupling diffraction gratings are configured to direct received light toward the out-coupling diffraction gratings. Each of the plurality of in-coupling diffraction gratings may have a different focal length such that when the in-coupling diffraction grating is positioned in different positions within the waveguide, the diffracted light is focused on the out-coupling grating. Multiple incoupling diffraction gratings may be wavelength-selective VBGs that reduce or eliminate crosstalk between two or more diffraction gratings. The out-coupling diffraction grating may be configured not to be wavelength selective, such that the out-coupling diffraction grating may out-couple any wavelength of light diffracted by the in-coupling diffraction grating. Each diffraction grating can be independently exposed to each wavelength during recording, which can allow tuning of δn (the amplitude of the index modulation) to achieve high diffraction efficiency in each of the incoupled diffraction gratings.

Wavelength multiplexed waveguide system 900 is an example implementation of waveguide system 102 (shown in Figure 1) and waveguide system 216 (shown in Figures 2 and 3). According to specific examples of the present invention, various techniques described above with respect to FIGS. 4A-8 may be applied to wavelength multiplexed waveguide system 900. According to a specific example, the wavelength multiplexed waveguide system 900 includes a first in-coupling diffraction grating 902 , a second in-coupling diffraction grating 904 and an out-coupling diffraction grating 906 disposed in the waveguide 908 .

A first incoupling diffraction grating 902 is positioned within the waveguide 908 and is configured to pass visible light and diffract light of a first wavelength 910 (eg, 1300 nm). The first in-coupling diffraction grating 902 is configured in a reflective operation or a transmissive operation to diffract the first wavelength light 910 toward the out-coupling diffraction grating 906 . The first wavelength light 910 propagates along the first optical path in the waveguide 908 toward the outcoupling diffraction grating 906 by total internal reflection (TIR). The first in-coupling diffraction grating 902 is configured to have a focal length F1 to focus the first wavelength light 910 onto the out-coupling diffraction grating 906 . The first in-coupling diffraction grating 902 may have a (eg, top view) surface area or two-dimensional footprint that is greater than the surface area or two-dimensional footprint of the out-coupling diffraction grating 906, such that the first in-coupling diffraction grating 902 may be configured to The first wavelength light 910 is focused along the longitudinal axis of the waveguide 908 (x-axis) and along the transverse axis of the waveguide 908 (y-axis).

To focus the first wavelength light 910 onto the out-coupling diffraction grating 906, the first in-coupling diffraction grating 902 may include a plurality of tilted grating planes 912. The tilted grating plane 912 may have a tilt angle that varies from the first end 914 to the second end 916 of the first incoupling diffraction grating 902 . The tilt angle of the tilted grating plane 912 enables the first in-coupling diffraction grating 902 to diffract the first wavelength light 910 at different angles depending on the incident light on the first in-coupling diffraction grating 902 . Each of the tilted grating planes 912 has grating lines and grating periods defined such that the first incoupling diffraction grating 902 transmits visible light and diffracts the first wavelength light 910 . Alternatively, according to a specific example, first in-coupling diffraction grating 902 may be configured to diffract a wavelength range (eg, 1250 to 1350 nm) centered around first wavelength light 910 (eg, 1300 nm). Tilted grating plane 912 may be arced toward outcoupling diffraction grating 906 and concavely curved to support focusing light laterally along the y-axis of waveguide 908. According to a specific example, the arc curvature of the inclined grating plane 912 can change from an arc with a smaller radius to an arc with a larger radius from the first end 914 to the second end 916 to support focusing the first wavelength light 910 onto the outcoupling diffraction grating 906.

A second incoupling diffraction grating 904 is positioned within the waveguide 908 and is configured to pass visible light and diffract light of a second wavelength 918 (eg, 940 nm). A second in-coupling diffraction grating 904 is positioned adjacent the first in-coupling diffraction grating 902 . The second in-coupling diffraction grating 904 may be in contact with the first in-coupling diffraction grating 902 , or a gap may exist between the in-coupling diffraction grating 902 and the in-coupling diffraction grating 904 . The first wavelength light 910 and the second wavelength light 918 may be separated by a buffer wavelength band (eg, a 200 nm or larger band) to reduce interference, and the two light wavelengths may be in the infrared or near-infrared range. The second in-coupling diffraction grating 904 may be configured in reflective operation or transmissive operation to diffract the second wavelength light 918 toward the out-coupling diffraction grating 906 . The first wavelength light 910 propagates along the second optical path in the waveguide 908 from the second in-coupling diffraction grating 904 toward the out-coupling diffraction grating 906 by means of TIR.

The second in-coupling diffraction grating 904 may be configured to have a focal length F2 to focus the second wavelength light 918 onto the out-coupling diffraction grating 906. Focal length F2 may be shorter than focal length F1. The second in-coupling diffraction grating 904 may have a (eg, top view) surface area or two-dimensional footprint that is greater than the surface area or two-dimensional footprint of the out-coupling diffraction grating 906, such that the second in-coupling diffraction grating 904 may be configured to The second wavelength light 918 is focused along the longitudinal axis of the waveguide 908 (x-axis) and along the transverse axis of the waveguide 908 (y-axis).

To focus the second wavelength light 918 onto the out-coupling diffraction grating 906, the second in-coupling diffraction grating 904 may include a plurality of tilted grating planes 920. The tilted grating plane 920 may have a tilt angle that changes from the first end 922 to the second end 924 of the second incoupling diffraction grating 904 . The tilt angle of the tilted grating plane 920 enables the second in-coupling diffraction grating 904 to diffract the second wavelength light 918 at different angles depending on the incident light on the second in-coupling diffraction grating 904 . Each of the tilted grating planes 920 has grating lines and grating periods defined such that the second incoupling diffraction grating 904 transmits visible light and diffracts secondary wavelength light 918 . Alternatively, according to a specific example, second in-coupling diffraction grating 904 may be configured to diffract a wavelength range (eg, 890 to 990 nm) centered around second wavelength light 918 (eg, 940 nm). Tilted grating plane 920 may be arced toward outcoupling diffraction grating 906 and concavely curved to support focusing light laterally along the y-axis of waveguide 908. According to a specific example, the arc curvature of the inclined grating plane 920 can change from an arc with a smaller radius to an arc with a larger radius from the first end 922 to the second end 924 to support focusing the second wavelength light 918 onto the outcoupling diffraction grating 906.

External coupling diffraction grating 906 is positioned in waveguide 908 and configured to receive first wavelength light 910 and second wavelength light 918 . The out-coupling diffraction grating 906 may be positioned a focal length F1 distance from the first in-coupling diffraction grating 902 and may be positioned a focal distance F2 distance from the second in-coupling diffraction grating 904. Outcoupling diffraction grating 906 may be configured (eg, with a tilted grating plane) to pass visible light and diffract both first wavelength light 910 and second wavelength light 918 . The outcoupling diffraction grating 906 may be enclosed in the frame of the HMD and block external light, so the outcoupling diffraction grating 906 may be configured to diffract wavelength ranges (eg, near-infrared, infrared, and/or visible light). The out-coupling diffraction grating 906 may be transmissive or reflective to diffract the first wavelength light 910 and the second wavelength light 918 .

Out-coupling diffraction grating 906 may have a smaller surface area than either of first in-coupling diffraction grating 902 and second in-coupling diffraction grating 904. According to one specific example, the smaller surface area of the outcoupling diffraction grating 906 makes it easier to conceal and place the outcoupling diffraction grating 906 within or on the frame of the HMD. The larger surface area of the in-coupling diffraction gratings 902, 904 may enable the reception and in-coupling of more light from the orbital region of the HMD or from the eyes of the user of the HMD. According to aspects of the present invention, although wavelength multiplexed waveguide system 900 depicts two in-coupling diffraction gratings, waveguide 908 may include two, three, or more in-coupling diffraction gratings (each configured to diffraction Different light wavelengths) to expand the area that can internally couple the light reflection from the orbital area of the eye.

10A, 10B, and 10C illustrate detailed side views of a wavelength multiplexed waveguide imaging system 1000 in accordance with aspects of the present invention. Wavelength multiplexed waveguide imaging system 1000 includes wavelength multiplexed waveguide system 1002, which is an example implementation of wavelength multiplexed waveguide system 900 (shown in Figure 9). Wavelength multiplexed waveguide system 1002 may be configured to diffract light using reflection operations similar to those described above with reference to waveguide imaging system 450 (shown in Figure 4B), for example. According to a specific example, the wavelength multiplexed waveguide system 1002 includes a first in-coupling diffraction grating 1004, a second in-coupling diffraction grating 1006 and an out-coupling diffraction grating 1008 positioned in the waveguide 1010. According to one specific example, wavelength multiplexed waveguide system 1002 is configured to redirect multiple wavelengths of reflected light from eye 204 and/or eye orbit 1012 onto image sensor 104 . By implementing two or more incoupled diffraction gratings, the wavelength multiplexed waveguide system 1002 can operate to detect light from a larger area (e.g., from a larger orbital region) than traditional or other eye tracking techniques. of reflected light.

FIG. 10A illustrates an example of light diffraction by the first in-coupling diffraction grating 1004. The first wavelength light 910 may include light ray 1014 and light ray 1016 representing the first wavelength light 910 . Light rays 1014 and 1016 have a first wavelength, and first in-coupling diffraction grating 1004 is configured to diffract light having the first wavelength. However, the propagation or transmission paths of light rays 1014 and 1016 are not affected by the second in-coupling diffraction grating 1006 because the second in-coupling diffraction grating 1006 can be configured to pass light having the first wavelength (without operating on it ). To illustrate this situation, the second in-coupling diffraction grating 1006 is depicted in dashed lines.

The first incoupling diffraction grating 1004 receives the first wavelength light 910 from the eye orbit region 1012 . According to one specific example, the first in-coupling diffraction grating 1004 can be configured to receive the first wavelength light 910 from the first portion 1018 of the eye movement orbital region 1012, and the second in-coupling diffraction grating 1006 can be configured to receive the eye movement from the first portion 1018 of the orbital region 1012. The second portion 1020 of the orbital region 1012 receives the second wavelength light 918 . The first incoupling diffraction grating 1004 diffracts light rays 1014 and 1016 in the waveguide 1010 . Light rays 1014 and 1016 are reflected in waveguide 1010 and propagate to outcoupling diffraction grating 1008 . Outcoupling diffraction grating 1008 diffracts light rays 1014 and 1016 toward image sensor 104 (eg, via lens 440 ).

FIG. 10B illustrates an example of light diffraction by the second in-coupling diffraction grating 1006. The second wavelength light 918 may include light rays 1022 and 1024 representing the second wavelength light 918 . Light rays 1022 and 1024 have a second wavelength, and second in-coupling diffraction grating 1006 is configured to diffract light having the second wavelength. However, the propagation or transmission paths of light rays 1022 and 1024 are not affected by the first in-coupling diffraction grating 1004 because the first in-coupling diffraction grating 1004 can be configured to pass light having a second wavelength (without operating on it ). To illustrate this situation, the first in-coupling diffraction grating 1004 is depicted in dashed lines.

The second incoupling diffraction grating 1006 receives the second wavelength light 918 from the orbital region 1012 . The second incoupling diffraction grating 1006 diffracts light rays 1022 and 1024 in the waveguide 1010 . Light rays 1022 and 1024 are reflected in waveguide 1010 and propagate to outcoupling diffraction grating 1008 . Outcoupling diffraction grating 1008 diffracts (eg, via lens 440 ) light rays 1022 and 1024 toward image sensor 104 .

10C illustrates an example of wavelength multiplex diffraction by a first in-coupling diffraction grating 1004 and a second in-coupling diffraction grating 1006 in accordance with aspects of the present invention. Based on the configuration of the first in-coupling diffraction grating 1004 and the second in-coupling diffraction grating 1006, the first wavelength light 910 and the second wavelength light 918 are each diffracted by one of the in-coupling diffraction gratings instead of both. By implementing two or more in-coupled diffraction gratings, the wavelength multiplexed waveguide imaging system 1000 can enable in-field eye tracking operations, including eye orientation from a larger expanded orbital region 1012 (compared to conventional techniques) information.

Considering power consumption, a variety of illumination techniques may be used to illuminate the orbital region 1012. In one specific example, the light source may be operated to illuminate one side of the eye orbital region with light of a first wavelength or the other side of the eye orbital region with light of a second wavelength, thus wavelength encoding portions of the eye orbital region. In one specific example, the HMD operates the light source to illuminate a portion of the orbital area in which the eye is currently oriented to reduce wasted power associated with illuminating a portion of the orbital area in which the eye is not oriented. In one specific example, light sources of two wavelengths are temporarily illuminated until eye orientation is initially determined. In one specific example, the HMD operates the light source to periodically alternate between a first wavelength of light and a second wavelength of light. The HMD may be configured to operate the light source to initially provide two light wavelengths and then switch to a lower power illumination sequence or mode based on, for example, the determined orientation of the eye.

Although one outcoupling diffraction grating is shown, in some embodiments, wavelength multiplexed waveguide imaging system 1000 may include multiple outcoupling diffraction gratings. Multiple outcoupling diffraction gratings can each be tuned to diffract specific wavelengths of light. For example, a first outcoupling diffraction grating may be configured to diffract light of a first wavelength, and a second outcoupling diffraction grating may be configured to diffract light of a second wavelength. The first and second output coupling gratings can be positioned side by side or one behind the other. More than two outcoupling diffraction gratings may be included to wavelength encode the eye orbital region with more than two light wavelengths.

Although a single image sensor is shown, image sensors for each wavelength may be used interchangeably. Each image sensor may be optically coupled to receive light from a corresponding out-coupled diffraction grating. In one embodiment, a lens may be used to direct light of a first wavelength to a first image sensor and direct light of a second wavelength to a second image sensor. Additional image sensors can be used to support wavelength multiplexing with additional light wavelengths.

Figure 11 illustrates a process 1100 for eye tracking according to one specific example. Process 1100 may be incorporated, at least in part, into one or more HMDs disclosed herein (eg, into controller 118). The order in which some or all of the process blocks appear in process 1100 should not be considered limiting. Indeed, those of ordinary skill in the art having the benefit of this disclosure will understand that some of the process blocks may be executed in various orders not illustrated, or even in parallel.

According to one specific example, at process block 1102 , the process 1100 directs a first light having a first wavelength and a second light having a second wavelength toward an orbital region to illuminate the eyes of a user of the HMD. Directing the first light and the second light toward the orbital region may include using two or more light sources (eg, LEDs) to emit infrared light toward the orbital region. According to a specific example, process block 1102 may proceed to process block 1104.

According to a specific example, at process block 1104, process 1100 utilizes a waveguide system to receive reflected light. The reflected light includes first light and second light having a first wavelength and a second wavelength respectively. The waveguide system may include any of the waveguide systems disclosed herein, and may include at least two incoupling diffraction gratings to incouple the first light and the second light from the orbital region of the eye. The waveguide system may also include an out-coupled diffraction grating to guide light to the image sensor. According to aspects of the invention, the diffraction grating may be positioned in the waveguide and may be a rolled diffraction grating. The waveguide system may be at least partially included in the lens member and may be at least partially positioned in the frame of the HMD. According to a specific example, process block 1104 may proceed to process block 1106.

According to one specific example, at process block 1106 , process 1100 utilizes a waveguide system to guide the reflected light to an image sensor. The image sensor can be positioned in or on the frame of the HMD to receive reflected light from a coupling diffraction grating external to the waveguide system. According to a specific example, process block 1106 may proceed to process block 1108.

According to one specific example, at process block 1108, process 1100 utilizes an image sensor to receive reflected light from a waveguide system. The image sensor can convert the reflected light from an optical signal into an electrical signal, and save the electrical signal as image data or provide it to the controller. According to a specific example, process block 1108 proceeds to process block 1110.

According to one specific example, at process block 1110 , process 1100 determines the orientation of the user's eyes based on image data representing reflected light of a first wavelength and a second wavelength.

Figure 12 illustrates a perspective view of the interaction of light with a wavelength multiplexed waveguide system 1200 in accordance with aspects of the invention. Wavelength multiplexed waveguide system 1200 displays light 1202 from the orbital region of the eye. According to one specific example, light 1202 is received through a waveguide 1204 having a plurality of in-coupling diffraction gratings 1206 that direct portions of the light 1202 toward an out-coupling diffraction grating 1208.

Figure 13 illustrates a diagram 1300 representing the spatial location of eye reflections mapped to corners in a waveguide, in accordance with aspects of the invention. Light clusters 1302 and 1304 represent the spatial locations of light received by multiple in-coupling diffraction gratings 1206 and then redirected (in the waveguide) to out-coupling diffraction grating 1208. Light cluster 1302 represents light received from one of the plurality of in-coupling diffraction gratings 1206, and light cluster 1304 represents light received from another of the plurality of in-coupling diffraction gratings 1206. The position, number, and pattern formed by light clusters 1302 and 1304 may change as the orientation of the eye in the orbital region changes.

Diagram 1300 includes the x-axis angle and the y-axis angle of the mapped light. The x-axis angle represents the diffraction angle of the in-coupling diffraction grating diffracting incident light in the longitudinal x-axis direction of the waveguide. The y-axis angle represents the diffraction of incident light along the transverse y-axis (toward the center of the waveguide). The x-axis angle can also represent the angle at which light is incident on the out-coupling diffraction grating along the x-axis of the waveguide. The y-axis angle can also represent the angle at which light is incident on the external coupling diffraction grating along the y-axis of the waveguide. Light clusters 1302 and 1304 show that the in-coupling diffraction grating encodes (or maps) the spatial position of the eye reflection into the diffraction angle within the waveguide, and shows that the out-coupling grating decodes the position into light directed to the image sensor. Light clusters 1302 and 1302 may represent clusters of pixels in an image sensor that respond to light from an outcoupling diffraction grating. Map 1300 can be used to help decode the origin of specific reflections of light, and can be viewed as a decoded map that can be used to reconstruct images of the orbital region of the eye. Additionally, because different wavelengths can be used to illuminate different portions of the eye orbit, in accordance with specific examples of the present invention, the disclosed techniques can use different wavelengths to encode (position to angle) different portions of the eye orbit.

Specific examples of the invention may include artificial reality systems or may be implemented in conjunction with artificial reality systems. Artificial reality is a form of reality that has been adjusted in a certain way before being presented to the user. It can include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), mixed reality or a combination and/or derivative thereof. Artificial reality content may include fully generated content or generated content combined with captured (eg, real-world) content. Artificial reality content may include video, audio, haptic feedback, or some combination thereof, and any of these may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect on the viewer). In addition, in some specific examples, artificial reality may also be related to applications, products, such as those used to generate content in the artificial reality and/or otherwise used in the artificial reality (e.g., performing activities in the artificial reality). accessories, services, or some combination thereof. Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs) connected to host computer systems, stand-alone HMDs, mobile devices or computing systems or capable of providing artificial reality content to Any other hardware platform for one or more viewers.

The term "processing logic" (for example, controller 118, processing logic 120) in the present invention may include one or more processors, microprocessors, multi-core processors, application-specific integrated circuits (Application-specific integrated circuits); ASIC) and/or Field Programmable Gate Array (FPGA) to perform the operations disclosed herein. In some embodiments, memory (not shown) is integrated into the processing logic to store instructions for performing operations and/or to store data. Processing logic may also include analog or digital circuitry to perform operations in accordance with embodiments of the invention.

The "memory or memories" described in this disclosure (eg, memory 122) may include one or more volatile or non-volatile memory structures. "Memory or memories" may be removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data. Example memory technologies may include RAM, ROM, EEPROM, flash memory, CD-ROM, digital versatile disk (DVD), high-definition multimedia/data storage disk or other optical storage, magnetic disk, Tape, disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device.

Computing devices may include desktop computers, laptop computers, tablet computers, phablets, smartphones, feature phones, server computers, or others. Server computers can be located remotely in a data center or stored locally.

The process explained above is described with respect to computer software and hardware. The techniques described may constitute machine-executable instructions embodied in a tangible or non-transitory machine (eg, computer) readable storage medium that, when executed by a machine, cause the machine to perform the described operations. Additionally, the process may be embodied in hardware, such as an application specific integrated circuit ("ASIC") or otherwise.

Tangible non-transitory machine-readable storage media includes any mechanism that provides (i.e., stores) information that can be used by a machine (e.g., a computer, a network device, a personal digital assistant, a manufacturing tool, a machine having one or more processors) any device, etc.) access form. For example, machine-readable storage media include recordable/non-recordable media (e.g., read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

The above descriptions of illustrated embodiments of the invention, including what is described in the Abstract, are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Although specific embodiments and examples of the invention are described herein for illustrative purposes, various modifications are possible within the scope of the invention, as those of ordinary skill in the art would recognize.

Such modifications may be made to the invention in view of the above detailed description. The terminology used in the following claims should not be construed as limiting the invention to the specific embodiments disclosed in this specification. The reality is that the scope of the present invention should be determined entirely by the following patent application scope, which will be interpreted in accordance with the established principles of patent application scope interpretation.

100, 202, 300: Head mounted display 102, 114, 216, 402, 452, 602: Waveguide system 104, 116: Image sensor 106:Frame 108, 124, 214, 302: Lens components 110A, 110B: Arm 112:Light source 118:Controller 120: Processing logic 122:Memory 126:Central area 200: Eye environment 204:eyes 206: Eye movement orbital side 208: Orbital area of eye movement 210: Scene light 212: Scene side 218:Non-visible light 220, 432, 434, 470, 474, 482, 484: surface 222, 406, 456, 606, 908, 1010, 1204: Waveguide 224, 608, 902, 904, 1004, 1206: Internal coupling diffraction grating 226, 610, 906, 1006, 1008, 1208: external coupling diffraction grating 228:Image data 230: Communication channel 232:Projector 234:Display 304:Waveguide optical layer 306: Display optical layer 308, 310: Optical layer 400, 450: Waveguide imaging system 404, 408, 454, 458: Diffraction grating 410, 1202: light 412, 442, 462, 914, 922: first end 414, 444, 464, 916, 924: second end 416, 418, 502, 502A, 502B, 502C, 502D, 502E, 614, 1014, 1016, 1022, 1024: light 420, 424: incident surface 422, 426: Exit surface 428, 478: first side 430, 480: Second side 436, 486: Lens member side 438, 488: Frame side 440:Lens 472, 472A, 472B, 472C, 506, 506A, 506B, 506C, 612, 912, 920: inclined grating plane 500, 600, 1300: Figure 504:Optical components 604: Chart 700, 800, 1100: process 702, 704, 706, 708, 710, 712, 714, 716, 802, 804, 806, 808, 810, 1102, 1104, 1106, 1108, 1110: Process block 900, 1002, 1200: Wavelength multiplexed waveguide system 910: first wavelength light 918: Second wavelength light 1000: Wavelength multiplexed waveguide imaging system 1012: eye movement 1018:Part One 1020:Part 2 1302, 1304: light cluster F1, F2: focal length

Non-limiting and non-exhaustive examples of the present invention are described with reference to the following drawings, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified. [Fig. 1] illustrates a head-mounted display according to an aspect of the present invention. [Fig. 2] illustrates an example implementation of a lens member for a head-mounted display according to an aspect of the present invention. [Fig. 3] illustrates an example implementation of a lens member according to an aspect of the present invention. [FIG. 4A] and [FIG. 4B] illustrate example implementations of a waveguide system that may be used in an HMD to support eye-tracking operations, in accordance with aspects of the present invention. [Fig. 5] A diagram illustrating a technique for defining characteristics of a rolled diffraction grating according to an aspect of the present invention. [Fig. 6] A diagram illustrating a top view of a waveguide system according to an aspect of the present invention and a rotation angle diagram for a rolled diffraction grating. [Fig. 7] A flowchart illustrating a process for manufacturing a rolled diffraction grating according to an aspect of the present invention. [Fig. 8] A flowchart illustrating a process for eye tracking according to aspects of the present invention. [Fig. 9] A diagram illustrating a top view of a wavelength multiplexed waveguide system according to an aspect of the present invention. [FIG. 10A], [FIG. 10B], and [FIG. 10C] illustrate example light propagation in a wavelength multiplexed waveguide system that may be used in an HMD to support eye tracking operations, in accordance with aspects of the present invention. [Fig. 11] A flowchart illustrating a process for eye tracking according to aspects of the present invention. [Fig. 12] A perspective view illustrating the interaction of light with a wavelength multiplexed waveguide system according to an aspect of the present invention. [FIG. 13] A diagram illustrating the spatial position of eye reflections mapped to corners in the wavelength multiplexed waveguide system of FIG. 12, in accordance with aspects of the present invention.

104:Image sensor

106:Frame

112:Light source

118:Controller

200: Eye environment

202:Head mounted display

204:eyes

206: Eye movement orbital side

208: Orbital area of eye movement

210: Scene light

212: Scene side

214:Lens component

216:Waveguide system

218:Non-visible light

220:Surface

222:Waveguide

224: In-coupled diffraction grating

226: External coupling diffraction grating

228:Image data

230: Communication channel

232:Projector

234:Display

Claims (20)

A lens component containing: waveguide; a first diffraction grating disposed in the waveguide and configured to couple first light from within the orbital region into the waveguide, the first light having a first wavelength; and A second diffraction grating disposed in the waveguide and configured to couple second light from within the orbital region into the waveguide, the second light having a second wavelength. The lens component of claim 1 further includes: A third diffraction grating disposed in the waveguide and configured to outcouple the first light and the second light from the waveguide to the image sensor, wherein the first diffraction grating has a function of coupling the first light to the image sensor. Light is focused to a first focal length of the third diffraction grating, and wherein the second diffraction grating has a second focal length to focus the second light to the third diffraction grating, the first focal length being longer than the second focal length . The lens component of claim 2, wherein the two-dimensional occupied area of the third diffraction grating is smaller than that of the first diffraction grating and the second diffraction grating. The lens component of claim 2, wherein the first optical path of the first light focused onto the third diffraction grating relies on total internal reflection (TIR) of the waveguide to propagate to the third diffraction grating, and wherein The second optical path of the second light focused on the third diffraction grating relies on the TIR of the waveguide to propagate to the third diffraction grating. The lens member of claim 1, wherein the first diffraction grating is positioned in the waveguide adjacent the second diffraction grating, wherein the first diffraction grating is configured to incouple from the first portion of the orbital region The first light, wherein the second diffraction grating is configured to incouple the second light from the second portion of the orbital region. The lens component of claim 1, wherein the first wavelength is approximately 1300 nm and the second wavelength is approximately 940 nm. The lens member of claim 1, wherein the orbital region is illuminated by a first light source configured to emit light at approximately 1300 nm and a second light source configured to emit light Light at approximately 940 nm. The lens component of claim 1, wherein the first diffraction grating is a first holographic optical element, wherein the second diffraction grating is a second holographic optical element, and wherein the first holographic optical element includes a configuration configured to A first plurality of tilted grating planes that transmit visible light and diffract the first light toward a third diffraction grating in the waveguide, wherein the second holographic optical element includes a first plurality of tilted grating planes configured to transmit visible light and toward the third diffraction grating in the waveguide. The triple diffraction grating diffracts a second plurality of inclined grating planes of the second light. The lens member of claim 8, wherein the first plurality of inclined grating planes are configured to diffract the first light with a first focal length, and wherein the second plurality of inclined grating planes are configured to diffract the first light with a second focal length. Second light. The lens component of claim 1, wherein the first diffraction grating is configured to be inoperable with the second light, and wherein the second diffraction grating is configured to be inoperable with the first light. The lens component of claim 1, wherein at least one of the first diffraction grating and the second diffraction grating operates diffractively in a reflective manner. The lens component of claim 1, wherein the first diffraction grating and the second diffraction grating are volume Bragg gratings. An eye tracking system consisting of: a controller configured to determine eye orientation based on the image data; Waveguide system, which includes: waveguide; a first diffraction grating disposed in the waveguide and configured to couple first light from within the orbital region into the waveguide, the first light having a first wavelength; and a second diffraction grating disposed in the waveguide and configured to couple second light from within the orbital region into the waveguide, the second light having a second wavelength; and An image sensor is optically coupled to the waveguide system to receive the first light and the second light from the waveguide, wherein the image sensor is configured to generate the image data. For example, the eye tracking system of claim 13 further includes: A third diffraction grating disposed in the waveguide and configured to outcouple the first light and the second light from the waveguide to the image sensor, wherein the first diffraction grating has a function of coupling the first light and the second light from the waveguide to the image sensor. A light is focused to a first focal length of the third diffraction grating, and wherein the second diffraction grating has a second focal length to focus the second light to the third diffraction grating, the first focal length being longer than the second focal length. The eye tracking system of claim 14, wherein the two-dimensional occupied area of the third diffraction grating is smaller than that of the first diffraction grating and the second diffraction grating. For example, the eye tracking system of claim 13 further includes: a first light source configured to emit the first light at the first wavelength toward the orbital region; and A second light source configured to emit the second light toward the orbital region at the second wavelength. The eye tracking system of claim 16, wherein the first wavelength is approximately 1300 nm and the second wavelength is approximately 940 nm. The eye tracking system of claim 16, wherein the controller is configured to selectively operate the first light source or the second light source based on the eye orientation in the orbital region. A head-mounted device containing: frame; a lens member coupled to the frame and configured to transmit scene light to the orbital region of the eye; A waveguide system coupled to the lens member and to the frame, wherein the waveguide system includes: waveguide; A first diffraction grating disposed in the waveguide and configured to couple first light from within the orbital region into the waveguide, the first light having a first wavelength; and a second diffraction grating disposed in the waveguide and configured to couple second light from within the orbital region into the waveguide, the second light having a second wavelength; and An image sensor is positioned in the frame to generate image data using the first light and the second light received from the waveguide. For example, the head mounted device of claim 19 further includes: A third diffraction grating disposed in the waveguide and configured to outcouple the first light and the second light from the waveguide to the image sensor, wherein the first diffraction grating has a function of coupling the first light and the second light from the waveguide to the image sensor. A light is focused to a first focal length of the third diffraction grating, and wherein the second diffraction grating has a second focal length to focus the second light to the third diffraction grating, the first focal length being longer than the second focal length.