TW202312731A - Angular light sensor and eye-tracking - Google Patents

Abstract

Angular sensors that may be used in eye-tracking systems are disclosed. An eye-tracking system may include a plurality of light sources to emit illumination light and a plurality of angular light sensors to receive returning light that is the illumination light reflecting from an eyebox region. The angular light sensors may output angular signals representing an angle of incidence of the returning light.

Description

Angle light sensor and eye tracking

The present invention relates generally to optics, and in particular to angle light sensors. Cross References to Related Applications

This application claims priority to U.S. Provisional Application No. 63/232,674, filed August 13, 2021, and U.S. Non-Provisional Application No. 17/878,634, filed August 1, 2022, which are hereby incorporated by reference incorporated in a manner.

Angle light sensors can be used in various scenarios, such as imaging, display and photovoltaic. Existing angle light sensors may be bulky, power inefficient, and/or have slow processing times. Therefore, the applications in which existing angle light sensors can be deployed may be limited.

An aspect of the present invention is an eye tracking system, which includes: a plurality of infrared light sources configured to emit infrared light to the window area; and a plurality of angle light sensors configured to receive returned infrared light, The returned infrared light is the infrared irradiation light reflected from the window area, wherein the plurality of angle photosensors are configured to output angle signals representing the incident angles of the returned infrared light relative to the positions of the angle photosensors.

In the eye-tracking system of the described aspect of the present invention, the angle light sensors have an angle detection range between 1 degree and 85 degrees relative to the photodetector surface of the angle light sensors.

In the eye-tracking system of the described aspect of the invention, the angle light sensors have a sensor area of less than 150 microns by 150 microns.

In the eye tracking system of said aspect of the present invention, the angle light sensors include: a first photodiode configured to receive the returned infrared light; a second photodiode configured to receive the return infrared light; and an inclined light barrier disposed between the first photodiode and the second photodiode, the inclined light barrier is opposite to the first photodiode and the second photodiode The surface normal to the sensing plane common to both polar bodies is tilted, wherein the ratio of the first signal generated by the first photodiode to the second signal generated by the second photodiode is indicative of the returned infrared light The angle of incidence.

In the eye tracking system according to the aspect of the present invention, the angle light sensors include: a light barrier; a first photodiode configured to receive the returned infrared light; a second photodiode configured to receive the returned infrared light; configured to receive the returned infrared light, wherein the first photodiode is disposed between the light barrier and the second photodiode; and processing logic configured to receive light generated by the first photodiode a first signal and a second signal generated by the second photodiode, wherein the processing logic generates a centroid value based on the first signal and the second signal, wherein the first signal is assigned a first weighting factor, the The first weighting factor is smaller than the second weighting factor assigned to the second signal when generating the centroid value.

In the eye-tracking system of the described aspect of the present invention, the field of view (FOV) of the angular detection range of the angular photosensors is inclined with respect to the surface normal of the photodetectors of the angular photosensors .

The eye tracking system of the aspect of the present invention further includes: a tilt mechanism configured to dynamically tilt the field of view (FOV) of the angle detection range of the angle photosensors.

In the eye-tracking system of said aspect of the invention, the tilt mechanism includes a micro-electro-mechanical system (MEMS) device.

In the eye-tracking system of said aspect of the present invention, each infrared light source is paired with an angle light sensor and spaced apart by less than 500 microns.

Another aspect of the present invention is a head-mounted device, which includes: a frame, which is used to fix the head-mounted device on the user's head; and an eye-tracking system, which includes: a plurality of infrared light sources , which is configured to emit infrared irradiating light to the window area; and a plurality of angle photosensors, which are configured to receive return infrared light, which is the infrared irradiating light reflected from the window area, wherein the plurality of The angle light sensors are configured to output angle signals representing the angle of incidence of the returning infrared light relative to the positions of the angle light sensors.

In the head-mounted device according to another aspect of the present invention, the plurality of infrared light sources and the plurality of angle light sensors are installed on the frame of the head-mounted device.

The head-mounted device according to another aspect of the present invention further includes: a lens held by the frame, wherein the lens transmits visible scene light from the external environment to the window area, and wherein the plurality of infrared light sources and The plurality of angle light sensors are arranged on the lens.

Another aspect of the present invention is an angle light sensor, which includes: a first light detector, which includes a first photodiode, a first high grating and a first low grating, wherein the first low grating is arranged between the first high grating and the first photodiode; and a second photodetector comprising a second photodiode, a second high grating, and a second low grating, wherein the second low grating Disposed between the second high grating and the second photodiode, wherein the second low grating is centrifugal relative to the second high grating, and wherein the first low grating is centered relative to the first high grating.

In the angular light sensor according to another aspect of the present invention, the second low grating is centrifugal with a shift factor of d /4 relative to the second high grating, where d is the second high grating, the The distance between the second low grating, the first low grating and the first high grating.

The angle light sensor of another aspect of the present invention further includes: a third light detector, which includes a third photodiode, a third high grating and a third low grating, wherein the third low grating disposed between the third high grating and the third photodiode, wherein the third low grating is centrifugal with respect to the third high grating by a shift factor of 3 d /8, where d is the third high grating Distance from the third low raster.

In the still further aspect of the angle photosensor of the present invention, the spacing has a 50% duty cycle.

The angle photodetector according to another aspect of the present invention further includes: a fourth photodetector configured as the first photodetector; a fifth photodetector configured as the second photodetector detector; and a sixth photodetector configured as the third photodetector, wherein the fourth photodetector, the fifth photodetector and the sixth photodetector are relative to the The first light detector, the second light detector and the third light detector are rotated 90 degrees.

In the angular light sensor of said still another aspect of the present invention, the grating material is chrome or copper for the second high grating, the second low grating, the first low grating, and the first high grating .

In the angle light sensor according to another aspect of the present invention, the angle light sensor is configured to measure an incident angle of light having a wavelength λ, and wherein the first low grating and the first high grating separated by a distance z , where z is 2 d ² /λ, and further wherein the second lower grating is also spaced apart from the second upper grating by the distance z .

In the angular light sensor according to another aspect of the present invention, an optically transparent substrate is disposed between the first high grating and the first low grating, and wherein the optically transparent substrate is also disposed between the second high grating between the raster and the second low raster.

Specific examples of angle light sensors and eye tracking are described herein. In the following description, numerous specific details are set forth in order to provide a thorough understanding of specific examples. One having ordinary skill in the relevant art will recognize, however, that the techniques described herein may be practiced without one or more of the specific details, or with other methods, components, materials, and the like. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

In some implementations of the invention, the term "near-to-eye" may be defined to include elements configured to be placed within 50 mm of a user's eye when utilizing a near-to-eye device. Thus, a "near-eye optic" or "near-eye system" would include one or more elements configured to be placed within 50 mm of a user's eye.

In aspects of the invention, visible light can be defined as having a wavelength range of approximately 380 nm to 700 nm. Invisible light can be defined as light having wavelengths outside the visible range, such as ultraviolet light and infrared light. Infrared light having a wavelength range of about 700 nm to 1 mm includes near-infrared light. In aspects of the invention, near-infrared light can be defined as having a wavelength range of approximately 700 nm to 1.6 μm.

In aspects of the invention, the term "transparent" may be defined as having a light transmittance greater than 90%. In some aspects, the term "transparent" may be defined as a material having a visible light transmission of greater than 90%.

In certain fields, including augmented reality (AR) and virtual reality (VR), the development of light sensors with lower power consumption, smaller form factor and Light sensors with/or shorter processing times are desirable. Solutions that do not require cameras such as photosensor oculography (PSOG) or position-sensitive detectors are particularly attractive because they require less processing time and can be compared to camera-based eye-tracking. The system detects eye movement faster. Additionally, being able to detect fixed eye movement (the most subtle type of eye movement) is of further interest as it carries meaningful cognitive and attentional information that can be used in AR/VR systems.

A potential non-invasive solution for small form factor, low power, low computation, fast eye tracking is to use a miniature sized angle sensor to detect light from a near-eye light source that has been reflected onto the surface of the cornea. Both the light source and the sensor may be placed in-field on (eg, in the lens of) the near-eye optics of a head-mounted device such as smart glasses or AR glasses. "In-field" means that light sources and sensors can be placed in the field of view of the user of the headset, such as the lens of near-eye optics held by the frame of the headset, where the user will see through the lens to view its external environment.

FIG. 1 shows a light source sensor pair for determining eye gaze angle according to an implementation of the present invention. FIG. 1 includes a light source 110 and a sensor 120 . The light source 110 can be an LED, a vertical-cavity surface-emitting laser (VCSEL), a photonic integrated circuit (PIC) with a luminous emission aperture, or others. The light source 110 may be an infrared light source. The light source 110 may be a near-infrared light source. According to the implementation of the present invention, the sensor 120 can be an angle light sensor. The sensor 120 and the light source 110 act as a sensor light source pair to obtain the eye gaze angle in the y direction by measuring the incident angle on the sensor 120 . Several sensors and light sources can be used simultaneously or by temporarily controlling the light emission. In some implementations, the light sources and sensors are not "in-field," but placed on the frame of smart glasses, AR glasses, or VR headsets.

Desirable attributes of an angular light sensor for use in AR/VR applications may include a miniature sensor (if it is placed in-field), high angular resolution, relatively large angular range, peak off-axis sensitivity, and/or Increased calculation speed. These properties would be especially desirable for angular light sensors in eye-tracking contexts.

FIG. 1 shows a light source 110 that illuminates a window region 195 with illuminating light 113 . The illuminating light 113 may be reflected/scattered as return light 117 from the cornea 194 of the eye 190 occupying the window region 195 . Point 192 represents the center of rotation of eye 190 and can be used as a reference for measuring eye gaze angle θ _gy . The light source 110 is selectively actuatable to emit illumination light 113 such that the light source 110 can be switched on and off. The gaze angle θ _gy of direct viewing is measured by using the light source 110 and the angle light sensor 120 , wherein the light source 110 and the angle light sensor are arranged at an eye-relief plane with an eye-relief distance 181 . In some examples, eye distance 181 may be approximately 16 mm or 20 mm. In the example shown in FIG. 1 , angle light sensor 120 is positioned at distance 182 from vector 173 representing eye 190 looking directly at, and light source 110 is positioned at distance 183 from vector 173 . The angle light sensor 120 measures the incident angle θ _y of the returning light 117 .

In the implementation of the present invention, a plurality of light sources and a plurality of angle light sensors can be used in an eye-tracking system to determine the gaze angle of the eyes of the wearer of the head-mounted device. Angular light sensors may be placed on the frame or "in-field" on the transparent or translucent near-eye optics of the headset. For "in-field" placement, the angle light sensor and light source may need to be unobtrusive to the user. This means that in some implementations, the sensor area can be smaller than 150x150 microns.

Other desirable attributes of angular light sensor based eye tracking for AR/VR are high angular resolution (between 1 arcmin to 60 arcmin) and/or relatively large angular detection range ( eg 1 degree to 90 degrees). The angle detection range of the angle light sensor can also be referred to as the field of view (FOV) of the angle light sensor in the present invention. Another desirable attribute of an angular light sensor may include off-axis angular peak sensitivity while blocking undesirable signal and stray light.

FIG. 2A shows a light sensor 250 with a sensing plane (y) and a surface normal vector 251 perpendicular to the sensing plane of the light sensor 250 . The light sensor 250 has a larger FOV 253 of approximately 160 degrees and is centered around the surface normal vector 251 . Sensor 250 has low angular resolution. Most currently commercially available sun tracking sensors are similar to sensor 250 .

FIG. 2B shows light sensor 260 having a smaller FOV and higher angular resolution than sensor 250 . The light sensor 260 has a sensing plane (y) and a surface normal vector 261 perpendicular to the sensing plane of the light sensor 260 . Light sensor 260 has a FOV 263 of approximately 10 degrees and is centered about surface normal vector 261 . These types of photosensor architectures are typically used for laser calibration or micro-electro-mechanical system (MEMS) mirror feedback, and often require multiple reflections using a grating or grating to achieve high angular accuracy.

FIG. 2C illustrates an angular light sensor 270 with a higher resolution and relatively larger FOV 273 in accordance with an implementation of the present invention. The angle light sensor 270 has a sensing plane (y) and a surface normal vector 271 perpendicular to the sensing plane of the angle light sensor 270 . The angle light sensor 270 has a FOV 273 of about 30 degrees and is inclined relative to the surface normal vector 271 . In detail, in FIG. 2C , the FOV 273 is centered around the center vector 272 , and the center vector 272 is inclined relative to the surface normal vector 271 . In some implementations, the angular light sensor 270 has an angular detection range of between 1 degree and 85 degrees relative to the photodetector surface of the sensor 270 .

3A and 3B illustrate an angle photosensor 300 having a first photodiode 311 , a second photodiode 312 and an inclined light barrier 333 according to an implementation of the present invention. The inclined light barrier 333 is disposed between the first photodiode 311 and the second photodiode 312 . The ratio of the first signal 316 generated by the first photodiode 311 to the second signal 317 generated by the second photodiode 312 indicates the incident angle of the returned light (eg, the returned light 117 ).

FIG. 3B shows that the inclined light barrier 333 is inclined relative to the surface normal vector 371 of the common sensing plane 373 of both the first photodiode 311 and the second photodiode 312 . In some examples, the angle a by which the slanted light barriers are offset relative to vector 371 is about 15 degrees. Other angles may be used. In one implementation, the length L of the inclined light barrier is 3 mm.

3C shows an example graph 380 illustrating the expected power of photodiodes separated by sloped light barriers, according to an implementation of the invention. Line 381 shows an example power output of a left photodiode (such as photodiode 311 ) at different angles of incidence θ, and line 382 shows a right photodiode (such as photodiode 312 ) at different angles of incidence Example power output within θ.

4A shows a portion of a head mounted device 400 including a frame 414 and near-eye optics 421A and 421B secured by the frame 414, in accordance with an implementation of the present invention. Although not specifically illustrated, the headset 400 may include arms coupled to the frame 414 that secure the headset to the user's head. FIG. 4A shows that the eyes 490 can view the external environment of the head-mounted device 400 through the transparent or translucent near-eye optical element 421A or 421B. In other words, scene light from the external environment can propagate through the near-eye optical element 421A and/or the near-eye optical element 421B. In some implementations, near-eye optical elements 421A and/or 421B may include all or a portion of a near-eye display system to provide augmented reality images to eye 490 . Near-eye optical elements 421A and 421B may be referred to as “lenses” of head mounted device 400 .

FIG. 4A shows near-eye optics 421B including three sensor light source pairs 430A, 430B, and 430C disposed "in-field." However, the pair of sensor light sources may be small enough to be unobtrusive and unobtrusive to the wearer of the head mounted device 400 . The sensor light source pair 430A includes a light source 433A and an angle light sensor 431A. The sensor light source pair 430B includes a light source 433B and an angle light sensor 431B, and the sensor light source pair 430C includes a light source 433C and an angle light sensor 431C. In addition, FIG. 4A shows a sensor light source pair 430D mounted to the frame 414 of the head mounted device 400 . The sensor light source pair 430D includes a light source 433D and an angle light sensor 431D. Although only one sensor light source pair is shown mounted to the frame 414, in some implementations, a plurality of sensor light source pairs may be mounted to the frame 414 (eg, around the frame). Light sources 433A, 433B, 433C, and 433D (collectively referred to as light sources 433 ) and angular light sensors 431A, 431B, 431C, and 431D (collectively referred to as angular light sensors 431 ) may have angular light sensing as described with respect to FIGS. 1-3B . The characteristics of the detector and light source. In a given sensor light source pair, the light source may be spaced less than 500 microns from the angular light sensor. In one implementation, the light source and angle light sensor are spaced about 200 microns apart.

FIG. 4B shows a section of near-eye optics 421B including sensor light source pair 430B, in accordance with an implementation of the invention. FIG. 4B shows that visible scene light 497 from the external environment can propagate through near-eye optics 421B to eye 490 . Thus, near-eye optics 421B may be used in an AR context.

The light source 433B can emit the illumination light 413 to the window area occupied by the eye 490 . The illuminating light 413 is reflected or scattered by the eye 490 as return light 417 and is measured/detected by the angle light sensor 431B. Processing logic 499 may receive one or more angle signals 419B from angle light sensor 431B. The angle signal can represent the incident angle of the returning light relative to the position of the angle light sensor. The angle signal can be generated from a photodetector (such as a photodiode) of the angle light sensor. Processing logic 499 may also receive one or more angle signals from additional angle light sensors (eg, angle signal 419A from sensor 431A and angle signal 419C from sensor 431C). Processing logic 499 may also drive light source 433 to selectively emit illumination light 413 . Processing logic 499 may reside in frame 414 or in an arm of headset 400 (not specifically shown).

5 shows a portion of a head mounted device 500 with a 3x3 array of sensor light source pairs for two-dimensional eye gaze detection, according to an implementation of the present invention. The head mounted device 500 includes a frame 414 and near-eye optical elements 521A and 521B fixed by the frame 414 . Although not specifically shown, the headset 500 may include arms coupled to the frame 414 that secure the headset 500 to the user's head. FIG. 5 shows that the eyes 490 can view the external environment of the head-mounted device 500 through the transparent or translucent near-eye optical element 521A or 521B. In other words, scene light from the external environment can propagate through the near-eye optical element 521A and/or the near-eye optical element 521B. In some implementations, near-eye optical elements 521A and/or 521B may include all or part of a near-eye display system to provide augmented reality images to eye 490 . Near-eye optical elements 521A and 521B may be referred to as “lenses” of head mounted device 500 .

5 shows near-eye optics 521B comprising nine sensor light source pairs 530A, 530B, 530C, 530D, 530E, 530F, 530G, 530H, and 530I (collectively referred to as sensor light source pairs 530) disposed "in-field." . However, sensor light source pair 530 may be small enough to be unobtrusive and unnoticeable to a wearer of head mounted device 500 .

The sensor light source pair 530A includes a light source 533A and an angle light sensor 531A. The sensor light source pair 530B includes a light source 533B and an angle light sensor 531B; the sensor light source pair 530C includes a light source 533C and an angle light sensor 531C; the sensor light source pair 530D includes a light source 533D and an angle light sensor 531D The sensor light source pair 530E includes a light source 533E and an angle light sensor 531E; the sensor light source pair 530F includes a light source 533F and an angle light sensor 531F; the sensor light source pair 530G includes a light source 533G and an angle light sensor 531G; the sensor light source pair 530H includes a light source 533H and an angle light sensor 531H; and the sensor light source pair 530I includes a light source 533I and an angle light sensor 531I. Light sources 533A, 533B, 533C, 533D, 533E, 533F, 533G, 533H and 533I (collectively referred to as light sources 533) and angle light sensors 531A, 531B, 531C, 531D, 531E, 531F, 531G, 531H and 531I (collectively referred to as The angle light sensor 531) may have the features of the angle light sensor and light source described with respect to FIGS. 1-3B.

The light source 533 can emit illuminating light to the window area occupied by the eyes of the wearer of the head mounted device 500, and the angle light sensor 531 can detect/measure the return light reflected by the cornea of the eye. Processing logic similar to processing logic 499 may receive angle signals output by a plurality of angle light sensors 531 . The angle signal can be generated from a photodetector (such as a photodiode) of the angle light sensor. The processing logic can also drive the light source 533 to selectively emit illumination light.

The signal on the angle light sensor depends on the light sensor placement and the user's biometric characteristics (such as eye fit or corneal curvature).

FIG. 6 shows an example angle photosensor 600 with more than two photodetectors and a photobarrier 630 according to an implementation of the invention. The angle light sensor 600 includes a light barrier 630 and photodetectors (such as photodiodes) 641 , 642 , 643 , 644 and 645 . The light source 601 illuminates the sensor 600 with light 603, and the photodetectors 641, 642, 643, 644, and 645 generate signals 651, 652, 653, 654, and 655, respectively. In one implementation, centroid calculations may be used to calculate interpolated positions based on data obtained at discrete steps. For the purposes of photosensitive arrays or strips of photodetectors in a column, centroid calculations can be used to determine where light is most concentrated on a surface. In this system, because the light barrier 630 casts a shadow on at least some of the photodetectors, the signal observed by some photodetectors is stronger than the signal observed by other photodetectors. As an example, assume that signals 651 and 652 are 10, signal 653 is 50, and signal 654 and signal 655 are 100. A weighting factor is assigned to each of the photodiodes based on their distance from the light barrier 630 . The first weighting factor assigned to the first signal 651 is 0, the second weighting factor assigned to the second signal 652 is 1, the third weighting factor assigned to the third signal 653 is 2, and the third weighting factor assigned to the fourth signal 654 is 0. The fourth weighting factor is four, and the fifth weighting factor assigned to the fifth signal 655 is four. Therefore, the following equation can be formed: Total signal = 10 + 10 + 50 + 100 + 100 Weighted signal = 0*10 + 1*10 + 2*50 + 3*100 + 4*100 Centroid = Weighted Signal/Total Signal

In the implementation of the present invention, the eye-tracking system includes an angle light sensor and a light source placed on the frame or in the field of the AR or VR device. Each light source may have an emission cone directed to the cornea of the eye. The angle light sensor may have an output connection (such as an electrical pin) for calculating the angle of incidence of the incident light. In some implementations, the FOV of the angular light sensor is tilted relative to a vector normal to the surface of the angular light sensor. The angular photosensor may have an angular resolution greater than 1 arc minute but less than 60 arc minutes. In some implementations, the angle light sensor can have an area between 5 microns by 5 microns and 1 mm by 1 mm. The thickness of the sensor can be between 5 microns and 500 microns.

In some implementations, the FOV of the angular light sensor, the light source, or both can be dynamically tilted (e.g., by MEMS devices) and/or increased/decreased (e.g., by adding a lens to the light source to increase divergence) to illuminate The larger or smaller segment of the window. The tilt mechanism can be configured to dynamically tilt the FOV of the angle detection range of the angle light sensor. In some implementations, the tilt mechanism can be a MEMS device.

In some implementations of the invention, the angle light sensor includes a Talbot sensor. Potential advantages of Talbot sensors are non-camera-based eye tracking, miniature size, larger detectable eye tracking systems, relatively high angular sensitivity, and shorter processing times.

FIG. 7A shows the location of the Talbot image plane behind grating 750 to image light 793 . FIG. 7A shows a Talbot image plane, various Talbot subimage positions, and an inverse Talbot image plane.

Figure 7B includes equations 781 and 782 which provide the Talbot distance z for the Talbot image with respect to the grating spacing d and the wavelength λ of the incident light on the Talbot sensor. Equation 783 of FIG. 7B provides the Talbot distance z with respect to the grating spacing d , the angle of incidence Θ, and the wavelength λ of the incident light for off-axis Talbot imaging.

8 shows graphs showing the relationship between the angle of incidence (AOI) of light on an angular photosensor and the transmittance of a Talbot sensor over an example period of 0.88 microns, in accordance with an implementation of the invention. Graph 800 .

9 illustrates an example eye-tracking system 900 that includes four sensor light source pairs 930A, 930B, 930C, and 930D (collectively referred to as sensor light source pair 930). Sensor light source pair 930A includes a near-infrared light source 933A (eg, LED or VCSEL) and an angular light sensor 931A including at least one Talbot detector. Sensor light source pairs 930B, 930C, and 930D also include a near-infrared light source and an angular light sensor 931 that includes at least one Talbot detector. The near-infrared illuminating light illuminates the cornea, and at least a portion of the near-infrared illuminating light is reflected back to the sensor 931 . The field of view (FOV) of the light source 933 may be between 5 degrees and 20 degrees. In some implementations, the FOV of light source 933 may be approximately 10 degrees.

In the illustration, the distance along the z-axis from the coordinate 0,0 (in the middle of the sensor light source pair 930) to the corneal apex is between 10 and 25 mm. This distance can be called "eye distance". In some implementations, the interocular distance is about 16 mm. In the example shown, the distance between sensor light source pair 930A(-1,-1) and sensor light source pair 930C(1,1) is 2 mm, as indicated by their coordinates. Similarly, the distance between sensor light source pair 930B (1,1) and sensor light source pair 930D (−1,1) is 2 mm. Of course, other suitable spacing dimensions are possible.

10 shows an example sensor structure 1000 including a first photodetector 1011 , a second photodetector 1012 and a third photodetector 1013 according to an implementation of the invention. The sensor structure 1000 can optionally include a photodetector 1010 that can be used to normalize the incident power without the need for a grating structure disposed over the photodiode 1070 . The sensor structure 1000 may be included in the sensor 931 of FIG. 9 .

Figure 10 shows an example Talbot sensor oriented in the XY plane responsible for measuring the angle of incidence (AOI) in the XY plane. Two layers of the grating (layers 1050 and 1060) are in the structure. The second layer of grating 1060 has a different eccentricity value. Here, three centrifugal values 0, d/4, 3d/8 are selected to be configured for photodetectors 1011, 1012, and 1013, respectively. In an example implementation, the pitch d is 2.55 microns. In the implementation shown, the width of each grating is d/2 such that the pitch has a 50% duty cycle. In other words, in the example illustration of FIG. 10, the grating elements are spaced apart from each other by the same distance as the width of the grating elements. In one implementation, the length L of the grating element is 10 microns. In one implementation, the first grating layer 1050 is spaced apart from the second grating layer 160 by a dimension z, where z is about 22 microns.

In FIG. 10 , the first light detector 1011 includes a first high grating 1051 , a first low grating 1061 and a first photodiode 1071 . The first photodiode 1071 receives incident light that encounters the first upper grating 1051 and then the first lower grating 1061 . The first lower raster 1061 is decentered with a shift factor of 0 relative to the first upper raster 1051 .

The second photodetector 1012 includes a second upper grating 1052 , a second lower grating 1062 and a second photodiode 1072 . The second photodiode 1072 receives incident light that encounters the second upper grating 1052 and then the second lower grating 1062 . The second lower raster 1062 is centrifugal by a shift factor of 3d/8 with respect to the second upper raster 1052 .

The third light detector 1013 includes a third upper grating 1053 , a third lower grating 1063 and a third photodiode 1073 . The third photodiode 1073 receives incident light that encounters the third upper grating 1053 and then the third lower grating 1063 . The third lower raster 1063 is centrifugal with respect to the third upper raster 1053 by a shift factor of 3d/8.

Photodiodes 1071, 1072, and 1073 can generate angle signals 1081, 1082, and 1083, respectively, in response to receiving incident light. In some implementations, the fourth, fifth, and sixth photodetectors are rotated 90 degrees relative to the first, second, and third photodetectors to form a sensor with six photodetectors.

Figure 11 shows a sensor structure 1100 according to an implementation of the invention. According to an implementation of the present invention, sensor structure 1100 is the same as or similar to sensor structure 1000 except that three sensors are oriented in the YZ plane and are responsible for measuring the AOI in the YZ plane. The sensor structure 1100 also optionally includes a photodetector 1110 (not shown), which can be used to normalize the incident power without requiring a grating structure (similar to a photodetector) placed over its photodiode. detector 1010). Photodiodes 1174, 1175, and 1176 may generate angle signals 1184, 1185, and 1186, respectively, in response to receiving incident light.

Figure 11 shows the fourth lower grating 1164 disposed between the fourth high grating 1154 and the fourth photodiode 1174, the fifth lower grating disposed between the fifth high grating 1155 and the fifth photodiode 1175 1165 and the sixth low grating 1166 disposed between the sixth high grating 1156 and the sixth photodiode 1176 .

FIG. 12 shows a combined sensor structure 1200 that combines sensor structure 1000 (sensors in the XY plane) with sensor structure 1100 (sensors in the YZ plane), according to an implementation. The sensors in the left column are oriented in the XY plane to measure AOI in the XY plane, and the sensors in the right column are oriented in the YZ plane to measure AOI in the YZ plane. The sensors in the first column have 0 eccentric values, the second column has d/4 eccentric values and the last column has 3d/8 eccentric values. In the illustrated implementation, the first grating layer 1250 may include gratings 1051 , 1052 , 1053 , 1154 , 1155 , and 1156 , while the second grating layer 1260 may include gratings 1061 , 1062 , 1063 , 1164 , 1165 , and 1166 .

The material used in the grating substrate can be copper, chrome or other suitable materials. The space between the two grating layers can be filled with _SiO2 or other suitable optically transparent substrates.

Embodiments of the invention may include or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been modified in some way before being presented to the user, which may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid Reality or some combination and/or derivative thereof. Artificial reality content may include all generated content or generated content combined with captured (eg, real world) content. Artificial reality content may include video, audio, haptic feedback, or some combination thereof, and any of these may be presented in a single channel or in multiple channels (such as stereoscopic video that creates a three-dimensional effect on the viewer). Additionally, in some embodiments, AR can also be used in conjunction with, for example, applications that create content in AR and/or that are otherwise used in AR (such as performing activities in AR), Products, accessories, services or some combination thereof. Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs) connected to host computer systems, standalone HMDs, mobile devices or computing systems or capable of delivering artificial reality content to Any other hardware platform for one or more viewers.

The term "processing logic" (eg, 499) in the present invention may include one or more processors, microprocessors, multi-core processors, application-specific integrated circuits (ASICs) and/or field-programmable A Field Programmable Gate Array (FPGA) is used to perform the operations disclosed herein. In some embodiments, memory (not shown) is integrated into the processing logic to store instructions for performing operations and/or to store data. Processing logic may also include analog or digital circuitry to perform operations in accordance with embodiments of the invention.

"Memory" as described herein may include one or more volatile or non-volatile memory structures. "Memory" may be removable or non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Example memory technologies may include RAM, ROM, EEPROM, flash memory, CD-ROM, digital versatile disk (DVD), high-definition multimedia/data storage disk or other optical storage, cartridge Tape, magnetic tape, disk storage, or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device.

A network may include any network or network system, such as but not limited to the following: peer-to-peer networks; Local Area Networks (LANs); Wide Area Networks (WANs); public networks such as The Internet; private networks; cellular networks; wireless networks; wired networks; combined wireless and wired networks; and satellite networks.

The communication channel may include or utilize IEEE 802.11 protocol, Bluetooth, Serial Peripheral Interface (SPI), Inter-Integrated Circuit (I ² C), Universal Serial Port (Universal Serial Port; USB) , Controller Area Network (Controller Area Network; CAN), cellular data protocols (such as 3G, 4G, LTE, 5G), optical communication network, Internet Service Provider (Internet Service Provider; ISP), peer-to-peer network, Local Area Network (LAN), Wide Area Network (WAN), public network (such as the "Internet"), private network, satellite network, or otherwise One or more wired or wireless communications are routed.

Computing devices may include desktops, laptops, tablets, phablets, smartphones, feature phones, server computers, or other devices. The server computer can be located remotely in the data center or stored locally.

The procedures explained above are described in terms of computer software and hardware. The described techniques may constitute machine-executable instructions embodied in a tangible or non-transitory machine (eg, computer)-readable storage medium, which, when executed by a machine, will cause the machine to perform the operations described. Additionally, the programs may be embodied in hardware, such as an application specific integrated circuit ("ASIC") or other hardware.

Tangible non-transitory machine-readable storage media includes any mechanism that provides (that is, stores) information for display by a machine (such as a computer, network device, personal digital assistant, manufacturing tool, A collection of any device, etc.) form of access. For example, machine-readable storage media include recordable/non-recordable media (such as read only memory (ROM; ROM), random access memory (read access memory; RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

The above descriptions of illustrated examples of the invention, including what is described in the Abstract, are not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific specific instances and examples for the invention are described herein for illustrative purposes, various modifications are possible within the scope of the invention, as those having ordinary skill in the relevant art will recognize.

Such modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in this specification. Rather, the scope of the invention should be determined solely by the following claims, which are to be construed in accordance with established principles of claim interpretation.

110: light source 113: Irradiate light 117: return light 120: Angle light sensor 173: vector 181: Eye distance 182: Distance 183: Distance 190: eyes 192: point 194: Cornea 195: window area 250: light sensor 251: surface normal vector 253:FOV 260: light sensor 261: surface normal vector 263: FOV 270: Angle light sensor 271: surface normal vector 272: Center vector 273:FOV 300: Angle light sensor 311: the first photodiode 312: second photodiode 316: The first signal 317:Second signal 333: Inclined Light Barrier 371: Surface normal vector 373: Sensing plane 380:Example Diagram 381: line 382: line 400: Head Mounted Device 413: Irradiate light 414: frame 417: return light 419A: Angle signal 419B: Angle signal 419C: Angle signal 421A: Near Eye Optical Components 421B: Near Eye Optical Components 430A: sensor light source pair 430B: sensor light source pair 430C: sensor light source pair 430D: sensor light source pair 431A: Angle light sensor 431B: Angle light sensor 431C: Angle light sensor 431D: Angle light sensor 433A: light source 433B: light source 433C: light source 433D: Light source 490: eyes 497:Visible scene light 499: Processing logic 500: Head Mounted Device 521A: near-eye optics 521B: Near-eye optics 530A: sensor light source pair 530B: sensor light source pair 530C: sensor light source pair 530D: sensor light source pair 530E: sensor light source pair 530F: sensor light source pair 530G: sensor light source pair 530H: sensor light source pair 530I: sensor light source pair 531A: Angle light sensor 531B: Angle light sensor 531C: Angle light sensor 531D: Angle light sensor 531E: Angle light sensor 531F: Angle light sensor 531G: Angle light sensor 531H: Angle light sensor 531I: Angle light sensor 533A: light source 533B: light source 533C: light source 533D: light source 533E: light source 533F: light source 533G: light source 533H: light source 533I: light source 600: Angle light sensor 601: light source 603: light 630: light barrier 641: photoelectric detector 642:Photoelectric detector 643: Photoelectric detector 644: photoelectric detector 645: photoelectric detector 651: signal 652:Signal 653:Signal 654:Signal 655:Signal 750: grating 781:equation 782:equation 783:equation 793: light 800: Curve 900:Eye Tracking System 930A: sensor light source pair 930B: sensor light source pair 930C: sensor light source pair 930D: sensor light source pair 931A: Angle light sensor 933A: near infrared light source 983:Corneal Centroid 1000: sensor structure 1010: Light detector 1011: The first light detector 1012: the second light detector 1013: The third light detector 1050: layer 1051: The first high grating 1052: second high grating 1053: The third high grating 1060: layer 1061: The first low raster 1062: second low raster 1063: The third low raster 1070: photodiode 1071: the first photodiode 1072: second photodiode 1073: the third photodiode 1081: Angle signal 1082: Angle signal 1083: Angle signal 1100: Sensor structure 1154: The fourth high grating 1155: fifth high grating 1156: sixth high grating 1164: The fourth low raster 1165: fifth low raster 1166: sixth low raster 1174: The fourth photodiode 1175: fifth photodiode 1176: The sixth photodiode 1184: Angle signal 1185: Angle signal 1186: Angle signal 1200: Combined sensor structure 1250: the first grating layer 1260: second grating layer d: grating spacing θ: angle of incidence λ:wavelength α: angle z: distance

Non-limiting and non-exhaustive specific examples of the present invention are described with reference to the following drawings, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.

[ Fig. 1 ] shows a light source sensor pair for determining an eye gaze angle according to an aspect of the present invention.

[ FIG. 2A ] shows a light sensor with a sensing plane and a surface normal vector perpendicular to the sensing plane of the light sensor.

[ FIG. 2B ] shows a light sensor with a smaller field of view (FOV) and higher angular resolution than the sensor in FIG. 2A .

[ FIG. 2C ] shows an angle light sensor with higher resolution and relatively larger FOV according to an aspect of the present invention.

[ FIG. 3A ] and [ FIG. 3B ] illustrate an angle photosensor having a first photodiode, a second photodiode, and an inclined light barrier according to aspects of the present invention.

[ FIG. 3C ] An example graph illustrating the expected power of photodiodes separated by sloped light barriers according to aspects of the present invention is shown.

[ FIG. 4A ] shows a part of a head-mounted device according to an aspect of the present invention, which includes a frame and a near-eye optical element fixed by the frame.

[ FIG. 4B ] shows a section of a near-eye optical element including a sensor light source pair according to an aspect of the present invention.

[ FIG. 5 ] Illustrates a portion of a head mounted device with a 3×3 array of sensor light source pairs for two-dimensional eye gaze detection according to an aspect of the present invention.

[ FIG. 6 ] shows an example angle photosensor with more than two photodetectors and light barriers according to aspects of the invention.

[FIG. 7A] shows the location of the Talbot image plane behind the grating.

[ FIG. 7B ] An equation including the Talbot distance z for a Talbot image provided according to aspects of the invention, the distance z relative to the grating spacing d and the incidence on the Talbot sensor The wavelength λ of light.

[ FIG. 8 ] shows the difference between the angle of incidence ( AOI ) of light on an angle light sensor and the Talbot sensor during an example period of 0.88 microns according to an aspect of the present invention. Graph of the relationship between transmittance.

[ FIG. 9 ] shows an example eye-tracking system according to aspects of the present invention, which includes four sensor light source pairs configured to illuminate the corneal-shaped centroid with near-infrared illumination light.

[ FIG. 10 ] shows an example sensor structure including a first photodetector, a second photodetector, and a third photodetector according to aspects of the present invention.

[ Fig. 11 ] shows a sensor structure with three sensors oriented in the YZ plane according to an aspect of the present invention.

[ Fig. 12 ] Shows a combined sensor structure having a sensor for the XY plane and a sensor for the YZ plane according to an aspect of the present invention.

110: light source

113: Irradiate light

117: return light

120: Angle light sensor

173: vector

181: Eye distance

182: Distance

183: Distance

190: eyes

192: point

194: Cornea

195: window area

Claims (20)

An eye tracking system comprising: a plurality of infrared light sources configured to emit infrared illuminating light into the window area; and A plurality of angle photosensors configured to receive return infrared light, which is the infrared irradiation light reflected from the window area, wherein the plurality of angle photosensors are configured to output to indicate that the return infrared light is relatively The angle signal of the angle of incidence at the position of these angle photosensors. The eye tracking system of claim 1, wherein the angle light sensors have an angle detection range between 1 degree and 85 degrees relative to the photodetector surface of the angle light sensors. The eye tracking system of claim 1, wherein the angle light sensors have a sensor area smaller than 150 microns x 150 microns. The eye tracking system as claimed in claim 1, wherein the angle light sensors include: a first photodiode configured to receive the returning infrared light; a second photodiode configured to receive the returning infrared light; and an inclined light barrier, which is disposed between the first photodiode and the second photodiode, and the inclined light barrier is relative to the common surface of both the first photodiode and the second photodiode The surface normal of the sensing plane is inclined, wherein the ratio of the first signal generated by the first photodiode to the second signal generated by the second photodiode is indicative of the angle of incidence of the returning infrared light. The eye tracking system as claimed in claim 1, wherein the angle light sensors include: light barrier; a first photodiode configured to receive the returning infrared light; a second photodiode configured to receive the return infrared light, wherein the first photodiode is disposed between the light barrier and the second photodiode; and processing logic configured to receive a first signal generated by the first photodiode and a second signal generated by the second photodiode, wherein the processing logic generates based on the first signal and the second signal A centroid value, wherein the first signal is assigned a first weighting factor that is less than a second weighting factor assigned to the second signal in generating the centroid value. The eye tracking system according to claim 1, wherein the field of view (FOV) of the angle detection range of the angle light sensors is inclined relative to the surface normal of the photodetectors of the angle light sensors. As the eye tracking system of claim 1, it further includes: A tilt mechanism configured to dynamically tilt the field of view (FOV) of the angular detection range of the angular photosensors. The eye-tracking system of claim 7, wherein the tilt mechanism comprises a micro-electro-mechanical system (MEMS) device. The eye tracking system according to claim 1, wherein each infrared light source is paired with an angle light sensor and spaced apart by less than 500 microns. A head-mounted device comprising: a frame for securing the headset to the user's head; and Eye tracking system, which includes: a plurality of infrared light sources configured to emit infrared illuminating light into the window area; and A plurality of angle photosensors configured to receive return infrared light, which is the infrared irradiation light reflected from the window area, wherein the plurality of angle photosensors are configured to output to indicate that the return infrared light is relatively The angle signal of the angle of incidence at the position of these angle photosensors. The head-mounted device according to claim 10, wherein the plurality of infrared light sources and the plurality of angle light sensors are installed on the frame of the head-mounted device. As the head-mounted device of claim 10, it further comprises: A lens is held by the frame, wherein the lens transmits visible scene light from the external environment to the window area, and wherein the plurality of infrared light sources and the plurality of angle light sensors are arranged on the lens. An angle light sensor comprising: a first photodetector comprising a first photodiode, a first high grating and a first low grating, wherein the first low grating is disposed between the first high grating and the first photodiode; and a second photodetector comprising a second photodiode, a second high grating and a second low grating, wherein the second low grating is disposed between the second high grating and the second photodiode, wherein the second low raster is centrifugal relative to the second high raster, and wherein the first low raster is centered relative to the first high raster. The angle light sensor of claim 13, wherein the second low grating is centrifugal with a shift factor of d /4 relative to the second high grating, wherein d is the second high grating, the second low grating, the second high grating The distance between the first low grating and the first high grating. The angle light sensor according to claim 14, which further includes: a third light detector, which includes a third photodiode, a third high grating and a third low grating, wherein the third low grating is placed on the Between the third high grating and the third photodiode, wherein the third low grating is centrifugal with a shift factor of 3 d /8 relative to the third high grating, where d is the distance between the third high grating and the first high grating Three low raster spacing. The angle light sensor according to claim 14, wherein the distance has a duty cycle of 50%. As the angle light sensor of claim 13, it further comprises: a fourth photodetector configured as the first photodetector; a fifth photodetector configured as the second photodetector; and a sixth photodetector configured as the third photodetector, wherein the fourth photodetector, the fifth photodetector, and the sixth photodetector are relative to the first photodetector The detector, the second photodetector and the third photodetector are rotated 90 degrees. The angle light sensor according to claim 13, wherein the grating material is chrome or copper for the second high grating, the second low grating, the first low grating, and the first high grating. The angle light sensor of claim 13, wherein the angle light sensor is configured to measure an incident angle of light having a wavelength λ, and wherein the first lower grating is separated from the first upper grating by a distance z , wherein z is 2 d ² /λ, and furthermore wherein the second lower grating is also spaced apart from the second upper grating by the distance z . The angle light sensor of claim 13, wherein an optically transparent substrate is disposed between the first high grating and the first low grating, and wherein the optically transparent substrate is also disposed between the second high grating and the second low grating between rasters.

Images