EP4634718A1

EP4634718A1 - Adaptive illuminator for head-mounted displays (hmds)

Info

Publication number: EP4634718A1
Application number: EP23841403.1A
Authority: EP
Inventors: Mehdi Aas; Rob Jacques Paul ENGELEN
Original assignee: Lumileds LLC
Current assignee: Lumileds LLC
Priority date: 2022-12-15
Filing date: 2023-12-05
Publication date: 2025-10-22
Also published as: WO2024129446A1

Abstract

Various embodiments include apparatuses and methods for providing adaptive illumination to portions of an eye-box in which an eye-pupil is located. In one example, an apparatus is disclosed that includes a light-emitting diode (LED) array having a plurality of LEDs with each of the plurality of LEDs being individually addressable in subsets of the LED array, a collimating element located between the LED array and the eye-box, a display device to generate images, and an eyepiece to receive light from at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within a volume of the eye-box. Other apparatuses and methods are described.

Description

ADAPTIVE ILLUMINATOR FOR HEAD-MOUNTED DISPLAYS (HMDS)

CLAIM OF PRIORITY

[0000] This application claims the priority benefit to U.S. Provisional Patent Application Serial No. 63/432,966, filed on 15 December 2022, and entitled “ADAPTIVE ILLUMINATOR FOR HEAD MOUNTED DISPLAYS (HMDS),” which is incorporated by reference herein in its entirety.

TECHNICAL FIELD

[0001] The subject matter disclosed herein relates to an adaptive illuminator for head-mounted displays (HMDs). More specifically, the disclosed subject-matter relates to systems, devices, and apparatuses to provide an adaptive illuminator to an HMD based on a location of an eye-pupil as determined by an eye-tracking unit.

BACKGROUND

[0002] Head-Mounted Displays (HMDs) initially emerged for defense applications. Now, HMD use cases are extended to be a wearable display for daily life. For example, a variety of HMDs from different companies with a range of different specifications are already developed as a commercial product in the form of, for example, wearable spectacles. A category of HMDs includes small reflective devices including Liquid Crystal on Silicon (LCoS) devices, which are substantially opaque to visible light, and liquid crystal display (LCD) devices, which are substantially transmissive to visible light. Both LCoS and LCD devices in the context of HMDs are frequently referred to as mini-display. Mini-displays in HMDs modify the intensity of a reflected or transmitted light intensity and include an optical system to magnify and project images from the minidisplay onto the eye location. Since the pupil size of a human eye is variable and can change between 2 mm up to 8 mm depending on the intensity of the ambient light, it is desirable to design an eye-box in an HMD to be large enough to cover rotational movements and alignment ranges of the eye of an HMD user relative to the projection system of an HMD.

[0003] The information described in this section is provided to offer the skilled artisan a context for the following disclosed subject-matter and should not be considered as admitted prior art.

SUMMARY

[0004] In various embodiments, the disclosed subject-matter describes an apparatus to provide adaptive illumination to portions of an eye-box in which an eye-pupil is located. The apparatus includes a light-emitting diode (LED) array having a plurality of LEDs. Each of the plurality of LEDs is individually addressable in subsets of the LED array with at least one of the plurality of LEDs within a subset of the LED array being configured to be lighted based on a location of the eye-pupil with the eye-box. The apparatus further includes a collimating element located between the LED array and the eye-box. A display device is located between the collimating lens and the eyebox; the display device is configured to generate images. An eyepiece element is located between the display device and the eye-box to receive light from the at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within an area of the eye-box.

[0005] In various embodiments, the disclosed subject-matter describes a head-mounted display (HMD) with an adaptive illumination apparatus. The HMD includes a light-emitting diode (LED) array having a plurality of LEDs. Each of the plurality of LEDs is individually addressable in subsets of the LED array, with at least one of the plurality of LEDs within a subset of the LED array being configured to be lighted based on a location of an eye-pupil with a volume of an eye-box within the HMD. A collimating element is located between the LED array and the eye-box. A display device is located between the collimating lens and the eye-box, with the display device being configured to generate images. An eyepiece element is located between the display device and the eye-box to receive light from the at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within a volume of the eye-box. At least one beam splitter is located between the LED array and the eyebox. The at least one beam splitter is configured to allow a wearer of the HMD to view a substantial entirety of a scene that is not blocked by the adaptive illumination apparatus.

[0006] In various embodiments, the disclosed subject-matter describes a method for providing adaptive illumination to portions of an eye-box in which an eye-pupil is located. The method includes determining a position of the eye-pupil; sending a signal to a lightemitting diode (LED) array having a plurality of LEDs, where each of the plurality of LEDs is individually addressable in subsets of the LED array; and lighting at least one of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box.

BRIEF DESCRIPTION OF THE FIGURES

[0007] FIG. 1A shows an example of a portion of a head-mounted display (HMD) with a backlighted, transmissive mini-liquid crystal display (mini-LCD) in an occluded architecture, in accordance with various embodiments of the disclosed subject matter; [0008] FIG. IB shows a portion of an HMD with a backlighted, transmissive mini-LCD in a folded, see-through architecture, in accordance with various embodiments of the disclosed subject matter;

[0009] FIG. 2A shows an example of an occluded optical- architecture for a near-to-eye HMD projection system with on-axis light-emitting diodes (LEDs) switched on for an eye-pupil alignment, in accordance with various embodiments of the disclosed subject matter;

[0010] FIG. 2B shows an example of an occluded optical- architecture for a near-to-eye HMD projection system with off-axis LEDs switched on for an eye-pupil alignment, in accordance with various embodiments of the disclosed subject matter;

[0011] FIG. 3A shows an example of an occluded optical- architecture for a near-to-eye HMD projection system with on-axis LEDs switched on for an eye-pupil alignment, in accordance with various embodiments of the disclosed subject matter;

[0012] FIG. 3B shows an example of an occluded optical- architecture for a near-to-eye HMD projection system with off-axis LEDs switched on for an eye-pupil alignment, in accordance with various embodiments of the disclosed subject matter;

[0013] FIG. 4 is an exemplary embodiment of a layout of an adaptive illuminator system for an HMD with an eye-tracking module, in accordance with various embodiments of the disclosed subject matter;

[0014] FIG. 5 shows an exemplary embodiment of an optical layout for a near-to-eye HMD projection system with an arbitrary LED-to- collimator distance and an eyepiece-to-eye-pupil distance, in accordance with various embodiments of the disclosed subject matter; [0015] FIG. 6 shows an exemplary embodiment of an eye-box size and location for an occluded on-axis near-to-eye HMD, in accordance with various embodiments of the disclosed subject matter;

[0016] FIG. 7 shows an exemplary embodiment of a see-through configuration of an HMD with an adaptive eye-box illuminator, in accordance with various embodiments of the disclosed subject matter;

[0017] FIG. 8 shows an exemplary embodiment of a compact-design alternative of a see-through HMD with an adaptive eye-box illuminator, in accordance with various embodiments of the disclosed subject matter; and

[0018] FIG. 9 shows an exemplary embodiment of a near-to-eye HMD folded-design with thin polarizing beam-splitters (PBSs) in a see-through HMD-system design.

DETAILED DESCRIPTION

[0019] The disclosed subject-matter will now be described in detail with reference to a few general and specific embodiments as illustrated in various ones of the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the disclosed subject matter. It will be apparent, however, to one skilled in the art, that the disclosed subject-matter may be practiced without some or all of these specific details. In other instances, well-known process steps or structures have not been described in detail so as not to obscure the disclosed subject matter.

[0020] Examples of different adaptive illumination systems and devices for head-mounted displays (HMDs) will be described more fully hereinafter with reference to the accompanying drawings. These examples are not mutually exclusive, and features found in one example may be combined with features found in one or more other examples to achieve additional implementations. Accordingly, it will be understood that the examples shown in the accompanying text and drawings are provided for illustrative purposes only and they are not intended to limit the disclosure in any way. Like numbers may refer generally to like elements throughout.

[0021] Further, it will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms may be used merely to distinguish one element from another. For example, a first element may be termed a second element and a second element may be termed a first element without departing from the scope of the disclosed subject matter. As used herein, the term “and/or” may include any and all combinations of one or more of the associated listed items. [0022] It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it may be directly connected or coupled to the other element and/or connected or coupled to the other element via one or more intervening elements. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements present between the element and the other element. It will be understood that these terms are intended to encompass different orientations of the element in addition to any orientation depicted in the figures.

[0023] Relative terms such as “below,” “above,” “upper,” “lower,” “horizontal,” or “vertical” may be used herein to describe a relationship of one element, zone, or region relative to another element, zone, or region as illustrated in the figures. A person of ordinary skill in the art will understand that these terms are intended to encompass different orientations of the device in addition to an orientation depicted in the figures. Further, whether the LEDs, LED arrays, optical elements, electrical components and/or electronic components are housed on one, two, or more electronics boards or optical boards, or in one or multiple physical locations may also depend on design constraints and/or a specific application.

[0024] Various embodiments of the disclosed subject-matter include an adaptive illumination system for HMDs. An eye-pupil tracking system is used to locate the position of the eye-pupil and the illumination system is adapted to generate only and locate the beams on the location of eye-pupil. As an example, consider a statically illuminated eye-box size of about 10 mm by about 8 mm. The eye-box size can be replaced with a dynamically illuminated eye-box size of only about 2 mm by about 2 mm. This smaller, dynamically illuminated eye-box size moves inside of the 10 mm by 8 mm eye-box based on signals from an eye-tracking sensor. As a result of the smaller portion of eye-box that is used, a power efficiency of the HMD system increase by 20 times due to the reduced area of illumination that is used. Moreover, although the disclosed subject-matter is disclosed in terms of an HMD, many of the concepts and designs disclosed herein may also be adapted for uses with other types of display systems, such as displays projected onto one or more portions of an automobile windshield or onto one or more portions of a windshield of the cockpit of a domestic or military plane or other vehicle.

[0025] FIG. 1A shows an example of a portion of a head-mounted display (HMD) 100 (an occluded projection system) with a backlighted, transmissive mini -liquid crystal display (mini-LCD) 103 in an occluded architecture, in accordance with various embodiments of the disclosed subject matter. The occluded architecture blocking only a portion of a scene viewable to a person wearing the HMD.

[0026] Optical rays are shown as emanating from a single point on the mini-LCD 103 in order to provide a better understanding of the portion of the HMD 100. FIG. 1 is also shown to include a backlight unit 101, an eyepiece lens 105, an eye-box 107, a human eye 109A, and an eye-pupil 109B. The mini-LCD 103 is located at a first distance 111, f_e, from the eyepiece lens 105. The first distance 111 can be considered as the focal length of the eyepiece lens 105. The eyepiece lens 105 is also located at a second distance 113, de, from the eye-box 107. In the occluded projection system of FIG. 1A, a user of the system is not able to see a scene located directly behind at least portions of the mini-LCD 103, rendering the human eye 109A to be in an occluded status. [0027] The backlight unit 101 can be any type of backlight, known in the relevant art, to illuminate a backside (that is, a side farthest from the eye-box 107) of the mini-LCD 103. As shown in FIGS. 1A and IB, the backlight unit 101 provides illumination to the backside of the mini-LCD 103 so as to substantially fill the eye-box 107 with light. Additionally, although shown as a single bi-convex lens, the eyepiece lens 105 can be any type of lens, or a combination of lenses, as would be recognized by a person of ordinary skill in the art upon reading and understanding the disclosed subject matter. Further, in general the eye-box 107 refers to an area, rather than an actual physical object, where the eye-pupil 109B receives an acceptable view of the image with respect to a set of criteria and thresholds, as defined in more detail below.

[0028] FIG. IB shows an example of a portion of an HMD 120 (a see-through projection system) with a backlighted, transmissive version of a mini-LCD 103 in a folded, see-through architecture, in accordance with various embodiments of the disclosed subject matter. In the portion of the HMD 120, a beam splitter 121 is added, thus allowing a user of the see-through projection system to see both the mini-LCD 103 as well as a scene through the beam splitter 121 (e.g., a sense to the left of the beam splitter 121).

[0029] As noted above, a size of the eye-pupil 109B is variable and can change from, for example, between about 2 mm up to about 8 mm depending on an intensity level of ambient light. A size of the eye-box 107 in an HMD is generally designed large enough to cover rotational movements and an alignment range of the human eye 109A of a user relative to the projection system of the portion of the HMD 100. As discussed in more detail below, when a location of the eye-pupil 109B of the user changes inside the eye-box 107, the user will still be able to see the mini-LCD 103. [0030] Overall, one element of the various embodiments of the disclosed subject-matter described herein is the adaptive feature of the eye-box 107. The size of the eye-box 107 is a function of an optical design of the portion of the HDM 100 (the projection system) and an angular luminance of the mini-LCD 103. In order to generate a suitable eye-box size to cover all or substantially all of the movements of the eye-pupil 109B, a specific value or values of the angular width of luminance of the mini-LCD 103 is considered.

[0031] For example, with reference again to FIG. 1A, the angular width of the rays coming from each pixel of the display is approximately:

Aa = 2 tan’¹ ( (1) where H_d is a height of the mini-LCD 103, and f_e is the focal length of the eyepiece, shown in FIG. 1A as the first distance 111. For an eyepiece lens 105 having a focal length of f_e = 24 mm, and the height of the mini-LCD 103 is H_d = 7 mm, an angular width of luminance from each pixel on the display is An is greater than or equal to 17°. Furthermore, the luminance on the display is desired to substantially be uniform.

[0032] Generally, to meet the conditions described above under the prior art, generating such a beam profile requires a combination of very small collimators with micro-LED arrays. Alternatively, a backlight with a very wide beam profile similar to a Lambertian emission is used. The earlier scenario using the micro-LEDs is potentially more power efficient than the latter one, using the Lambertian-like backlight, since it can reduce the light dissipation with one or more collimation optics. However, both illumination systems of the prior art have to generate a wide range of angles. For a wearable HMD that is desired to be lightweight, the power efficiency is a parameter that can be used to elongate the play time of the display.

[0033] FIG. 2A shows an example of an occluded optical- architecture for a portion of a near-to-eye HMD projection system 200 with light-emitting diodes (LEDs) 201A, 20 IB, 201C (although the only on-axis LED 20 IB is illuminated), switched on (illuminated) for alignment with an eye-pupil 209B, in accordance with various embodiments of the disclosed subject matter. The occluded optical- architecture blocking only a portion of a scene otherwise viewable to a person wearing the HMD.

[0034] The LEDs 201A, 20 IB, 201C may be considered as being part of an LED array. As with the LED arrays disclosed in each of the embodiments herein, each of the LEDs is addressable individually to be turned on or off. Also, LEDs in the LED arrays disclosed herein may be grouped into subgroups, with each subgroup containing at least one LED but fewer than all of the LEDs within the LED array. Therefore, each of the subgroups comprise a subset of all LEDs in the LED array. Further, the subgroups may comprise LEDs that are overlapping or not over-lapping. For example, a first subgroup of an overlapping subgroup of LEDs may comprise a first LED and a second LED. A second subgroup of an overlapping subgroup of LEDs may comprise the second LED and a third LED. In an example of a nonoverlapping subgroup of LEDs, a first subgroup of a non-overlapping subgroup of LEDs may comprise a first LED and a second LED. A second subgroup of a non-overlapping subgroup of LEDs may comprise a third LED and a fourth LED (but not the second LED). Upon reading and understanding the disclosed subject matter, a person of ordinary skill in the art will recognize how other subgroups of LEDs may be grouped, where the grouping and grouping sizes depend at least partially on an overall number of LEDs within the LED array (e.g., an LED array comprised of a 3x3 matrix of LEDs, a 5x5 matrix of LEDs, or a 100x100 matrix of LEDs).

[0035] Unlike the backlight unit 101 of FIGS. 1A and IB, which are constantly illuminated, generally across an entire area of the backlight unit 101, the LED array can be only partially lighted in order to track a current location of the eye-pupil 209B (e.g., only a single LED is lighted or a subgroup of LEDs is lighted). Since the LED array provides light to the eye-pupil 209B and not to the entirety of the eye-box 107, a smaller amount of energy is needed to produce light directed to only a portion of the eye-box 107. Optical rays are shown as emanating from a single point of an uppermost one of the LEDs 201A in a direction toward a human eye 209A, in order to provide a better understanding of the portion of the HMD 100. As shown in FIG. 2A, the human eye 209A is substantially centered, from top to bottom, within the eye -box 107.

[0036] Moreover, although only three LEDs are shown, a skilled artisan, upon reading and understanding the disclosed subject-matter will recognize that any number of LEDs may be used. Also, the skilled artisan will recognize that although only a two-dimensional view is shown of the near-to-eye HMD projection system 200, a third dimension (into the plane of the sheet) may also be populated with a number of LEDs (as part of the LED array) to provide adaptive lighting to the eye-pupil 209B of the human eye 209A, and a resulting movement of the eye-pupil 209B, as it moves or rotates within the third dimension.

[0037] FIG. 2A is also shown to include a collimator lens 207, an LCD display 203, and an eyepiece lens 205. The LCD display 203 may be the same as or similar to the mini-LCD 103 of FIG. 1A and FIG. IB. Also, the eyepiece lens 205 may be the same as or similar to the eyepiece lens 105 of FIG. 1A. The LED array, comprising the LEDs 201A, 20 IB, 201C, is located at a first distance 211, f_c, from the collimator lens 207. The first distance 211 can be considered as the focal length of the collimator lens 207. The collimator lens 207 is located at a second distance 213, d, from the LCD display 203. The LCD display 203 is located at a third distance 215, f_e, from the eyepiece lens 205. The eyepiece lens 205 is located at a fourth distance 217, also f_e, from the eye-box 107. The third distance 215 and the fourth distance 217 can be considered as the focal length of the eyepiece lens 205.

[0038] Although shown as a single bi-convex lens, the eyepiece lens 205 can be any type of lens, or a combination of lenses, as would be recognized by a person of ordinary skill in the art upon reading and understanding the disclosed subject matter. Therefore, the eyepiece lens 205 can be considered to be an eyepiece element and is arranged to project images generated by the LCD display 203 toward a location of the eye-pupil 209B within a volume of the eye-box 107. Additionally, although shown as a single bi-convex lens, the collimator lens 207 can be any type of lens, or a combination of lenses, as would be recognized by a person of ordinary skill in the art upon reading and understanding the disclosed subject matter. Therefore, the collimator lens 207 can be considered to be a collimating element.

[0039] The optical rays of FIG. 2A, shown only for single points on the center of one LED, indicate how switching on only the on-axis LED, the LED 20 IB, directs light from the LCD display 203 to the on- axis location of the human eye 209A and, specifically, to the eye-pupil 209B. Consequently, since power is only being supplied to an appropriate one of LEDs in the LED array, in this case the LED 20 IB, to illuminate the on-axis location of the eye-pupil 209B, significant power is saved by not switching power to each of the LEDs 201A, 20 IB, 201C within the LED array. However, upon reading and understanding the disclosed subject matter, a person of ordinary skill in the art will recognize that there may also be one or more LEDs switched on in the third dimension depending on a location of the human eye 209A relative to the third dimension.

[0040] FIG. 2B shows an example of an occluded optical- architecture for a near-to-eye HMD projection system 220 with LEDs 201A, 20 IB, 201C (although only the lower of the two off-axis LEDs, the LED 201C, is lighted), switched on for an eye-pupil 209D alignment, in accordance with various embodiments of the disclosed subject matter. In contrast with an on-axis location of the eye-pupil 209B of FIG. 2A, the eye-pupil 209D of a human eye 209C is shifted to an uppermost portion of the eye-box 107. The optical rays of FIG. 2B, shown only for single points on the center of one LED, indicate how switching on only the lower of the two off-axis LEDs, the LED 201C, directs light from the LCD display 203 to the off-axis location of the human eye 209C and, specifically, to the eye-pupil 209D.

Consequently, since power is being supplied only to an appropriate one of LEDs in the LED array, in this case the LED 201C, to illuminate a location of the off-axis location of the eye-pupil 209D, significant power is saved by not switching power to each of the LEDs 201A, 20 IB, 201C within the LED array.

[0041] In the embodiments shown in FIGS. 2A and 2B, there is an LED array located on the focal plane of the collimator lens 207. Collimated rays from each LED within the LED array impinges on a mini-LCD, the LCD display 203, which functions in transmission mode. The eyepiece lens 205 is located at its focal distance, f_e, from the mini-LCD and generates a virtual image of the LCD at infinity relative to the viewer (the location of the human eye 209A, 209C). Power switching as shown in FIGS. 2 A and 2B is on the input end, at the LED array, prior to the transmissive optics (e.g., the collimator lens 207 and the eyepiece lens 205.

[0042] FIG. 3A shows an example of an occluded optical- architecture for a near-to-eye HMD projection system 300 with LEDs 301A, 30 IB, 3010 (although the only on-axis LED 30 IB is hghted), switched on for an eye-pupil 309B alignment, in accordance with various embodiments of the disclosed subject matter. As with FIGS. 2A and 2B, the LEDs 301A, 30 IB, 301C may be considered as being part of an LED array. Again, unlike the backlight unit 101 of FIGS. 1A and IB, which are constantly lighted, only one LED in the LED array can be lighted in order to track a current location of the eyepupil 309B. Since the LED array provides light to the eye-pupil 309B and not to the entirety of the eye-box 107, a smaller amount of energy is needed to produce light directed to only a portion of the eye-box 107. Optical rays are shown as emanating from a single point of an uppermost one of the LEDs 201A in a direction toward a human eye 209A, in order to provide a better understanding of the portion of the HMD 100. As shown in FIG. 2A, the human eye 209A is substantially centered, from top to bottom, within the eye-box 107.

[0043] FIG. 3A is also shown to include a collimator lens 307, an LCD display 303, and an eyepiece lens 305. The LCD display 303 may be the same as or similar to the mini-LCD 103 of FIGS. 1 and IB, or the LCD display 203 of FIGS. 2 A and 2B. Also, the eyepiece lens 305 may be the same as or similar to the similar components of FIGS. 1A, IB, 2 A, and 2B. The LED array, comprising the LEDs 301 A, 30 IB, 301C, is located at a first distance 311, f_c, from the collimator lens 307. The first distance 311 can be considered as the focal length of the collimator lens 307. The collimator lens 307 is located at a second distance 313, d, from the LCD display 303. The LCD display 303 is located at a third distance 315, fe, from the eyepiece lens 305. The eyepiece lens 305 is located at a fourth distance 317, also f_e, from the eye-box 107. The third distance 315 and the fourth distance 317 can be considered as the focal length of the eyepiece lens 305.

[0044] The optical rays of FIG. 3A, shown only for a single pixel on the LCD display 303, indicate how switching on only the on-axis LED, the LED 30 IB, directs light from the LCD display 303 to the on-axis location of the human eye 309A and, specifically, to the eye-pupil 309B.

[0045] FIG. 3B shows an example of an occluded optical- architecture for a near-to-eye HMD projection system 320 with LEDs 201A, 20 IB, 201C (although only the lower of the two off-axis LEDs, the LED 301C, is lighted), switched on for an eye-pupil 309D alignment, in accordance with various embodiments of the disclosed subject matter. In contrast with an on-axis location of the eye-pupil 309B of FIG. 3A, the eye-pupil 309D of a human eye 309C is shifted to an uppermost portion of the eye-box 107. The optical rays of FIG. 3B, shown only for a single pixel on the LCD display 303, indicate how switching on only the lower of the two off-axis LEDs, the LED 301C, directs light from the LCD display 303 to the off-axis location of the human eye 309C and, specifically, to the eye-pupil 309D.

Consequently, since power is being supplied only to an appropriate one of LEDs in the LED array, in this case the LED 301C, to illuminate the off-axis location of the eye-pupil 309D, significant power is saved by not switching power to each of the LEDs 301A, 30 IB, 301C within the LED array.

[0046] With continuing reference to FIGS. 3A and 3B, the third distance 315 between the eyepiece lens 305 and the LCD display 303 can be modified to bring the virtual image closer to the viewer. [0047] FIG. 4 is an exemplary embodiment of a layout of an adaptive illuminator system 400 for an HMD with an eye-tracking module 423, in accordance with various embodiments of the disclosed subject matter. The eye-tracking module 423 keeps track of a position of an eye-pupil of a human eye 417 within an eye-box (not shown explicitly within FIG. 4, but understandable based on descriptions of other eye-boxes disclosed herein). The eye-tracking module 423 can comprise, for example, a camera, as is known in the art, or another type of eye-tracking device known in the relevant art. Further, although the eye-tracking module 423 is shown coupled to the adaptive illuminator system 400 with a particular type of display device, the same or similar type of eye-tracking module can be used with any of the embodiments disclosed herein. For example,, an eyetracking module can be used with the near-to-eye HMD projection system 200 of FIG. 2.

[0048] With continuing reference to the adaptive illuminator system 400 of FIG. 4, once the eye-tracking module 423 determines a location of the eye-pupil, a signal is sent to a processor and commanding unit 421. The processor and commanding unit 421 sends an additional signal directing an illumination driver 401, coupled to an LED array 403 (e.g., the same as or similar to the LED arrays of FIG. 2A, 2B, 3A, and 3B), to switch on (or off) one or more corresponding LEDs within the LED array 403, for a given determined location of the eye-pupil as determined by the eyetracking module 423. FIG. 4 is further shown to include a collimator lens 407, a first beam splitter 409, a reflective, liquid crystal on silicon (LCoS) display- device 411, an eyepiece lens 413, and a second beam splitter 415. In a specific exemplary embodiment, the beam splitters 409, 415 may comprise a polarizing-type beam splitter to account for the change in polarization state once illumination provided from the LED array 403 reflects from the LCoS displaydevice 411. In embodiments, the beam splitters may comprise, for example, plate beam splitters or cube beam splitters. In other embodiments, one or more of the beam splitters may be substituted by any optical device that is capable of splitting a beam of light into a partially transmitted beam and a partially reflected beam (e.g., a partially silvered mirror).

[0049] In various embodiments, the LED array 403 can be an array of only white LEDs, or it can be an array of RGB LEDs, or various other types, colors, and combinations of colors of LEDs. When using a white version of the LED array 403, the LCoS display- device 411 can include RGB filters to generate colored images. Alternatively, when using an RGB version of the LED array 403, the LCoS display-device 411 can be used without color filters and can be synchronized with the red, green, and blue LEDs with corresponding patterns for red, green, and blue frames to generate colored images.

[0050] With reference now to FIG. 5, FIG. 5 shows an exemplary embodiment of an optical layout for a near-to-eye HMD projection system 500 having an LED 501, a collimator lens 503, an LCD display 505, an eyepiece lens 507, and a human eye 509A with an eye-pupil 509B, in accordance with various embodiments of the disclosed subject matter. FIG. 5 is also shown to include a first distance 511, si (e.g., an arbitrary LED-to-collimator distance), a second distance 513, d, a third distance 515, f_e, and a fourth distance 517, S2 (e.g., an arbitrary eyepiece-to-eye-pupil distance).

[0051] FIG. 5 is provided so that a person of ordinary skill in the art may better appreciate how locations of various components within any of the adaptive illumination systems (e.g., projection systems) disclosed herein may be determined. Upon reading and understanding the disclosed subject matter, a person of ordinary skill in the art can calculate the size of an adaptive eye-box using the magnification of the LED image on the eye-pupil location to a first- order Gaussian approximation. For example, with continuing reference to a general optical layout of FIG. 5, with the first distance, an arbitrary LED-to-collimator distance of s₁₍ and a fourth distance 517, an eyepiece to eye-pupil distance of s₂, adaptive eye-box location is at:

An LED image on the location of eye-pupil is magnified by a factor of:

[0052] If an LED-to-collimator distance is equal to focal length of the collimator lens such that = f_c, then the eye-box is located at f ^s2 ⁼ fe and magnification is m = — . For example, for a collimator lens fc having a focal length of 8 mm, and an eyepiece lens having a 24mm focal length, the magnification is m = 3. Therefore, to build a 2 mm by 2 mm adaptive eye-box, the LED dimensions should be about 0.67 mm by about 0.67 mm. An area defined by these dimensions for the LED could include a single emitter or multiple mini- or micro -emitters with, for example, RGB colors.

[0053] FIG. 6 shows an exemplary embodiment of an eye-box size and location diagram 600 for an occluded on-axis near-to-eye HMD, in accordance with various embodiments of the disclosed subject matter. FIG. 6 is shown to include an LCD display 601, having a first dimension Di (height Ha), an eyepiece lens 603, having a second dimension D2 (height H_e), and an eye-box 107 having an open aperture with a third dimension D3 (height Hb). The third dimension D3 (height Hb) defines a region of the eye-box 107 without vignetting. FIG. 6 is also shown to include a display-to-eyepiece-lens distance 605fe), an eyepiece-lens-to-eye-box distance 607 de), and an eyepiece- lens-to-focal length through the eye-box distance 609 (f_e). A region 611 of a volume of the eye-box 107 is shown that is substantially free of vignetting.

[0054] In FIG. 6, a relative size of the eye-box 107 is calculated for an HMD with the LCD display 601, having a size of H_d, and eye piece aperture size of H_e, at a distance of d_e from the eye piece lens as indicated by equation (4).

[0055] This is substantially the maximum size of an eye-box for an occluded system that is substantially free of vignetting (the region 611). It should be noted that this eye-box is different than the adaptive eye-box. The adaptive eye-box is a subset of the system eyebox which can be scanned (or positioned) to the location of an eyepupil. For example, for eyepiece lens-to-eye-pupil distance of d_e = f_e = 24 mm and a diameter of the eyepiece lens 603 of H_e = 14 mm and a size of the LCD display 601 of H_d = 7 mm, a size of the eye-box 107 determined in accordance with equation (4) is H_b = 7 mm that is substantially without vignetting.

[0056] FIG. 7 shows an exemplary embodiment of a see-through configuration of an HMD with an adaptive eye-box illuminator, in accordance with various embodiments of the disclosed subject matter. The adaptive eye-box illuminator system of FIG. 7 is shown to include LEDs 701A, 70 IB, 701C (with only the on-axis LED 70 IB being lighted for this example), a collimator lens 703, a first cube-type beam-splitter 705, having a first beam-splitter surface 707, a reflective, liquid crystal on silicon (LCoS) display-device 709, an eyepiece lens 711, and a second cube-type beam-splitter 713, having a second beam-splitter surface 715. As shown, the second beam-splitter surface 715 reflects light from the LCoS display-device 709, illuminated by the LED 70 IB, to an eye-pupil 717B of a human eye 717A. The collimator lens 703 and the eyepiece lens 711 may the same as or similar to, for example, the collimator lens 207 and the eyepiece lens 205, respectively, of FIG. 2B.

[0057] The “folded” arrangement of the see-through configuration of an HMD with the adaptive eye-box illuminator of FIG. 7, allows a wearer of the HMD to view a scene that lies behind (to the left of) the second cube-type beam-splitter 713. Upon reading and understanding the disclosed subject matter, a person of ordinary skill in the art will recognize how to apply the adaptive eye-box illuminator in this see- through configuration to versions of occluded embodiments disclosed herein.

[0058] In a specific exemplary embodiment, the cube-type beam splitters 705, 713 may each comprise a polarizing-type beam splitter to account for the change in polarization state once illumination provided from a selected one or ones of the LEDs 701A, 70 IB, 701C reflects from the LCoS display-device 709. In embodiments, the cubetype beam splitters 705, 713 may instead comprise, for example, plate beam-splitters. In other embodiments, one or more of the beam splitters may be substituted by any optical device that is capable of splitting a beam of light into a partially transmitted beam and a partially reflected beam (e.g., a partially silvered mirror). The LEDs 701A, 70 IB, 701C may be considered as being part of an LED array.

[0059] FIG. 8 shows an exemplary embodiment of a compact-design alternative of a see-through HMD with an adaptive eye box illuminator, in accordance with various embodiments of the disclosed subject matter. FIG. 8 is shown to include LEDs 801A, 80 IB, 8010 (with only the on-axis LED 80 IB being lighted for this example), a collimator lens 803, a reflective, liquid crystal on silicon (LCoS) display- device 809, and a single, prismatic component 805A. The a single, prismatic component 805A includes a first beam-splitter surface 807, a second beam-splitter surface 811, and an eyepiece lens 805B (e.g., a concave, reflecting lens or reflecting lens element). As shown, the second beam-splitter surface 811 reflects light from the LCoS display- device 809 illuminated by the LED 80 IB, and reflected by the eyepiece lens 805B, to an eye-pupil 807B of a human eye 807A. The collimator lens 803 may the same as or similar to, for example, the collimator lens 207 of FIG. 2B. The LEDs 801A, 80 IB, 801C may be considered as being part of an LED array.

[0060] The “folded” arrangement of the see-through configuration of the HMD with the adaptive eye-box illuminator of FIG. 8 having the single, prismatic component 805A, allows a wearer of the HMD to view a scene that lies beyond the second beam-splitter surface 811 and therefore behind (to the left of) the single, prismatic component 805A. Upon reading and understanding the disclosed subject matter, a person of ordinary skill in the art will recognize how to apply the adaptive eye-box illuminator in this see-through configuration of FIG. 8 to versions of occluded embodiments disclosed herein.

[0061] FIG. 9 shows an exemplary embodiment of a near-to-eye HMD folded-design with thin polarizing beam-splitters (PBSs) 905, 911 in a see-through HMD-system design. FIG. 9 is shown to include LEDs 901A, 90 IB, 901C (with only the on-axis LED 90 IB being lighted for this example), a collimator lens 903, a reflective, liquid crystal on silicon (LCoS) display- device 907, a first polarizing-type beam-splitter 905, a second polarizing-type beam-splitter 911, and an eyepiece lens 909. As shown, the second polarizing-type beam-splitter 911 reflects light from the LCoS display- device 907 illuminated by the LED 90 IB, and directed through by the eyepiece lens 909, to an eyepupil 913B of a human eye 913A. The collimator lens 903 and the eyepiece lens 909 may the same as or similar to, for example, the collimator lens 207 and the eyepiece lens 909, respectively, of FIG. 2B. The LEDs 901A, 90 IB, 901C may be considered as being part of an LED array.

[0062] The “folded” arrangement of the see-through configuration of the near-to-eye HMD of FIG. 9 allows a wearer of the HMD to view a scene that lies behind (to the left of) second polarizing-type beamsplitter 911, and an eyepiece lens 909. Upon reading and understanding the disclosed subject matter, a person of ordinary skill in the art will recognize how to apply the adaptive eye-box illuminator in this see-through configuration of FIG. 9 to versions of occluded embodiments disclosed herein..

[0063] The description above includes illustrative examples, devices, systems, and apparatuses that embody the disclosed subject matter. In the description, for purposes of explanation, numerous specific details were set forth in order to provide an understanding of various embodiments of the disclosed subject matter. It will be evident, however, to those of ordinary skill in the art that various embodiments of the subject matter may be practiced without these specific details. Further, well-known structures, materials, and techniques have not been shown in detail, so as not to obscure the various illustrated embodiments.

[0064] As used herein, the term “or” may be construed in an inclusive or exclusive sense. Further, other embodiments will be understood by a person of ordinary skill in the art upon reading and understanding the disclosure provided. Further, upon reading and understanding the disclosure provided herein, the person of ordinary skill in the art will readily understand that various combinations of the techniques and examples provided herein may all be applied in various combinations.

[0065] Although various embodiments are discussed separately, these separate embodiments are not intended to be considered as independent techniques or designs. As indicated above, each of the various portions may be inter-related and each may be used separately or in combination with other types of electrical controldevices, such as dimmers and related devices. Consequently, although various embodiments of methods, operations, and processes have been described, these methods, operations, and processes may be used either separately or in various combinations.

[0066] Consequently, many modifications and variations can be made, as will be apparent to a person of ordinary skill in the art upon reading and understanding the disclosure provided herein. Functionally equivalent methods and devices within the scope of the disclosure, in addition to those enumerated herein, will be apparent to the skilled artisan from the foregoing descriptions. Portions and features of some embodiments may be included in, or substituted for, those of others. Such modifications and variations are intended to fall within a scope of the appended claims. Therefore, the present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0067] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. The abstract is submitted with the understanding that it will not be used to interpret or limit the claims. In addition, in the foregoing Detailed Description, it may be seen that various features may be grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as limiting the claims. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.

THE FOLLOWING NUMBERED EXAMPLES ARE SPECIFIC EMBODIMENTS OF THE DISCLOSED SUBJECT-MATTER

[0068] Example 1: An embodiment of the disclosed subject-matter describes an apparatus to provide adaptive illumination to portions of an eye-box in which an eye-pupil is located. The apparatus includes a light-emitting diode (LED) array having a plurality of LEDs. Each of the plurality of LEDs is individually addressable in subsets of the LED array with at least one of the plurality of LEDs within a subset of the LED array being configured to be lighted based on a location of the eye-pupil with the eye-box. The apparatus further includes a collimating element located between the LED array and the eye-box. A display device is located between the collimating lens and the eyebox; the display device is configured to generate images. An eyepiece element is located between the display device and the eye-box to receive light from the at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within an area of the eye-box.

[0069] Example 2: The apparatus of Example 1, wherein the subsets of the LED array comprise at least one LED and fewer than all of the LEDs within the LED array.

[0070] Example 3: The apparatus of either Example 1 or Example 2, wherein the display device is a liquid crystal display (LCD) device.

[0071] Example 4: The apparatus of any one of the preceding Examples, further comprising an eye-tracking module to determine the location of the eye-pupil within the eye-box.

[0072] Example 5: The apparatus of Example 4, further comprising an illumination driver coupled to the LED array to switch one or more of the plurality of LEDs within the subset of the LED array that correspond to the determined location of the eye-pupil with the eyebox.

[0073] Example 6: The apparatus of Example 5, further comprising a processor and commanding unit configured to send a signal directing the illumination driver to switch the one or more of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box.

[0074] Example 7: The apparatus of any one of the preceding Examples, wherein the apparatus is a portion of a head-mounted display (HMD).

[0075] Example 8: The apparatus of Example 7, wherein the apparatus is a portion of an occluded optical-architecture of the HMD, whereby at least a portion of a scene viewable to a wearer of the HMD is blocked by the apparatus. [0076] Example 9: The apparatus of Example 7, further comprising at least one beam splitter located between the LED array and the eyebox, wherein the at least one beam splitter allows a wearer of the HMD to view a substantial entirety of a scene that is not blocked by the apparatus.

[0077] Example 10: An embodiment of the disclosed subject-matter describes a head-mounted display (HMD) with an adaptive illumination apparatus. The HMD includes a light-emitting diode (LED) array having a plurality of LEDs. Each of the plurality of LEDs is individually addressable in subsets of the LED array, with at least one of the plurality of LEDs within a subset of the LED array being configured to be lighted based on a location of an eye-pupil with a volume of an eye-box within the HMD. A collimating element is located between the LED array and the eye-box. A display device is located between the collimating lens and the eye-box, with the display device being configured to generate images. An eyepiece element is located between the display device and the eye-box to receive light from the at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within a volume of the eye-box. At least one beam splitter is located between the LED array and the eyebox. The at least one beam splitter is configured to allow a wearer of the HMD to view a substantial entirety of a scene that is not blocked by the adaptive illumination apparatus.

[0078] Example 11: The HMD of Example 10, wherein the subsets of the LED array comprise at least one LED and fewer than all of the LEDs within the LED array.

[0079] Example 12: The HMD of either Example 10 or Example 11, wherein the display device is a liquid crystal display (LCD) device. [0080] Example 13: The HMD of any one of Example 10 through Example 12, further comprising an eye-tracking module to determine the location of the eye-pupil within the eye-box.

[0081] Example 14: The HMD of Example 13, further comprising an illumination driver coupled to the LED array to switch one or more of the plurality of LEDs within the subset of the LED array that correspond to the determined location of the eye-pupil with the eyebox.

[0082] Example 15: The HMD of Example 13, further comprising a processor and commanding unit configured to send a signal directing the illumination driver to switch the one or more of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box.

[0083] Example 16: An embodiment of the disclosed subject-matter describes a method for providing adaptive illumination to portions of an eye-box in which an eye-pupil is located. The method includes determining a position of the eye-pupil; sending a signal to a lightemitting diode (LED) array having a plurality of LEDs, where each of the plurality of LEDs is individually addressable in subsets of the LED array; and lighting at least one of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box.

[0084] Example 17: The method of Example 16, further comprising: receiving light from the illuminated portions of LED array by the at least one of the plurality of LEDs within the subset of the LED array and projecting images generated by a display device toward a location of the eye-pupil within a volume of the eye-box. [0085] Example 18: The method of either one of Example 16 or Example 17, further comprising switching the one or more of the plurality of LEDs within the subset of the LED array that correspond to the determined location of the eye-pupil with the eye-box.

Claims

What is claimed is:

1. An apparatus to provide adaptive illumination to portions of an eye-box in which an eye-pupil is located, the apparatus comprising: a light-emitting diode (LED) array having a plurality of LEDs, each of the plurality of LEDs being individually addressable in subsets of the LED array, at least one of the plurality of LEDs within a subset of the LED array being configured to be lighted based on a location of the eye-pupil with the eye-box; a collimating element located between the LED array and the eyebox; a display device located between the collimating lens and the eye-box, the display device being configured to generate images; and an eyepiece element located between the display device and the eyebox to receive light from the at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within an area of the eye-box.

2. The apparatus of claim 1, wherein the subsets of the LED array comprise at least one LED and fewer than all of the LEDs within the LED array.

3. The apparatus of claim 1, wherein the display device is a Equid crystal display (LCD) device. apparatus of claim 1, further comprising an eye-tracking module to determine the location of the eye-pupil within the eye-box. apparatus of claim 4, further comprising an illumination driver coupled to the LED array to switch one or more of the plurality of LEDs within the subset of the LED array that correspond to the determined location of the eye-pupil with the eye-box. apparatus of claim 5, further comprising a processor and commanding unit configured to send a signal directing the illumination driver to switch the one or more of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box. apparatus of claim 1, wherein the apparatus is a portion of a head-mounted display (HMD). apparatus of claim 7, wherein the apparatus is a portion of an occluded optical-architecture of the HMD, whereby at least a portion of a scene viewable to a wearer of the HMD is blocked by the apparatus. apparatus of claim 7, further comprising at least one beam splitter located between the LED array and the eye-box, wherein the at least one beam splitter allows a wearer of the HMD to view a substantial entirety of a scene that is not blocked by the apparatus.

10. A head-mounted display (HMD) with an adaptive illumination apparatus, the HMD comprising: a hght-emitting diode (LED) array having a plurality of LEDs, each of the plurality of LEDs being individually addressable in subsets of the LED array, at least one of the plurality of LEDs within a subset of the LED array being configured to be lighted based on a location of an eye-pupil with a volume of an eye-box within the HMD; a collimating element located between the LED array and the eyebox; a display device located between the collimating lens and the eye-box, the display device being configured to generate images; an eyepiece element located between the display device and the eyebox to receive light from the at least one of the plurality of LEDs within the subset of the LED array and to project the images generated by the display device toward a location of the eye-pupil within a volume of the eye-box; and at least one beam splitter located between the LED array and the eyebox, the at least one beam splitter is configured to allow a wearer of the HMD to view a substantial entirety of a scene that is not blocked by the adaptive illumination apparatus.

11. The HMD of claim 10, wherein the subsets of the LED array comprise at least one LED and fewer than all of the LEDs within the LED array.

12. The apparatus of claim 10, wherein the display device is a liquid crystal display (LCD) device.

13. The apparatus of claim 10, further comprising an eye-tracking module to determine the location of the eye-pupil within the eye-box.

14. The apparatus of claim 13, further comprising an illumination driver coupled to the LED array to switch one or more of the plurality of LEDs within the subset of the LED array that correspond to the determined location of the eye-pupil with the eye-box.

15. The apparatus of claim 13, further comprising a processor and commanding unit configured to send a signal directing the illumination driver to switch the one or more of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box.

16. A method for providing adaptive illumination to portions of an eye-box in which an eye-pupil is located, the method comprising: determining a position of the eye-pupil; sending a signal to a light-emitting diode (LED) array having a plurality of LEDs, each of the plurality of LEDs being individually addressable in subsets of the LED array; and lighting at least one of the plurality of LEDs within the subset of the LED array based on the determined location of the eye-pupil with the eye-box.

17. The method of claim 16 further comprising: receiving light from the illuminated portions of LED array by the at least one of the plurality of LEDs within the subset of the LED array; and projecting images generated by a display device toward a location of the eye-pupil within a volume of the eye-box.

18. The method of claim 16, further comprising switching the one or more of the plurality of LEDs within the subset of the LED array that correspond to the determined location of the eyepupil with the eye-box.