WO2009126264A2 - Proximal image projection system - Google Patents

Abstract

A display system 10 for projecting an image onto the retina of an eye is disclosed, in which a diffractive proximal optic 16 has a plurality of distinct redirection areas that correspond to, intersect, and alter respective trajectories a light beam. A frame 12 is adapted to support the proximal optic 16 and a dual beam optical projector 20, such that a foveal beam 46 and a peripheral beam 44 cooperate to form the projected image on respective areas of the retina, while accommodating eye rotation. An enhanced eyeglasses system 190 is also disclosed, that enables additional electronic features.

Description

PROXIMAL IMAGE PROJECTION SYSTEM

BACKGROUND Technical Field

The present invention generally relates to imaging systems for individual use and, in particular, head-worn display devices, or "personal display devices" that display images to the individual while permitting the individual to observe the real world scene within the individual's field of view. Related Applications

The present application claims benefit of the following provisional patent applications: U.S. Patent Application No. 61/042,762, filed 6 April 2008; U.S. Patent Application No. 61/042,764 filed 6 April 2008; U.S. Patent Application No. 61/045,367 filed 16 April 2008; U.S. Patent Application No. 61/050,602 filed 5 May 2008; U.S. Patent Application No. 61/056,056 filed 26 May 2008; U.S. Patent Application No. 61/ 042,766 filed 6 April 2008; U.S. Patent Application No. 61/050,189 filed 2 May 2008; U.S. Patent Application No. 61/077,340 filed 1 July 2008; U.S. Patent Application No. 61/057,869 filed 1 June 2008; and U.S. Patent Application No. 61 ,110,591 filed 2 November 2008. Description of the Related Art

Display systems for individual use, particularly head-worn display devices that enable an individual to see natural or electronically produced images are of significant commercial interest (see J. Rolland and O. Cakmakci, "Head-worn Displays: The Future Through New Eyes," Optics and Photonics News, April 2009, hereby incorporated by reference in its entirety). In addition, there is notable interest in overlaying computer-generated imagery on real-world scenes - a concept known as "augmented reality" - by displaying the computer-generated images while simultaneously allowing the individual to see the real world through a "proximal optic." For example, there is interest in an individual display that allows an individual to see the real world while also viewing video content (either pre-recorded or live streaming video from cameras), interactive electronic gaming, computing, viewing "heads up display" information, discreetly reading text, viewing maps, receiving notifications, and so forth. Electronically produced images may even be substantially indistinguishable from an actual scene viewed through traditional eyeglasses.

If the proximal optic is at least partly transparent, a constructed image may be superimposed upon the image of a background scene i.e., the viewer's physical surroundings. A proximal optic may be integrated into an eyeglass lens, through which a viewer perceives a combination of constructed and background scene images. Although a proximal optic may, in such an embodiment, appear to be the same as a traditional eyeglass lens, its structure and method of manufacture may differ considerably. A significant design challenge is recognized to be posed by the requirements of a mobile, miniaturized light projection sub-system offering high perceived resolution, wide field of view and an inconspicuous integration into the conventional eyeglasses form factor. The underlying technology, however, can be applied even more widely. If the proximal optic were used as a simple screen for displaying a superimposed projected image, it would be necessary for the viewer to focus on an image at a typical eyeglass distance of about 2 - 3 cm in front of the eye. However, rays from objects closer than about 25 cm (the "near point") cannot be focused by most adult human beings. Hence, a problem posed by proximal optic displays (for example, a helmet-mounted or eyeglass-mounted heads-up display) is that the images are simply too close up to see. Therefore, images are preferably focused directly onto the retina of the eye, the proximal optic serving as a reflective or diffractive optical element that directs light through the pupil to illuminate the retina. Focusing an image directly onto the retina of the eye consequently poses further challenges. One challenge concerns maximizing the field of view by steering the light beam toward foveal and peripheral receptors, corresponding to central and peripheral regions of the retina.

European patent EP 0 473 343 A1 discloses writing an image directly onto the retina, but does not provide a heads-up display. European patent EP 0 367 534 A2 provides a heads-up display but does not provide for direct beam writing onto the retina. U.S. Patent No. 4,513,317 (hereafter, the ^"317 patent) is directed to a television display having dynamic resolution that responds to eye movement, such that the high resolution capability is preferentially used for that portion of a display

"33-001350 upon which the eye is focused. However, the '317 patent is not directed to the construction of an image on the retina. Finally, existing proximal optic designs for heads-up proximal displays typically employ a flat, spherical, or otherwise continuous, regular surface to re-direct the incident light beam into the eye and have limited field of view and/or perceived resolution, and in many cases cumbersome form factors.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a top-down schematic of a personal display system integrated into an eyeglasses frame, positioned with respect to an eye.

Fig. 1 A is a cross-sectional view of a proximal optic integrated into the lens of the eyeglasses configuration shown in Fig. 1

Fig. 2 is an optical block diagram of a dual-beam projector, shown in Fig. 1 as attached to the arm of the eyeglasses.

Fig. 3 is an illustration of the trajectories of a foveal scanner beam and a peripheral scanner beam.

Fig. 4 is a pictorial view of a design for a two-dimensional array of Bragg reflectors, according to a preferred embodiment. Fig. 5 is a pictorial view of image spot sizes "painted" on the retina when the dual- beam projector of Fig. 2 is used to scan the array of Fig. 4.

Fig. 6 is an optical schematic of an apparatus that could be used to generate a volume hologram of an arrangement of angled mirrors, according to a preferred embodiment. Fig. 7 is a flow chart describing a feedback-control process for accommodating eye movement.

Figs. 8A and 8B are side views of an enhanced eyeglass frame, indicating placement of enhancement components according to a preferred embodiment.

Figs. 8C and 8D are front views of an enhanced eyeglass frame, indicating placement of enhancement components according to a preferred embodiment.

Figs. 9A, 9B, and 9C are block diagrams indicating sets of features provided by the enhancement components shown in Figs. 8A - 8D.

"^33-001350 Fig. 1OA is a top view of a projector component mounted to the enhanced eyeglass frame shown in Figs. 8A - 8D.

Fig. 1OB is a front view of one lens of an enhanced eyeglass equipped with a proximal optic, in which a dashed line area is illuminated by the projector of Fig. 10A. Fig. 10C is a top-down view of the top edge of the eyeglasses lens frame shown in

Fig. 10B.

Fig. 11 A is a front view of enhanced eyeglasses indicating the locations of embedded components, according to one embodiment.

Fig. 11 B is a side view of a frame arm indicating the locations of embedded components, according to one embodiment.

Fig. 11C is a top-down view of the enhanced eyeglasses frame showing embedded components.

Figs 12A - 12C is a front view of an enhanced eyeglasses frame showing preferred locations of audio and video transducer components. Figs. 13 A - 13C are top views of the corner of an enhanced eyeglass frame, showing the positioning of electrical wiring relative to the hinge, for three alternative embodiments

Fig. 13D is a front view of one half of an enhanced eyeglasses frame showing the locations of hinges. Figs. 14A and 14B are top-down views of enhanced eyeglasses shown configured with a payload.

Fig. 15A is a top-down view of an eyeglasses case (or a charging station) configured with coils for powering the enhanced eyeglasses.

Fig. 15B is a front view of enhanced eyeglasses configured with coils for inductively or capacitively coupling to the case shown in Fig. 15A.

Fig 16A is a pictorial view of a coil design that may be manufactured by a subtractive patterning process.

Figs. 16B and 16C are side views of the earpiece of an eyeglasses frame equipped with the coil of Fig. 16A. Fig. 16B shows a coil internal to the earpiece; Fig. 16C shows a coil embedded in a lanyard boot.

Fig. 16D is a cross-sectional view of a lanyard boot equipped with the coil shown in

Fig. 16A.

Fig. 17A is a side view of an eyeglass arm equipped with an arm attachment.

^"33-001350 Fig. 17B is a side view of an eyeglass arm that accommodates fasteners for applying the arm attachment of Fig. 17A

Fig. 17C is a side view of an arm attachment equipped with male fasteners and a clip. Figs 18A - 18C are side views of an eyeglass arm designed and equipped with hinges that allow for electrical communications.

DETAILED DESCRIPTION OF EMBODIMENTS Overview of a Preferred Embodiment Referring to Fig. 1 , a preferred embodiment of a display system, in this case a head-worn personal display system 10, generally comprises an eyeglasses frame 12, an eyeglasses lens 14 mounted in frame 12, a proximal optic 16 attached to the back side of the lens 14 (the side of the lens closest to the eye 18 of the person wearing the eyeglasses), and an optical projector 20 supported by an arm 22 of the eyeglasses closest to the lens 14. In this exemplary embodiment, the projector forms an image on the retina 24 of the eye 18 by scanning one or more beams of light into the eye to write individual image pixels onto the retina of the eye. (These image pixels are also referred to herein as "retinal pixels.") It is to be understood that the eyeglasses 12 would ordinarily have a complementary portion for the other eye of the person wearing the eyeglasses, which may, but does not necessarily need to, include another display system

In this preferred embodiment, the proximal optic preferably has a plurality of distinct redirecting areas which enable a beam of light from the projector having a selected trajectory to be redirected into the eye 18 and to a selected location on the retina 24 of the eye so as to write an image, or retinal pixel at that location. This is so generally regardless of the direction of the optical axis 26 of the eye 18, that is, regardless of the direction that the eye is looking. To reduce the bandwidth of information needed to produce an acceptable image on the retina 24, a novel type of foveal imaging is used. That is, a collimated beam of light of sufficient diameter is scanned over the high acuity foveal region 28 of the retina 24 to produce a high resolution image on that region, while a beam with a larger spot size at the retina is written onto the peripheral region 30 of the retina, where the lower resolution produced thereby does not detract significantly from the overall perceived quality of

"33-001350 the image. As will be explained more fully below, to accomplish these results generally regardless of the direction the eye is looking, and in color, the proximal optic comprises a diffractive optic, specifically a volume hologram 32, as illustrated in cross-section by Fig. 1A. Additional well-known electronic systems, such as a video input source for modulating the light beams; wireless communications electronics; power supply; user interface; a microprocessor/controller and associated memory devices; interface, data processing and image processing hardware and software; and other sensory transducers may be provided in the display system. Descriptions of such components along with embodiments of form factors for supporting the electronic functionality of the display system 10 are disclosed below in a section describing "enhanced eyeglasses." The Projector Preferably, the optical projector 20 of the head-worn display system 10 comprises a dual beam scanning system according to a preferred embodiment as shown more fully in Fig. 2. In that scanning system, a light source, generally denoted 34, comprises, for example, a source of "red" light, a source of "green" light, and a source of "blue" light, the spectral content of those sources preferably being chosen so that, combined, they produce a desired color gamut. To provide coherent light to the proximal optic 16, the sources of light in the exemplary preferred embodiment comprise lasers, that is, red laser 36r, green laser 36g, and blue laser 36b. It is preferred that the lasers comprise diode lasers, each having a collimating optic (not shown) at its output as will be understood by a person having ordinary skill in the art (a "POSA".) Thus, laser 36r produces red light beam 38r, laser 36g produces green light beam 38g, and laser 36b produces blue light beam 38b, each beam of which is a substantially monochromatic, collimated beam of light of suitable spectral bandwidth, as will be understood by a POSA. The bandwidth of each of the three light sources, in some embodiments, is preferably very narrow, so as to minimize chromatic aberrations. A consequence of narrow bandwidth is that interference phenomena are possible. A set of laser modulation controllers 38 separately modulates light beams 36r, 36g and 36b, as will be understood by a POSA, thereby imparting color-specific video information to each respective light beam. While amplitude modulation is preferably used, it is to be understood that

^"33-001350 other types of modulation, such as duty cycle or pulse width modulation, may also be used without departing from the principles of the invention.

The three monochromatic beams 38r, 38g and 38b, are preferably combined into a single light beam 40. Many ways to accomplish this are known, but one example is described here in detail. That example uses dichroic mirrors, or filters, that selectively pass some wavelengths of light while reflecting others. Thus, red beam 38r is reflected by a first dichroic mirror 42r; green beam 38g is reflected by a second dichroic mirror 42g that reflects green light passes red light; and blue beam 38b is transmitted by a third dichroic mirror 42b, that reflects red and green light, but passes blue light, thereby combining the three beams into one beam 40, as will be understood by a POSA. A variety of different folding optics or other means for routing light to different locations could be used without departing from the principles of the invention. Combined beam 40 propagates toward a beam splitter 42, which produces a transmitted beam 44 and a reflected beam 46 propagating in different directions. Reflected beam 46 propagates toward a first, foveal scanner 48, while transmitted beam 44 propagates to a beam splitter 50, and then to a second, peripheral scanner 52.

Foveal Scanner The foveal scanner 48 preferably comprises a first reflective mirror 54 and a second reflective mirror 56 that preferably has a larger surface area than mirror 54. Reflected beam 46 reflects off mirror 54 toward mirror 56, where it is reflected toward a beam expander 58. Mirror 54 ( a "tip-tilt" mirror") is supported by an actuator 55 that, in response to a control signal, causes the mirror 54 to tip a selected amount in one dimension and tilt a selected amount in an orthogonal dimension so as to control the angle and position at which beam 46 thereafter strikes mirror 56. Mirror 56 is supported by actuator 58 that operates in a resonant mode so that mirror 56 tips and tilts periodically in two orthogonal dimensions so that the mirror periodically maps a full cone of angles. The resonant mode is used for mirror 56 to maintain scan speed with mirror 54 in view of the larger mass of mirror 56, as will be understood by a POSA. Together, mirror 54 and mirror 56 control the trajectory of beam 46 into the beam expander 60 by controlling the position on mirror 56 and the angle at which

^"33-001350 beam 46 is reflected from mirror 56. Actuators 55 and 58 are tipped and tilted in response to signals from a programmable mirror controller 64.

The beam expander 60 is a well-known optical component, used here in a novel fashion, that may comprise a first input lens 66 having a first focal length and a second, output lens 68 having a second focal length, the lenses being spaced from one another so as to be afocal, that is, the back focal point 70 of lens 66 is the same as the front focal point of lens 68. Consequently, collimated light that enters input lens 66 will exit output lens 68 collimated, but having a larger diameter. The purpose of the beam expander may be taken to be twofold. First, it allows that when beam 46 exits the foveal scanner it has a diameter that substantially matches the diameter over which the eye is diffraction limited, to give high resolution in this example. Diffraction effects caused by the diameter of the beam do not increase the diameter of the focused spot at the retina beyond the normal resolution of approximately 1 minute of arc, such as for those embodiments directed at high perceived resolution. Second, the beam expander enables the mirror to be smaller, and therefore have less mass, than otherwise. This facilitates providing a higher frequency scanner. Beam 46 exits the beam expander 60 at a selected combination of position and angle, that is, with a selected trajectory, that is determined by the positions of mirror 54 and mirror 56 in response to mirror controller 54. In this way, the foveal scanner causes beam 46 to intersect the proximal optic 16 at a selected location and with a selected angle, and thereafter to enter the eye pupil at a selected angle and offset relative to the optical axis 26 of the eye, as will be explained below.

Peripheral Scanner A second, peripheral scanner 72 preferably comprises a tip-tilt reflective mirror 74 supported by an actuator 76, and an individually-activated micro-mirror array 78, both operating under control of respective signals from programmable mirror controller 64. In this case, transmitted beam 44 propagates toward mirror 74 and is reflected thereby in a selected direction toward micro-mirror array 78 so as to intersect, and thereby reflect off a selected one of micro-mirrors 80i_-n of that array. The angle and position at which beam 44 intersects a micro-mirror, together with the attitude of the particular micro-mirror, determines the position and angle at which the beam 44 is reflected from that micro-mirror.

8 33-001350 The position and attitude of the selected micro-mirror determine the location and angle at which beam 44 intersects the proximal optic 16. The beam 44 intersects the proximal optic 16 at the selected location and with the selected angle, and thereafter enter the eye pupil at a selected angle and offset relative to the optical axis 26 of the eye, as will be explained below.

Beam 44 differs from beam 46 as they exit from the projector 20 in that beam 44 may be much smaller than beam 46 and, consequently diverges significantly as it propagates from the projector to the eye 18. This causes the spot size of beam 44 at the retina to paint larger retinal pixels on the peripheral regions of the retina, where high resolution is unnecessary. Concomitantly, this reduces the bandwidth of image data that is needed to modulate beams 44 and 46, and thereby contributes to the practicality of the preferred embodiment described herein. Additional functions that may be added to projector 20 to further treat light beams 44 and 46 include variable focus, auto-focus and alignment, and beam conditioning. To ensure that the foveal light beam 46 and the peripheral light beam 44 intersect the right areas for their selected positions and angles with respect to the optical axis 26 of the eye 18, a system for tracking the rotation of the eye is provided. Many techniques for this are known, but an example is described herein. Other systems for tracking eye rotation may be used without departing from the principles of the invention. Specifically, light from the peripheral beam 44 reflected back from the eye is detected by photodetector 82 in the projector 20 is used to determine the angular position of the eye. That is, some such reflected light will return to the projector along the same path as beam 44, and be propagated from tip-tilt reflective mirror 74 toward beam splitter 50, through which it travels to photodetector 82. The location and size of the pupil can be sensed using peripheral beam 44 in that, when beam 44 illuminates the edge of the pupil, its reflection will cause an intensity difference measured by the photodetector, signifying the edge of the iris or sclera of the eye. By scanning beam 44 so as to cause intersections with the iris at several locations and noting the attitudes of the mirror 74 and the identity and attitude of the selected micro-mirror 80i_-n, the approximate position and orientation of the optical axis 26 of the eye 18 can be computed as well as the diameter of the pupil.

The Proximal Optic

^"33-001350 A preferred embodiment of a personal display system 10 according to the present invention employs a diffractive optic as the proximal optic. In particular, because the display system is preferably able to display multiple colors, a volume hologram may be used as the proximal optic 16. Two advantageous properties of Bragg reflectors relevent to the present explanation, that they are highly sensitive to wavelength and also highly sensitive to angle. The former is advantageous because the Bragg reflector can be made to reflect only the narrow spectral band provided by the light source, remaining substantially transparent for the remainder of the spectrum. This minimally impacts the scene from the real world, which would pass through the proximal optic, substantially unaltered. The advantages of the latter property (angular sensitivity) will be described below.

Referring now to Fig. 4, a preferred embodiment of the proximal optic 16 is a volume hologram 32 constructed so as to have a first plurality of relatively small, peripheral redirection areas 88i_-p that are for redirecting peripheral beam 44 from different locations on the proximal optic 16. A second plurality of relatively large, foveal redirection areas 90i_q are interspersed among the peripheral redirection areas so as to redirect foveal beam 46 from different locations on the proximal optic 16. Each of the redirection areas 88i_-p and 90i_q can be considered a distinct Bragg reflector. Preferably, the foveal redirection areas 88i_-p have a diameter of about 1650 microns, and are spaced about 2300 microns apart, center-to-center, while the peripheral redirection areas 9O_1-^ have a diameter of about 125 microns and are spaced apart about 250 microns, center-to-center.

Among other things, distinct redirection areas are provided for foveal scanning and peripheral scanning. This prevents significant cross talk between them. That is, the system would not perform as well if light from the larger, foveal beam 46 were to illuminate an area that is designed to redirect the peripheral beam 44, as that could send light from the foveal beam into a peripheral region. Likewise, it is undesirable for light from the peripheral beam to illuminate a foveal redirection area, as that could direct peripheral light into the foveal region. In that regard, cross talk is also reduced by the spatial frequency sensitivity of Bragg reflectors. That is, if light from the

"wrong" projector strikes a Bragg reflector, it necessarily does so at the wrong angle, and is therefore not reflected toward the eye. Also, for a given orientation of the eye 18, it is desirable for the peripheral redirection areas to surround the foveal

33-001350 10 redirection area. To accomplish this and also accommodate a plurality of eye orientations, a periodic pattern is desirable. The foveal redirection areas 90i-q are relatively large to accommodate the relatively large diameter foveal beam 46, while the peripheral redirection areas 88-i_p are much smaller because the peripheral beam 44 is much smaller.

Each foveal redirection area 90i-q is oriented to redirect projected foveal beam 46 to the center of the pupil when the eye rotates so that its optical axis 26 is aimed at the center of the foveal redirector.

In practice, each of a subset of foveal redirectors 90i-q illuminates a corresponding "tile" in the foveal region of the eye, each tile comprising a plurality of retinal pixels, and a number of tiles being required to cover the entire foveal region. As can be seen in Figure 3, the foveal beam 46 intersects each selected foveal redirector 90 of the subset at a plurality of angles 91 to write each of the pixels within the corresponding tile, then moves on to another foveal redirector of the subset and so forth. Meanwhile, the peripheral beam 44 scans the various surrounding peripheral redirectors 88i_-pso as to write the much larger pixels in the peripheral region. For example, a peripheral beam may intersect a peripheral redirector at location 93. In this example, The foveal redirection areas yield a retinal spot size of about 15 microns at the retina and cover a 2700 micron region, while the smaller peripheral redirection areas produce a 120 micron spot size at the retinal and cover a 5200 micron region, assuming a minimum 2.7mm eye pupil diameter. Fig. 5 shows the areas of the retina 24 that are illuminated by scanning the redirector pattern shown in Fig. 4.

Fig. 6 shows a simplified embodiment of a hologram printing apparatus 110 producing in a thin film emulsion multiple interference patterns that will each have the effect of a single array of angled mirrors. A laser beam source 112 produces a laser beam 114, which is immediately split into two laser beams 116 and 118. Laser beam 114 is reflected from a first tip-tilt mirror 120, and laser beam 118 is reflected from a second tip-tilt mirror 122 such that when reflected laser beams 116 and 118 are co-incident on thin film emulsion 124, they interfere. The resultant interference pattern is captured in thin film emulsion 124 as a miniaturized holographic image of the two angled mirrors 120 and 122. Although apparatus 110 of Fig. 6 is not part of

11

^"33-001350 personal display system 10, equipment for printing a volume hologram is needed either to generate or to properly modify proximal optic 16.

Enhanced Eyeglasses

Because eyeglasses are ubiquitous they provide a ready platform for a variety of "high-tech" applications beyond passive vision enhancement, eye protection and aesthetics. Enhanced eyeglasses may enable, in addition to improved vision, an "augmented reality," by integrating electronic features such as, for example, video images, voice interfaces, user controls, text communication, video/audio content playback, eye tracking, and meteorological monitoring.

Referring to Figs 8 and 9, an enhanced eyeglasses system 190 includes electronic components 192. Figs. 8A - 8D show examples of the placement of components 192 according to a preferred embodiment. In general, components 192 may be disposed on substrates (not shown) that are covered, laminated, or over- molded. Substrates may be surface-mounted or covered by elements adhered by fasteners, welds, adhesives, or the like. Components 192 may also be mounted directly to structural or aesthetic elements or layers of the frame, for example, by patterning interconnects into the frame itself, as discussed below. Fig. 8A shows a side view of a pair of glasses configured with example components 192 mounted along the length of a substantially straight arm 193 and along an angled earpiece 194. Fig. 8B shows placement of staggered components 195 in a removable, interchangeable arm 196. Fig. 8C indicates a component 192 that may be placed in an auxiliary payload discussed below. Fig. 8D indicates some examples of placing components 192 in a frame front 198 of the eyeglasses frame, surrounding a lens 199.

Referring to Figs 9A - 9C, components 192 provide enhancement features 200 that may be categorized into three functional families: "human interface" features 202 that convey user commands and provide informational feedback to the wearer; "content capture" features 204 that obtain, process, and supply information to the wearer; and "infrastructure" features 206 that include common elements supporting the other parts and functions of the system.

Fig. 9A shows human interface features 202 provided by the following: mono or stereo audio transducers 208 that transmit sound to, or capture sound from the

12 33-001350 wearer, by coupling airborne or environmental sounds with vibrations in the user's bone structure or elsewhere on the user's body; an image projection device 210 provides images including text, still pictures, and video. Tactile and/or proximity interfaces 211 allow the wearer to provide input through touch and proximity gestures. An eye-tracker and blink detector 212 are other examples of user interface. Feedback to the wearer may be through tactile sensation, such as vibration or temperature, and may generally inform the wearer via a silent alert.

Fig. 9B shows "content capture" features include the following: clock functions 214 such as time of day and date, environmental sensors 216 such as temperature, barometric pressure and relative humidity; bio-monitoring including sensing heart rate, blood pressure, etc; light sensors 220 including visible, infrared, and ultraviolet wavelengths; location sensors such as GPS (global positioning system) and inertial sensors that detect head and body movements and gestures for purposes of image adjustment; audio sensors 224, capable of external sound capture through microphones, cancellation of extraneous sound, and sound identification according to direction and spectral density.

Fig. 9C shows infrastructure features 206 including the following: power supply and transmission 226 through either contact-less or contact-based conduits; signal transmission 228 between devices and between eyeglasses system 190 and the outside world, including for example, radio frequency, infrared, or wired connections to support both local and remote signal transmission; a microcontroller 230 including a processor and electronic memory for controlling and managing devices within the system including retaining content, preferences, and status. If eyeglasses system 190 is divided into detachable parts or parts worn separately, connection between the parts includes a "split interface" 232 to detect the presence and configuration of the parts, and to control, via a power switch 234, the supply or automatic interruption of power to system components as necessary, for example when the eyeglasses are removed from the head, or when storage is detected; system monitoring parameters 236 such as temperature, flexure, and security provisions controlling access to operate the system, for example, by verifying characteristics of the wearer.

Fig. 10A shows the right corner of enhanced eyeglass system 190, configured with components 192, according to a preferred embodiment. A hinge 237 connects

13 133-001350 arm 193 with frame front 198 of the eyeglass frame. In this example, components 192 include image projection device 210 together with eye tracker and blink detector 212, mounted in a common enclosure, and placed at an angle to arm 194 so as to aim a projected image toward lens 299. A light beam generated within image projection device 210 propagates along a first trajectory 242, illuminates an area 244 of lens 199, and reflects into the eye of the wearer so that an image is projected onto the retina. Similarly, light reflected from the retina, and from other portions of the eye, is also captured.

Fig. 10B shows a front plan view of frame front 198, in which the illuminated area 244 is indicated by a dashed line. Fig. 10C is a cross-sectional view of lens 199, showing a thin film coating applied to the inner surface. Thin film coating 246 within illuminated area 244 preferably interacts with the projected light, directing the light beam into the pupil of the eye or receiving light reflected from the eye. Thin film 246 is preferably a dichroic coating that reflects a limited range of visible wavelengths. Dichroic coatings, particularly those featuring a narrow "band-pass" design, are advantageous because they limit light reflections from the lens that may interfere with clear vision.

Eye tracker and blink detector 212 captures images of the eye, particularly the iris and the sclera. In order to determine the rotational position of the eye, images are matched with templates from pre-recorded reference images. In a training phase, a user may provide such reference images by smoothly scrolling the eye to display the entire surface. Subsequently, real-time images of the eye may be matched to the reference images to track rotational eye motion.

Referring to Figs. 11 A - 11C, exemplary configurations for motion, proximity, and touch sensing are shown in accordance with a preferred embodiment. Area sensors are indicated, consistent with the use of capacitive type sensors, such as those available from Analog Devices (model AD7142) and the Quantum QT118H, although alternative sensing technologies may be employed. Touch interfaces may be used to sense proximity gestures. For example, the wearer may adjust a brightness or sound level by a sliding gesture along one or the other sidearm or by a rotational gesture around the perimeter of the frame front of the eyeglass frame (as if rotating a knob). Grasping an arm between the thumb and finger(s) turns the sidearm into a makeshift keyboard for individual finger or chord entry. Positioning

14 33-001350 the thumb may act as a "shift" key. In examples where images are provided to the wearer, preferred embodiments indicate the positions of the fingers, preferably distinguishing between proximity and touching. Also, the meaning of locations is preferably shown, whether static, such as for particular controls, or dynamic, such as for selection between several dynamic text options.

Figs 11 A - 11C show sample placements of several area sensors: in Fig. 11 A, frame front sensors 248 are shown embedded in frame front 198; in Fig. 11 B, proximity and "slider" sensors 250 are shown as converging lines embedded in arm 193. Positional sensors 252 are shown as two alternating patterns of strips, that may detect touch positions as well as sliding gestures. Furthermore, in Fig. 11C, the top edge 250 of frame front 198 is shown embedded with additional area sensors 248 and 252. Hinges 237 can be seen connecting the frame front to the earpiece sidearm. Sensors line the edges including the parts shown.

Referring to Figs. 12A - 12C, exemplary configurations for audio transducers 208 are shown in accordance with a preferred embodiment. One example of an audio transducer 208 is a microphone. Another is a bone conduction device that transmits and receives sound waves through bones of the skull. For example, sound becomes audible to the wearer if sound waves are transmitted to the inner ear, and likewise, the wearer's speech may be transmitted by sensing vibrations in the skull. Fig. 12A shows the "bridge" portion 254 of an eyeglass frame that rests on the bridge of the nose of the wearer, thus providing a point of contact that may be used for conducting sound through bone. For example, audio transducers 208 may rest directly on the nose, or they may be configured to conduct sound through other elements, such as pads or a metal bridge. A pair of stereo transducers 208 is shown; however, a single transducer may suffice.

Fig. 12B shows an alternative location for placing a bone conduction transducer is shown. A temple audio output transducer 256 is mounted on the inside end of earpiece 294 so that the transducer contacts the skull substantially behind the ear as shown. Some pressure may be preferably provided for improved sound conduction.

Fig. 12C shows an audio/video pickup transducer 258 embedded in the upper right corner of frame front 198, that detects environmental, ambient sound and captures the wearer's speech. In one embodiment, an alert sound may be

15 - 33-001350 generated, to help locate the eyeglasses, or for ultrasonic ranging. Audio /video pickup transducer 258 may take the form of a video camera or infrared night vision camera, aimed forward, sideways, or even backwards. Audio transducers 208, 256, and 258 may be obtained from Bose Corporation of Framingham, Massachusetts, or from Digi-Key, Inc. of Thief River Falls, Minnesota.

Referring to Figs. 13A - 13D, exemplary configurations for mechanical and signal transmission 228 and power switch 234 functions mounted between arm 193 and frame front 198 are shown in accordance with a preferred embodiment. Figs. 13A and 13B are primarily directed at "on/off" power switching at hinge 237; Figs. 13C - 13D are primarily directed at supplying power through hinges 237. However, the two aspects are related in some examples, in which a slip-coupling includes power switching capability or where switch contacts are used for providing power.

Fig. 13A shows the right corner of the eyeglasses configured with a mechanical button 260 at the junction between arm 193 and frame front 198. Power switch 234 is shown included in frame front 198, disposed adjacent to a mechanical button 260 protruding in a direction in which arm 193 contacts frame front 198 in the open wearable position. When the frame is in use or otherwise open, mechanical button 260 is depressed by the end of arm 193 so that power switch 234 is closed and supplies power to components 192. When the frame is not fully open, however, power switch 234 is open such that power to components 192 is interrupted. In some cases, mechanical button 260 may be spring-loaded, and may comprise one or more contacts between the two components of the frame. Micro-switches such as these are commercially available, for example, a DH Series manufactured by Cherry or the D2SW-P01 H made by Omron Corporation of Schaumberg, Illinois. Fig. 13B shows an alternative power shutoff switch, comprising a "reed switch" 264 and a permanent magnet 266. Such micro-switches are known to be small, for example, those disclosed by Torazawa and Arimain in "Reed Switches Developed Using Micro-machine Technology," Oki Technical Review, p. 76 - 719, April 2005. When the frame is open, magnet 266 is sufficiently close to activate reed switch 264; when the frame is closed, magnet 266 is far enough away and/or oriented such that reed switch 264 closes.

Fig. 13C shows an arrangement allowing wire conductors 268 to pass through a hollow embodiment of hinge 237. The conductors may be completely hidden, such

33-001350 16 as those disclosed for doors by WoIz et al. in U.S. Patent No. 4,140,357. In other examples, wire conductors 268 are in the form of a ribbon cable and may not pass through the hinge.

Fig. 13D shows a plan view of half of frame front 198, including hinges 237, including two exemplary hinge parts, one for each of separate parts of an electrical circuit. The parts are separate hinge components, cooperating to form a strong hinge assembly. Hinges 237 are mounted to substantially insulating material, such as plastic resin from which the frame is formed. Each hinge part forms in effect a so- called slip coupling and, as is known for such couplings, such as disclosed by Gordon in U.S. Patent 3,860,312, can have provisions to interrupt or cut off power in certain ranges of angular positions.

Referring to Figs. 14A - 14B, an external auxiliary device 270 may be connected to enhanced eyeglass system 190 in accordance with a preferred embodiment. Two examples are shown in which the eyeglasses are fully open and viewed from the top. Hinges 237 are visible, joining the temples to the frame front.

Fig. 14A shows a preferably detachable retainer cord 272 that attaches to the end of each arm 293. Retainer cord 272 may be detachable with low force in any direction, such as by a magnetic connector or a clip that provides a circuit contact (not shown). External auxiliary device 270 may take the form of a payload 274, preferably flat, and preferably disposed between the two retainer cords 272. Payload 274 may perform multiple functions including increasing comfort for the wearer, and serving as a decorative accoutrement. In a preferred embodiment, however, payload 274 provides an alternative location for components 292, that support or augment enhancement features 200. For example, payload 274 may contain a power supply or power supply charger, a radio transceiver, an electronic memory, or a connection port through which memory devices or other interface devices can access system 190. Further examples include audio transducers and additional touch panel surfaces, such as those described and shown in Figs. 11 and 12 above. Furthermore, functions that are performed by the payload configured in a tethered mode may also be performed by a wireless embodiment of payload 274, connected by radio frequency, optical, audio, or other mobile communication technologies. Because it need not be physically attached to system 190, a wireless

17

_ _ 33-001350 payload 274 has the advantage of being carried separately, or being attached to a belt buckle, skin patch, mobile phone, handheld computer, wristwatch, or the like. A wearer, as an example, may input selections or other information by gesturing near or touching the payload while receiving visual feedback of these motions through eyeglass system 190 via image projection device 210.

Fig. 14B shows an alternative method of attaching payload 274 to eyeglass system 190 that entails use of a tethered necklace 276. A feed 278 tethers necklace 276 together with payload 274, via a connector 280, preferably detachable. Tethered necklace 276 may itself serve as an antenna. Figs. 15A and 15B show, according to another preferred embodiment, an external auxiliary device 270 that may take the form of an enhanced eyeglasses case 282, containing devices that communicate information or serve as an inductive power charger for eyeglass system 190 when stored in eyeglasses case 282. External auxiliary device 270 may alternatively take a similar form of a stand or charging station, instead of a case. Such a power charger may entail use of sets of spatially overlapping coils 284 in close proximity to each other. Other suitable power and communication coupling means may be used such as, for example, capacitive and optical coupling. Power and communication components in the case or stand shown may perform docking synchronization and data transfer functions between the glasses and the outside world, such as, for example, downloading/uploading content, and updating clock settings.

Fig. 15A shows four coils 284 within the enhanced eyeglasses case 282 or storage stand. A single coil disposed within each of the eyeglass frame front 298 and eyeglasses case 282 is sufficient to couple the frame front 198 to case 282 so long as eyeglasses case 282 prescribes a certain orientation for the eyeglasses. If eyeglasses case 282 allows all four orientations (upside down and flipped left-to- right), and if the glasses contain two eye coils 286 (one facing forward and the other facing backward when folded), the eyeglass system can always be charged. If eyeglasses case 282 contains four copies of one type of coil 284 (two in the bottom half of case 282, as shown, and two in the top half of case 282, similarly oriented), and if the glasses contain at least one such coil, any orientation of the eyeglasses with respect to case 282 allows coupling.

18

"33-001350 Fig. 15B shows coils located within earpiece 194 or around the perimeter of the glasses, or both. Coils 284 are preferably formed by printing, etching, or winding, on substrates or layers within the frame or on its surface or on detachable or permanently affixed modules. Fig. 16A shows how coil 284 may be used to transfer power and high-speed data between external devices and eyeglass system 190, according to inductive coupling techniques disclosed by K. Chandrasekar et al. in "Inductively Coupled Board-to-Board Connectors," Electronic Components and Technology Conference, 2005. Coil 284 may be patterned by etching away conductive areas on a substrate. Alternatively, capacitive coupling may also be used.

Fig. 16B shows a first coil 184a, connected to eyeglass system 190 via an earpiece conducting wire 288, embedded near the free end 190 of earpiece 194. Example methods of fabricating such a structure include first, patterning the coil structure itself, and then adhering, or laminating coil 284a onto the surface of earpiece 194. Alternatively, coil 284a may be embedded during the manufacture of earpiece 194.

Figs. 16C and 16D show, an end boot 192 of a mating lanyard 194, overlapping free end 190 of earpiece 194. End boot 192, which is preferably deformable, contains a second coil 184b, connected to lanyard 194 by a lanyard conducting wire 195, which, when end boot 192 is mated with earpiece 194, couples to coil 184a within earpiece 194. End boot 192 is held in place by the elasticity of materials from which mating lanyard 194 and earpiece 194 are made. Fig. 16D shows a cross-sectional view through a coupling boot such as end boot 192 surrounding earpiece 194, along a dashed cut line 196 shown in Fig. 16C. Two sets of windings 198, corresponding to the two superimposed coils 284a and 284b, are shown in Fig. 16D. Earpiece 194 is shown surrounded by lanyard end boot 192, thereby positioning the two coils 284a and 284b in close proximity to each other. Referring to Figs. 17A - 17C, detachable accessories may be provided in accordance with a preferred embodiment. Although a wide range of configurations and styles of accessories are possible, some examples are illustrated in the form of a generic arm attachment 200. Adhesives, fasteners, clamps and the like for securing arm attachment 200 are omitted for clarity. Attachment 200 preferably includes galvanic, inductive, or capacitive coupling for power and data transfer.

33-001350 19 Fig. 17A shows arm attachment 200 attached to arm 193 but preferably not attached to frame front 198, although attachment to frame front 198 is not prohibited. Referring to Fig. 17B, arm attachment 200 is affixed to arm 193 by fasteners such as, for example, snaps, magnets, hooks or loops. Fig. 17B shows female fasteners 202 mounted to arm 193 and corresponding male fasteners 204 mounted to arm attachment 200. The attachment as shown in Fig. 17C is upside down so as to expose the fasteners. Also, shown in the example is a representative example of a component 192 such as a camera or light source, as described above. Fig. 17C also shows an edge-on view of an additional attachment means that fits over and clips onto arm 193 lengthwise.

Referring to Figs. 18A - 18C, two examples of detachable arm configurations are shown in accordance with a preferred embodiment. A plain detachable arm 206 is shown in Fig. 18A. Plain detachable arm 206 is distinguished from arm 193 by a detachable hinge 208. A one-piece detachable arm configuration that combines an arm 193 with an arm attachment 200 into a detachable accessory arm 210, also configured with a detachable hinge 208, is shown in Fig. 18B accompanied by the frame front 198 to which both types detachable arms 206 and 210 attach. Detachable arms 206 and 210 are advantageous for swapping out accessories with different functions, as well as allowing for repairs, thereby improving reliability of the overall eyeglass system. Detachable accessory arm 210 is shown configured with an example accessory component 192 (e.g., an image projection device 110). Detachable hinge 208 includes two hinge "knuckles" 212 mounted on frame front 198, and three hinge clips 214 within each of detachable arms 206 and 210. Fig. 18C shows a detailed side view of a preferred structure for hinge 208, in which hinge "knuckles" 212 preferably include an electrically insulating (e.g., dielectric ) spacer 216, shown in Fig. 18C as a thin white vertical line that ensures electrical power and signal connections may be safely routed through detachable hinge 208 without risk of a short circuit. When hinge clips 214 and hinge knuckles 212 are interdigitated and a spring force is applied, they snap together, attaching arm 206 or 210 to front face 198, thereby forming hinge 208 as shown in Fig. 18C. A detent, such as a ball mating in a curved cavity, omitted for clarity, snaps the two together.

The foregoing described embodiments depict different components contained within, or connected with, different other components. It is to be understood that

20 " 33-001350 such depicted architectures are merely exemplary, and that in fact many other architectures can be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being "operably connected", or "operably coupled", to each other to achieve the desired functionality. While particular embodiments of the present invention have been shown and described, it will be obvious to those skilled in the art that, based upon the teachings herein, changes and modifications may be made without departing from this invention and its broader aspects and, therefore, the appended claims are to encompass within their scope all such changes and modifications as are within the true spirit and scope of this invention. Furthermore, it is to be understood that the invention is solely defined by the appended claims. It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but is not limited to," etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., "a" and/or "an" should typically be interpreted to mean "at least one" or "one or more"); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if

21

33-001350 a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, typically means at least two recitations, or two or more recitations).

SUPPLEMENTAL DISCLOSURE

The construction of visual images via reflection or diffraction of light from what will herein be called a "proximal screen" or simply a "screen" substantially placed in close proximity to one or both of a person's eyes is of significant commercial interest. Such images may be employed in viewing what will herein be called "constructed" images, whether static or dynamic, such as those comprising movies, video games, output from cameras, so-called "heads up display" application content, text of all types, notifications, and so on. If a proximal screen is at least partially transparent, the constructed image may be viewed in "superposition" with, or also herein "combined" with, what will herein be referred to as the "actual scene" or "scene" image. Such an actual scene is what is typically transmitted through the so-called "lenses" of eyeglasses and originates from the viewer's physical surroundings. A proximal screen may, in just one example, be realized in the form of one or both the lenses of a pair of eyeglasses and result in perception of the combination of the constructed and scene images. Although a proximal screen may in such embodiments have size substantially the same as a traditional eyeglass lens, its morphology and manufacture may differ. Accordingly, in one exemplary aspect, it is an object of the present invention to construct images that conveniently allow users to read a limited amount of text, such as is used today in so-called text messaging and instant messaging. Other examples of text include time, date, temperature, appointments, incoming message alerts, emails, posts, web content, articles, poems, and books. In some settings such text may be desired to be read without conspicuous action by the user and/or without substantially interfering with the view related to other activities. Activation and/or control of a text view may even be, for instance, in some examples, by the direction gaze.

22 33-001350 In another exemplary aspect, it is an object of the present invention to construct video images that conform to available video content. In some examples video content is available in formats such as NTSC-DVD 720x480 (PAR≠1), PAL- DVD-16:9e 720x576 (PAR≠1), HD-720i/p 1,280x720, HD-1080i/p 1920x1080, DCI- 2k 2048x1080, DCI-4k 4096x2160, and UHDV 7,680x4,320. In some cases these formats include separate left and right eye views. So-called "field of view," often expressed as an angle subtended by the "virtual" screen, whether from side-to-side or along the diagonal, is sometimes regarded as a figure of merit in such viewing systems. Being able to render such content in a way that is appreciated by the viewer, such as where the effective screen position and size can be adjusted, is also believed desirable. Moreover, a wide color gamut and reduced artifact perception are also believed desirable.

In yet another exemplary aspect, it is an object of the present invention to accept and provide constructed images responsive to real-time image streams, including video cameras and other sensor systems. In some examples, such sensors are arranged to correspond to the user's field of view and even point of regard. Various wavelengths of electromagnetic energy for such sensors are anticipated, such as ultraviolet, visible, infrared, and so forth. In some examples constructed images are arranged so as to be combined with the actual scene and augment it for enhanced or altered viewing by the viewer.

In still another exemplary aspect, it is an object of the present invention to construct images aimed at providing rich interactive environments generally, for instance such as engaging electronic gaming. Image content is generated by the gaming system, whether local and/or remote. However, it may in some examples be responsive to the actual scene observed by the player, such as captured by cameras or sensors.

In a further exemplary aspect, it is an object of the present invention to construct images that are perceived to be substantially similar to actual scenes viewed by the user. In some example embodiments, an observer using the present invention may experience constructed images that are substantially indistinguishable from an actual scene viewed through traditional eyeglasses.

In yet a further exemplary aspect, the user's point of regard and/or the amount of focus of the user eye and/or the dilation of the pupil may be input to the system

23 ' ""'"133-001350 and are measured from time to time or continuously. They are used, for example, in formation of images, construction of images, rendering of images, in the present inventive systems and more generally are optionally stored and/or supplied to automated systems. In still yet a further aspect, in order to enhance the perception of the constructed image, in some embodiments, it may be corrected substantially in accordance with characteristics of the viewer's eyes.

The forgoing as well as practical, low-cost, efficient, robust, and user-friendly image construction systems are among the objects of the invention. Other objects, features and advantages will be more fully appreciated when the present specification is read in conjunction with the drawing figures and any related co- pending applications by the present applicant(s) that are hereby included in their entirety. BACKGROUND

Accordingly, an understanding will be appreciated of the manner in which light is incident from an actual scene onto a transparent screen proximal to the eye. This will allow for more ready comprehension of the novel image construction systems disclosed here, which in some embodiments substantially attempt to duplicate the wavefront structures that would be produced at the proximal screen by actual scenes. As will be disclosed, not all aspects of an actual scene's wavefront structure may be necessary in order to construct an image that substantially re-creates a viewer's perception of the scene. As one example, it is believed that, in an inventive aspect to be described in more detail, the proximate screen need not be continuous. Turning now to Figure 101 A, rays from specific actual scene points are shown passing through a visual input plane located close to an eye substantially where the proximal screen may be located in some embodiments. Such actual scene points will for clarity, as will be appreciated, be considered as "pixels" or "scene pixels" here. The set of scene pixels, as will be understood, are considered for clarity to cover the scene so as to provide the effective view of it. The distance from the input plane to the eye will be considered, for concreteness in exposition, to be on the order of an inch, as is believed typical with eyeglasses.

24

^"33-001350 Figure 101 A shows two points on the input plane, denoted X₁ and X₂. Also shown are rays incident from an actual scene. Rays A, B, and C from point locations in the actual scene, substantially scene pixels, are commonly labeled for each of the points xi and X₂. It will be understood that rays from each scene pixel pass through each of the two points on the input plane, as is believed substantially the case for all points on an unobstructed input plane. The rays of the commonly-labeled points are shown as parallel, in accordance with the example actual scene being distant. Rays from a single pixel of a close actual scene, such as those comprised of objects relatively close to the observer would, as is known, not be parallel. In such cases of close objects, the eye is believed to typically adjust its power so as to focus the non- parallel rays onto the retina. The eye accommodates for closer object distances until the input wavefront is so curved (rays so angled) that the eye cannot focus them. While individual eyes have differing powers, a typical rule of thumb is that rays from objects closer than about 25 cm cannot be focused by most adults of certain approximate age and the object distance of 25 cm is typically referred to as the "near point." The image forming method and device disclosed will be able to present input rays of various degrees of parallelism and therefore construct scenes having various perceived distances from the eye.

Figure 101 C shows how, for those rays passing through a particular point on the input plane, only those falling within a certain solid angle Δθ_p actually enter the pupil (even though rays from many more widely positioned scene pixels may pass through that point on the input plane). The solid angle of rays from the scene that is captured by the eye from each point on the input plane typically varies from point-to- point on the input plane as will be appreciated from the depicted geometry and formula provided in the figure. It should particularly be appreciated that each point on the input plane actually supplies light for what will herein be called a "set" of pixels in the actual scene. Adjacent points in the input plane provide illumination for partially-overlapping sets of such scene pixels. The degree of overlap decreases as the points become further apart. For large enough lateral displacements along the screen, points on the screen source disjoint sets of pixels to the retina. Since light reaching the retina from each scene pixel passes through the input plane at multiple points, it will be understood that some of those points can be obstructed without substantially eliminating the pixel's optical input to the eye. It will also be

25 r,.._^- .. „<,.,».>•>»..* «"««33.001350 appreciated that, although Figure 101 is drawn in a plane for clarity, the concepts introduced are readily translated to the full three-dimensional case and to a proximate screen of a planar or curved form. Accordingly, but provided that the optical power from the various pixels is reasonably balanced, it is believed that obscuration of portions of the input plane, and hence substantially of a proximate screen, may not lead to substantial and/or uncorrectable perceived image degradation.

It will also be appreciated that if a region on the input plane were to receive and transmit a substantially collimated input beam from the scene, changes in the angle of that beam would in effect steer the output beam leaving the plane to varying points on the retina.

Figure 101 D provides an indication of how wide a region on the input plane is required to accept all of the light from the scene entrant to the eye from the angular range Δθ_p. This may be relevant for some embodiments, since each proximal screen point would supply light from some range of constructed input solid angles Δθ_p. In order to capture all rays from scene pixels that are entrant to the retina from a single point on the input plane it is believed that a region is needed whose diameter is approximately twice the diameter D of the pupil.

Figure 101 E is aimed at explicating how far points on the input plane need be laterally separated before the set of actual scene pixels that they effectively source becomes disjoint. Minimally separated input plane points having disjoint pixel sets are shown. When the points are separated by approximately the pupil diameter, the set of retinal pixels that they provide light to become disjoint. The distribution of input plane points is shown for clarity in two dimensions, but may be extended into three dimensions in some examples as an array of points covering the proximal screen with separations of approximately D to provide for sourcing all desired actual scene pixels.

Although, an array of screen source points with approximate spacing of D can in principle provide retinal illumination for all scene pixels, in constructing a practical device, screen source points in some examples may usefully be separated by somewhat less than D, for instance 0.5D to 0.9D. One example reason for such reduced spacing in some embodiments is believed to be in order to provide redundancy for pixels at the edge of each spot's coverage area since for such pixels

26 133-001350 some clipping of source beams at the edge of the pupil may occur, reduce retinal spot size, and available pixel power. Power lost to pupil clipping can, however, be compensated for in a modulation device in accordance with some aspects of the invention that controls power sent to the various pixels. It is accordingly believed advantageous in some embodiments to configure the screen source spot array with spacing less than D so as to reduce the clipping effect.

Turning to Figure 104, a two-dimensional array of spots on the input plane is shown arranged on a grid of spacing D-ε. The spot width is a. If ε=0, the spots provide light to largely disjoint sets of retinal pixels. As ε increases compared to D, each spot would need supply only a portion of its accessible retinal pixels since the pixel sets served by adjacent spots increasingly overlap.

Referring to Figure 101 F, a beam emanating from one source spot near the extreme of angles (pixels) sourced is shown in dashed lines. For such extreme angles, the source beam may be partially clipped by the pupil and be diminished in power relative to the case when the source beam addresses more centrally located pixels as already described with reference to Figure 101 E.

Referring to Figure 101 G, the power reaching retinal pixels from a single input plane spot as a function of angle relative to the central angle Q₁₊₂ is depicted schematically. The power falloff at the edges of the spot's angular sourcing region is believed to occur substantially because of beam clipping by the pupil. Suitable values of ε and a it is believed can be employed to reduce this effect, as can dynamic power compensation on the part of the modulation means used to source light to the various pixels as mentioned and will be described further.

ADDRESSING A SINGLE PIXEL

It is believed, at least to a first approximation and for distant images, that the eye substantially maps rays of the same propagation direction onto a single point on the retina. For scenes that are near, the eye maps "families" or beams of light rays appearing to emanate from a scene pixel onto a single retinal pixel. Owing to the diffractive nature of light, there is an inverse relationship between the size of a scene pixel and the minimal divergence of the rays emanating from it. As a result, it is believed preferable to use substantially finite areas on the proximate screen to

27

-.,.^- *~~ . _"133-001350 launch beams that will illuminate small spatial regions on the retina. The constraints of image formation, as will be appreciated, may be related to the relationship between spot size on the proximate screen and spot size on the retina. The smaller the retinal spot, it is generally believed, the higher resolution that can be perceived, at least until the eye's intrinsic resolution limits are reached.

In Figure 101 D, a schematic depiction is provided of a light beam directed from the proximate screen into the eye that represents a single pixel of a distant scene. The beam occupies a spot of diameter a on a proximate screen, distance d from the eye having diameter d', passes through the pupil, which is of diameter D, and is focused onto the retina, resulting in a spot size of a'.

Turning now to Figure 102, the approximate retinal spot size is plotted as a function of proximal screen spot size, for d=d-25 mm and for light of substantially mid-visible wavelength. Aberrations from the eye are neglected, in accordance with the teachings of the present invention to be described further, as they can be compensated for by suitable conditioning of the light beam launched from the proximate screen. In the calculation of the plot, the light leaving the proximate screen is assumed "diffraction limited" in that its divergence is set to the diffractive minimum. As will be appreciated from the plot, once the source size on the proximate screen drops much below the pupil diameter, the retinal spot size increases rapidly and results in fewer discrete pixel spots on the retina.

PIXEL-BY-PIXEL IMAGE PAINTING

The present invention includes, as will be appreciated, embodiments and their combinations directed at constructing a retinal image via reflection or diffraction from a proximate screen.

By controlling the angle with which a light beam from a specific finite diameter spot of the proximate screen travels towards the eye, it is believed that control is achieved over which one of many retinal pixels is addressed. Modulation of said light beam as its pupil-incident angle changes provides for the differential illumination of each pixel, such as in grayscale, binary, monochrome, multiple separate colors, or various combined color gamuts. The number of pixels in each dimension, as well as in some examples even the aspect ratio of pixels, may vary as mentioned earlier with

28 133-001350 reference to legacy formats; one related advantage of pixel by pixel imaging will be understood to be the ability of a single device, within the range of its capabilities, to construct pixels of different aspect ratios and sizes as desired at different times. It is also believed as has been mentioned that only some areas of the proximate screen need be used for particular pixels. For a single proximate screen area, scene pixels within a solid angle of approximately Δθ_p (as has been described already with reference to Figure 101 B) may be written on the retina.

(If, for instance, the pupil and proximate screen spot are both taken as 2 mm in diameter and the eye is looking straight ahead at the proximate screen 25 mm in front, the angular diameter of the vision cone that can be supplied by the spot is believed to be somewhat less than about 6 degrees.)

Roughly 250 by 250 pixels of resolution are believed available in such an example angle with a retinal spot size of 15 microns. One example way to address this array of pixels is to vary, according to a raster or other pattern, the direction from which light is incident on the proximate screen. Variation of the incident angle provides, via the law of reflection or principles of diffraction (such as if a grating surface is employed), the needed variation in the angle of the light beam propagating from the proximate screen toward the eye. It is not necessary, however, that the spot on the proximate screen be located directly in front of the eye. It is believed that, at least for some orientations of the eye, the spot may be located anywhere on the proximate screen so long as the light reflected or diffracted from it can be aimed to enter the pupil.

Turning now to Figure 103, an optical delivery system is schematically shown that allows for the pixel-by-pixel writing of a scene, such as having the approximate characteristics described in the preceding paragraph. The light source 3101 provides optical power and optical modulation. It may for example be monochromatic, entail successive writing of multiple colors such as three "primary colors," or combine various frequencies of light at the same time. In some examples it may, for instance, provide diffraction-limited light or spatially filtered light preferably with similar divergence properties. In the case of three successive colors, for instance, individual pixels are scanned three successive times and each time at a corresponding power level, so as to create the perception of full color as would be understood.

29

— - , ₁33-001350 Optional lens 3105, or another suitable optical element such as a curved mirror or diffractive lens in the optical path, alters the optical wavefront as needed to provide the wavefront curvature desired as it enters the eye. Adjustment of lens 3105, such as by varying its position or effective curvature as is know for variable focus optical elements, can result in images of different apparent distances from the eye. In some exemplary embodiments focus is controlled to provide that the combined actual scene transmitted through the proximate screen and the constructed images reflected or diffracted from the screen have the same apparent distance and are superimposed so as to be simultaneously in focus. The two example moveable mirrors comprising the exemplary mirror system shown for clarity in two dimensions are preferably moved in cooperation with each other. Mirror 3109, for instance, displaces the optical signal beam on mirror 3113, while mirror 3113is rotated so as to keep the optical beam at substantially the same spot 3117 on the proximate screen. The spot size on the mirrors and the proximate screen may be set to be similar to the pupil size so as to provide the substantially high resolution described earlier (or assuming a minimum pupil size). In an exemplary three-dimensional embodiment and mirror arrangement, as will be readily understood, both mirrors will optimally provide angular rotation about two axes, for example, about the horizontal and the vertical. Referring now to Figure 103A, the rays incident on and emergent from the proximate screen spot in appear not to obey the law of reflection relative to the proximate screen surface 3135, which can be realized in a number of ways, some of which will be described as examples. The proximate screen differs from the input plane already described in that it is of a substantial thickness and, in some exemplary embodiments, is in effect formed by a method comprising two steps. In the first step, a slanted mirror surface 3125, oriented to connect input and output rays via the law of reflection, is produced. This surface is partially reflective, such as resulting from a metal coating, for instance aluminum, or a dielectric stack. The reflective coating or layer, or another material of substantially similar transmisivitiy, preferably spans gaps between spots on the proximate screen so that transmitted images from the actual scene remain substantially uniform, albeit somewhat dimmer at least for some wavelengths. In the second fabrication step, a second layer of material is combined having a substantially smooth exterior surface 3135. The

30 33-001350 smoothness of this surface is intended to provide that transmitted scene images are substantially undistorted.

Figure 103B shows schematically another illustrative example of a proximate screen that includes diffractive structures in keeping with the teachings of the invention. In the example these structures are formed on operative surfaces.

The input and output angles to the proximate screen shown in Figure 103A may be realized in other examples by forming a diffractive structure in or on the proximate screen. The angular-relationship between input and output rays relative to a diffractive are governed by the grating equation suitably applied as known in the art. Suitable choice of the grating period and orientation are believed to allow, as will be appreciated, a substantially wide range of input-output beam configurations. Should a diffractive structure for instance be formed on the inner surface of the proximate screen, the input beam may require shaping so that the beam diffracted toward the eye is roughly circular in settings where such substantial circularity or other shape is desired.

Diffractive structures are selected to provide that only one diffractive output order enters the eye, at least in some exemplary preferred embodiments, as will be understood by those of skill in the art. The properties of the diffractive are chosen to provide the input-output beam angular-orientation desired. With the diffractive geometry shown in Figure 103B, where the diffractive surface 3225 is nearly normal to the input signal beam, advantageous inventive "angle change amplification" is believed obtained. Owing to the properties of diffractives, when the incidence angle of a beam approaching in the vicinity of the normal is changed by Δθ_in, the angular change, Δθ_out, of an output beam oriented far from the normal will change by more than the change of the input beam. In particular, it is believed that Δθ_out /Δθj_n = cosθj_n /cosθ_out, where θ_in and θ_out are the respective angles of the input and output beams relative to the diffractive normal. In exemplary embodiments anticipated here, this is believed to mean for instance that the angular range requirement for the input mirrors may be reduced while still providing means to reach all pixels addressable from a given proximal screen location. Diffractive mechanisms are generally known to be dispersive and accordingly have output beam angle that depends on color. In some examples, different frequencies are sourced sequentially, as has been mentioned, and the

31 33-001350 same retinal pixel may be addressed by the same screen spot for all the frequencies. In other examples, the same retinal pixel may be addressed by different spots on the screen owing to the different angles corresponding to different colors.

Turning to Figure 103C, yet another illustrative example in keeping with the teachings of the invention includes delivery to the eye of the image information via the proximate screen surface. Exemplary delivery spot 3317 is positioned so that the law of reflection provides the needed input-output beam configuration. The constructed image is observed substantially when the eye is positioned in a particular orientation as shown. Diffractive structures, and such structures formed on dichroic coatings so as to be responsive to limited frequency bands, are anticipated generally here and are another example for use in such embodiments. It will be appreciated that the proximate screen may not be transparent and may in some examples be substantially a part or attached to a part of the frame of a pair of eyeglasses. In the forgoing exposition, it has been assumed for clarity that the steering mirrors acted so as to change the angle of the light beam as it enters the eye, but keep its intersection with the proximate screen fixed. Alternatively, the steering mirrors may act to translate the proximate screen spot while at the same time controlling the angle of the beam as it enters the eye. There is an advantage to the latter procedure since then the optical signal beam may remain centered on the pupil rather than moving toward the pupil side (see Figure 103A) and potentially experiencing a clipping on the pupil side. The clipping effect may result in a lower power delivery to outlying pixels as well as some diffractive blurring.

The proximate screen can, as will be appreciated, generally for instance be flat or curved. On transmission, it may have zero optical power or any net optical power or powers as commonly desired to provide the user good images from the transmitted scene during use, as is well known in the eyeglasses art. The flat internal reflector referred to in the above paragraphs may also be curved to provide optical power for the reflected signal and/or to enlarge the spot size on the retina, although it is believed preferable in at least some examples to provide any needed optical correction in lens 3105 or via additional wavefront shaping optical elements. As the internal surface of the proximate screen also affects the wavefront of the

32

^"133-001350 signals reflected or diffracted from internal faces, that surface can be employed to control the wavefront of the beams writing the constructed images.

Turning now to Figure 104, depicted schematically is an exemplary grid of spots on the proximate screen in keeping with spirit of the invention. For clarity in exposition and ease of understanding, the example spots are square and in a rectangular array; however, it will be understood that any suitable pattern of spots may be used and that other patterns may offer advantages, such as more efficient packing or less regular structure. The spots have side a, grid spacing D-ε, and inactive zones of width g surrounding each. Spot 405, for example, controls a certain angular range of pixels contributing to an image. In some embodiments, such a single spot may provide the number of pixels and angular range sufficient for the intended display function; for example, when a substantially small text construction is visible from a particular eye position. More generally, more spots may be added to provide wider angular ranges over which images can be viewed. Exemplary values of a and ε, in some embodiments, are believed on the order of D/2 and 0.2D, respectively, though not limited to such values.

Turning now to Figure 105, another exemplary array 501 of proximate screen spots is shown schematically. Also shown are corresponding regions 513 of the retina. Each proximate screen spot correlates to a specific retinal region. Pixels within a given retinal region are addressed by the angle of the beam incident on the corresponding spot on the proximate screen. After completing a scan of the pixels in one retinal region, in some examples, the mirrors may be adjusted to access another screen spot and thereby access the pixels in its corresponding retinal region, and so on. In other examples, however, the scan pattern includes partial filling of spots to create an effect, related to so-called "interleaving" or the multiple images per frame in motion picture projection, that allows at least some users to better experience lower true frame rates.

The incidence angles used to scan a retinal region's pixels will preferably be correlated with the rotational position of the eye so that, for a particular eye rotational position, pixels associated with a particular screen spot will in fact enter the eye. The retinal region correlated with a given screen spot will substantially change depending on the eye's orientation. In some example embodiments, display control mechanisms may take such remapping into account when assigning which scene

133-001350 33 pixels to route to the various screen spots. Also, in some examples, as the head moves relative to a scene or constructed scene, the constructed content may be shifted so as to create the illusion of a scene fixed relative to the environment.

MULTI-PIXEL DISPLAY

In some settings, it may be desirable to display multiple pixels simultaneously rather than a single pixel sequentially as described earlier.

Turning to Figure 106, an exemplary means for displaying multiple pixels simultaneously is depicted schematically in accordance with the teachings of the present invention.

The upper portion of the figure shows the central rays; the lower portion of the figure shows the corresponding pixel beam including marginal rays.

Starting from the left, light source 603 is depicted along with its cone of coverage. The light source may, for instance be a laser, LED, Vixel, or whatever source of light, preferably of sufficient power to cause the eye to see pixels simultaneously originated in the exemplary transmissive two-dimensional pixel modulator 620. The central rays can be seen starting from pixel sources 601 that are part of the spatial light modulator 620. Spatial modulator 620 may comprise a one or two-dimensional array of modulation devices wherein each separate spot acts to control the amount of light incident from source 603 that passes through modulator 620. Other example spatial light modulator schemes are also anticipated, such as for example reflective, e.g. so-called LCOS, or emissive, such as so-called OLED or other self-illuminated image forming devices, obviating the need for a separate source as would be readily understood. Also, combinations of sources and modulators, such as a row or other configuration of sources, are also anticipated as will be understood.

Following source 603 is a lens 605, which images the source onto a following lens 607. Since the rays from source 603 pass through the center of lens 607, they are believed not substantially deflected by it. The light emerging from source pixels of 601 may be mutually coherent or incoherent. Lens 607 is configures to create a image, reduced in the arrangement shown, of the pixel source 601 at location 611, substantially directly in front of lens 609. Lens 609 acts to collimate central pixel

34

^„..- 33-001350 rays that passed substantially undeflected through the center of lens 607. Since lens 609 and image 611 are substantially co-located, lens 609 creates a virtual image for lens 613 that is co-spatial with image 611. Lens 613 is placed a focal length from image 611 and lens 609, thereby creating a virtual image of the pixel source at negative infinity to be viewed by the eye's lens 615. Lens 613 also acts to focus the prior collimated pixel central rays substantially to a common spot preferably at or just prior to entry into the eye. The light from the pixel source will typically reflect or diffract from the proximate screen as described above between lens 613 and entry into the eye substantially at 615 and impinges on retina 617. The convergence angle of the central pixel rays established by lens 613 determines the separation of pixels on the retina and therefore the apparent size of the pixel array. Spacings between various components are given above and connecting equations are given in the middle of the figure, as will be appreciated.

The bottom portion of Figure 106 depicts the corresponding evolution of a pixel beam including marginal rays. The divergence of the pixel beam emergent from the spatial modulator 620 is preferably set substantially by either diffraction through the pixel aperture or the angular size of the source convolved with the pixel aperture — whichever is larger. Preferably, this emergent divergence is configured so that the pixel beam entering the pupil 615 is comparable to the pupil size. On emergence from spatial modulator 620, the pixel beam diverges until reaching lens 607 and then converges to form image 611. Being substantially in immediate proximity to lens 609, the pixel beam passing through image 611 diverges through lens 609 and is subsequently collimated by lens 613. The pixel beam remains collimated until reaching the pupil where the eye's focusing power acts to focus the beam onto the retina in a small spot. This spot is believed at least potentially near diffraction limited when, as already mentioned, the pixel beam is substantially comparable in size to the pupil prior to entry. Exemplary ways to control the exact wavefront of the pixel beam prior to eye entry, so as to bring the projected image into focus simultaneously with the actual scene transmitted by the proximate screen, include adjustment to the positions or effective power of the various lenses or other optical elements performing similar functions. As mentioned earlier, the pixel source light may conveniently be reflected or diffracted from the proximate screen between lens 613 and entry into the eye at 615.

33-001350 35 The apparatus of Figure 106 will provide for the simultaneous illumination of all pixels within a certain retinal region. For example, such means are optionally applied to provide a low-resolution display, such as text readout, that is viewable substantially only for a particular limited viewing direction of the eye. Turning to Figure 107, depicted is a multi-pixel viewing mechanism similar to that of Figure 106 except substantially that the multi-pixel source emits over a wider solid angle so that the light incident on lens 713 from each pixel nearly fills or in fact overfills the lens aperture. In this case, the eye may rotate throughout a spatial region whose size is comparable in lateral dimensions to that of lens 713. Such a situation is believed useful for allowing the viewer to naturally adjust the so-called "point of regard" so as to take advantage of the retina's regions of high acuity.

Turning finally to Figure 108, shown is an exemplary image forming mechanism similar to that shown in Figure 106 and similar to that of Figure 107, except for movable mirrors 803 and 805, is depicted in accordance with an aspect of the invention. The proximate screen (not shown for clarity in the schematic views of Figure 106 through Figure 107) deflects the light transmitted by lens 813, following mirror 803, so as to enter the eye 832. In some preferred exemplary configurations, lens 813 and the preceding image generation and handling elements will be mounted on the side of the head providing for light transmitted through lens 813 to hit the proximate screen and reflect or diffract into the eye at a certain rotational position or range of positions. The mirrors shown in Figure 108 allow for the concatenation of multiple separate multi-pixel images to form a stitched image having a higher pixel count than conveniently generated via the multi-pixel image generator 801. A pixel array image 822 is again formed. The mirrors act to shift the angular orientation of the pixel beams' central rays while at the same time applying a spatial shift to provide for continued transmission through lens 813 and illumination of the pupil when oriented to view the corresponding image. The angular shift introduced by the mirrors is typically applied discretely and configured so as to provide angular steps approximately equal to the full angular spread of pixel beam central rays so that after application of an angular shift image pixels fall onto the retinal surface immediately adjacent to a retinal region illuminated for a different mirror setting. While it may be typical to position successive multi-pixel display regions contiguously, optionally in some embodiments

36 33-001350 non-contiguous pixel placement is provided, wherein the eye will perceive image-free regions between multi-pixel display images.

Eyeglasses including prescription glasses and sunglasses are already worn by a large fraction of the population and they can provide a platform for a variety of applications beyond passive vision enhancement, eye protection and aesthetics. For instance, enhanced eyeglasses are anticipated that offer improved vision, integration of features requiring separate devices today, and inclusion of capabilities not currently available. Examples include provision of video images, voice interfaces, user controls, text communication, video/audio content playback, eye tracking, monitoring of various meteorological and biological parameters, and so forth.

The present invention is accordingly directed at such enhancements and related features. Additional objectives are practical, efficient, economical, compact, user-friendly, convenient embodiments, as will be more fully appreciated when the remainder of the specification is read in conjunction with the drawing figures. Turning now to Figure 109, a detailed exemplary overall block and functional diagram is shown in accordance with the teachings of the present invention. Exemplary parts of the eyeglasses system disclosed are shown in an exemplary division into three general functional families: "infrastructure," including those common elements supporting the other parts and functions; "human interface," those interfaces substantially aimed at providing information and feedback to the wearer and obtaining instructions and feedback from the wearer; and "content capture," those components and systems directed at obtaining or developing information that can be supplied to the wearer. As will be appreciated, in some examples components or functions cross the boundaries between the families and other elements not shown for clarity may be included broadly within the families.

Referring to Figure 109A, what will here be called "infrastructure" is shown comprised of several components. The device in some embodiments comprises its own what will be called "power source," whether electrical or otherwise, is typically stored in portable devices and/or supplied, for purposes of "charging" or operation, through contact-less or contact-based conduits, as will be described later in detail. Batteries, charging circuits, power management, power supplies and power conversion, comprising typical examples, are all widely known in the electrical engineering art.

37 ~...^ *~~_n^~~..r ««"«133.001350 The device in some embodiments comprises a "communication interface" between the device and the outside world, such as through radio frequency, infrared, or wired connection, whether local to the wearer or wider area is anticipated. Various communication means suitable for communication between a portable device and other devices, such as portable or stationary, whether remote, local, carried, worn, and/or in contact from time to time, are known in the art. Examples include inductive, capacitive, galvanic, radio frequency, infrared, optical, audio, and so forth. Some non-limiting examples popular today comprise various cellular phone networks, Bluetooth, ultra-wideband, Wi-Fi, irDA, TCP/IP, USB, FireWire, HDMI, DVI, and so forth.

The device in some embodiments comprises what will be called "processing means and memory means," such as to control the function of the other aspects of the device and to retain content, preferences, or other state. Examples include computers, micro-computers, or embedded controllers, such as those sold by Intel Corporation and DigiKey Inc., as are well known to those of skill in the digital electronics art. Other example aspects comprise memory or associated memory circuits and devices and all manner of specialized digital hardware, comprising for instance gate arrays, custom digital circuits, video drivers, digital signal processing structures, and so forth. When the device is divided into parts, such as detachable parts and/or parts worn separately, connection between the parts includes what will be referred to here as "split interface" means, such as to detect the presence and configuration of the parts and to provide for the communication of such resources as power and information. Many of the communication interface means already mentioned are applicable. For galvanic, optical, infrared and some other connection schemes, parts and systems are available from companies such as DigiKey.

In the case of portable devices that include power locally, provision in some embodiments is provided to allow power to be substantially turned off by what will be called an "on/off switch or function when not in use, by the user and/or automatically such as when no use, removal from the head, folding, or storage is detected, as will be mentioned further.

Other exemplary infrastructure functions include "monitoring" parameters, such as temperature, flexure, power level, to provide alarms, logs, responses or the

38 33-001350 like, as are well known to those of skill in the systems art. What will be called "security" provisions are whatever means and methods aimed at ensuring that non- owners are unable to operate the device, such as by detection of characteristics of the wearer and/or protected structures generally. Mechanisms such as user identification, authentication, biometrics, rule bases, access control, and so forth are well known to those of skill in the field of security engineering.

Referring to Figure 109B, what will here be called "human interface" provisions, which are means and methods for allowing the wearer and/or other persons to communicate information optionally with feedback to the infrastructure already described. Several example aspects are shown. What will be called "audio transducers" are capable of providing sound to the wearer, such as monaural or stereo, by coupling through air or body parts such as bones. Such transducers, such as are available from Bose and DigiKey Inc. in some examples, or special transducers in other examples, can provide audio capture, such as utterances by the wearer and sounds from the environment. Audio feedback, such as the sound snippets commonly played by computers to provide feedback to their users or verbal cues, are well known examples of interface design to those of skill in the user interface art. Audio provides a way to get information to the user, obtain information from the user, and also provide feedback to the user while information is being conveyed. Another aspect of an audio transducer interface is the control of the audio itself, such as setting the volume and or other parameters of the sound and, for example, rules for when and how sounds are played to the user and when and how sounds are captured from the user and/or environment. Feedback for such input, may be visual or tactile for instance.

What will be called a "visible image controller interface" optionally provides images, such as text, stills, and video to the wearer. Such an interface is also directed at providing feedback to the wearer, accepting input from the wearer by wearer gestures and/or actions visible to the system, and also for providing the wearer a way to influence parameters of video playback, such as brightness and contrast, and rules for when and how visible imagery is provided to the wearer. Feedback for such input, may also be tactile or auditory for instance.

39

^"33-001350 What will be called "tactile/proximity" interfaces allow the wearer to provide input through touch and proximity gestures. Feedback for such input, whether tactile, auditory, or visible, for instance, is also anticipated. Eye-tracking and blink detection are other examples of user interface inputs, as are well known to those of skill in the user interface art. Feedback to the wearer may be through tactile sensation, such as with vibration or temperature may generally inform the wearer and optionally provide the wearer with silent notification alerts or the like.

Referring finally to Figure 109C₁ what will be called "content capture" provisions are shown comprised of several examples. An internal clock provides such things as time of day and/or date; such clocks are readily available, such as from DigiKey Inc. Temperature is another example of generated content that wearers may be interested in; temperature sensors are available for instance from DigiKey Inc. Other types of weather-related information, such as barometric pressure and relative humidity, are also anticipated and measurement devices are well known in the meteorological art. All manner of body monitoring, such as heart rate, blood pressure, stress, and so forth are anticipated, as are well known in the medical devices and bio-feedback arts. More rich are images, such as visible, infrared, ultraviolet, and the like, obtained by image capture sensors, facing in whatever direction, as are well known in the imaging sensor art. Location sensing, such as through so-called GPS and inertial sensors, allows location in a macro-sense and also head/body movement, such as for purposes of image adjustment and gestures. Another exemplary content capture is external sound through "microphones." Various combinations of audio sensors provide cancellation of extraneous sound or locking-in on particular sound sources owing to such aspects as their direction and spectral density.

Turning to Figure 110, exemplary component placement is shown in plan view in accordance with the teachings of the present invention. In some examples, components are populated on substrates that are then covered, laminated, or over- molded; in other examples, substrates may be mounted on the surface and/or covered by elements adhered by fasteners, welding, adhesives, or the like.

Components may also be mounted directly to the structural or aesthetic components or layers of the frame, such as using printed or other connector technologies.

40

-133-001350 Referring to Figure 11OA, a side view is shown of an exemplary pair of glasses with some example components, such as have already been described with reference to Figure 100, placed relative to a temple sidearm.

Figure 110B gives another example of related component placement in a removable member, shown as an interchangeable sidearm, such as will be described further with reference to Figure 110.

Figure 11OC indicates some example component placement in the auxiliary device illustrated with reference to Figure 107.

Finally Figure 110D indicates some examples of component placement in the front face of the eyeglasses frame.

Turning to Figure 111 , exemplary configurations for projection of images visible to the wearer and/or capture of images from the eye in accordance with the teachings of the present invention are shown. Figure 111A shows a section through the horizontal of a right corner of a pair of glasses that include an image projection device and/or a camera oriented angularly onto the "lens" of the eyeglasses. The light is sent back from the lens into the eye of the wearer and an image impinges on the retina; similarly, light reflected from the retina, including that projected, as well as light reflected from other portions of the eye is captured. Figure 111 B shows a front plan view of the example one of the eyeglasses eyes with the part of the lens used in the example imaging indicated by a dashed line. Figure 111C is a cross-section of the example lens indicating that it includes a coating surface, such as preferably on the inner surface. The coating preferably interacts with the projected light to send it into the pupil of the eye and/or return light from the eye to the camera. Coatings are known that reflect substantially limited portions of the visible spectra, such as so- called "dichroic" coatings. These have the advantage that they limit the egress of light from the glasses and can, particularly with narrow "band-pass" design, interfere substantially little with vision by the wearer through the glasses.

In one example type of eye tracking system, the camera described here captures images of the eye and particularly the iris and the sclera. In order to determine the rotational position of the eye, images of these features of the eye are matched with templates recorded based on earlier images captured. In one example, a training phase has the user provide smooth scrolling of the eye to display the entire

41

"33-001350 surface. Then, subsequent snippets of the eye can be matched to determine the part of the eye they match and thus the rotational position of the eye.

Turning to Figure 112, exemplary configurations for wearer gesture, proximity and touch sensing are shown in accordance with the teachings of the present invention. The sensors are shown as regions, such as would be used in capacitive sensors, but are not intended to be limited to any particular sensing technology. All manner of touch interfaces including proximity gestures are anticipated. For example, the wearer might adjust a level, such as brightness or sound level, by a sliding gesture along one or the other sidearm or by gestures simulating the rotating a knob comprised of an "eye" of the frame, being one of the frame portions associated with one of the lenses. In another example, grasping a temple sidearm between the thumb and finger(s), the sidearm becomes something like a keyboard for individual finger or chord entry. Position of the thumb in some examples acts as a "shift" key. In examples where images are provided to the wearer, preferred embodiments indicate the positions of the fingers, preferably distinguishing between proximity and touching. Also, the meaning of locations is preferably shown, whether static, such as for particular controls, or dynamic, such as for selection between various dynamic text options.

Referring to Figure 112A, shown are exemplary placement of sensors on the frame front of a pair of eyeglasses. One common sensing technology is so-called "capacitive," as is well known in the sensing art and implemented in chips such as the Analog Devices AD7142 and the Quantum QT118H.

Referring to Figure 112B, shown are some other example placements of various sensors. For instance, two converging lines are shown on the temple arm, to suggest proximity sensing and so-called "slider" sensing, also shown in the example of capacitive sensors. Additionally, positional sensors are shown as two alternating patterns of strips, such as would be understood to detect one or more touch positions as well as sliding. Furthermore, the edge of the frame front is shown with sensors arrayed around it. Referring to Figure 112C, a top and/or bottom view of an eyeglasses frame arrayed with sensors is shown. The hinges can be seen connecting the frame front to the earpiece sidearm. Sensors line the edges including the parts shown.

42

^"33-001350 Turning to Figure 113, exemplary configurations for audio transducers are shown in accordance with the teachings of the present invention. One example type of audio transducer is a microphone. Another is a so-called "bone conduction" device that sends and/or receives sound through bones of the skull. For example, sound is rendered audible to the wearer by sound conducted to the inner ear and/or spoken utterances of the wearer are picked up from the skull.

Referring to Figure 113A, shown is an advantageous and novel arrangement in which the "bridge" portion of the eyeglass frame structure substantially rests on the nose bone of the wearer and these points of contact are uses for bone conduction of sound. For instance, the transducers may rest directly on the nose, as shown for clarity, or they may be configured to conduct through other elements, such as pads or a metal bridge. A pair of transducers is shown for clarity and possibly for stereo effect; however, a single transducer is also anticipated.

Referring to Figure 113B, shown is an alternate example placement of a bone conduction transducer. It is mounted on the inside of the temple so that it contacts the skull substantially behind the ear as shown. Some pressure is preferably provided for good sound conduction.

Referring to Figure 113C₁ shown is an audio and/or imaging pickup transducer. In some examples it is aimed at detecting sounds in the environment of the wearer as well as optionally utterances made by the wearer. Multiple such sensors and arrays of such sensors are anticipated. In some examples, a sound is generated, such as to alert people, help the owner find the spectacles, and/or for ultrasonic ranging or the like. In other examples the sensor is a video camera or night vision camera, aimed forward, sideways, or even backwards. Turning to Figure 114, exemplary configurations for mechanical and signal connection and power switching between sidearm and frame front are shown in accordance with the teachings of the present invention. Figure 114A-B are primarily directed at so-called "on/off' switching at the hinge; Figure 114C-D primarily at power provision through the hinges. However, the two aspects are related in some examples, such as where a slip-coupling includes a power switching capability or where switch contacts are used for providing power.

Referring to Figure 114A, shown is a section through the horizontal of the right corner of a pair of glasses configured with a mechanical button at the junction

43

„.._ ' -"'"^33-001350 between the sidearm and the frame front. In this example, the hinge can be whatever type including a standard hinge. A switch body is shown included in the frame front with a button protruding in the direction of where the sidearm contacts the frame in the open wearable position. When the frame is being worn, or in some examples when it is lying open, the button is substantially pushed by the end of the sidearm and power is supplied for various purposes, such as those described elsewhere here; when the frame is not open, however, such as folded, power is substantially cut off. In some examples the spring-loaded button comprises one or more contacts between the two components of the frame. Such switches are known to be small, such as the DH Series manufactured by Cherry or the D2SW-P01 H by Omron.

Referring to Figure 114B, an alternate shutoff switch arrangement is shown comprising a so-called "reed switch" and permanent magnet. Such switches are known to be small, such as that disclosed by Torazawa and Arimain in "Reed Switches Developed Using Micro-machine Technology," Oki Technical Review, p76- 79, April 2005. When the frame is open, the magnet is sufficiently close to activate the switch, as is known; when the frame is closed, the magnet is far enough away and/or oriented such that the switch closes.

Referring to Figure 114C, an arrangement allowing wire conductors to pass through an eyeglasses hinge is shown also in horizontal section. The conductors pass through a substantially hollow hinge. In some examples the conductors can be completely hidden, such as disclosed for doors by WoIz et al. in US Patent 4,140,357. In other examples, the conductors are in the form of a ribbon and may not pass through the hinge.

Referring to Figure 114D₁ a plan view of a single eye of a frame front including hinge parts is shown. There are two example hinge parts, one for each of separate parts of an electrical circuit. The parts are substantially separate hinge components, cooperating to form a substantially adequately strong hinge assembly; however, they are mounted to substantially insulating material, such as plastic resin from which the frame is formed. Each hinge part forms in effect a so-called slip coupling and, as is known for such couplings, such as disclosed by Gordon in US Patent 3,860,312, can have provisions to interrupt or cut off power in certain ranges of angular positions.

Turning to Figure 115, exemplary external connected auxiliary device configurations are shown in accordance with the teachings of the present invention. Two examples are shown in substantially similar plan view with the eyeglasses fully open and viewed from the top. The hinges can be seen along their axis of rotation joining the temples to the front face.

Referring to Figure 115A, a so-called "retainer" cord arrangement is shown. Ends of each cord are shown emanating from respective ends of corresponding temple arms. In some examples, the connection to the arm is detachable, such as a connector not shown for clarity. In some particular examples the cords are detachable with low force in substantially any direction, such as by a magnetic connector as are known. Another example is the rubber ring clips currently used, but where each clip provides a contact for a different part of a circuit.

The "payload," shown configured between the two cords and substantially flat for convenience in wearing, may perform multiple functions. In one example it performs a cushioning role; in another it is decorative. In further example functions, however, it includes component parts that support or augment functions of the glasses. For instance, it may contain power storage or generation such as batteries that supply power to the glasses, whether for charging onboard power storage or for operation. Another example is memory for content or a connection through which memory devices and/or other interface devices can be accessed. In still another example, a radio transceiver is included. Yet further examples include audio microphones to augment sound capture and additional touch panel surfaces, such as those described with reference to Figure 113.

Moreover, whatever functions may be performed by the payload configured in a tethered mode, may also be performed by a wirelessly connected payload, such as one connected by radio frequency, optical, audio, or other communication technologies and wherever attached or carried on the body or among the accessories of the wearer. For instance, a belt buckle, skin patch, portable phone/computer, wristwatch, or the like may serve at least in part as such a payload. A wearer, as an example, may input selections or other information by gesturing near and touching such a payload while receiving visual feedback of their gestures and touches through the glasses display capability.

Referring to Figure 115B, a tethered necklace configuration is shown as another example. The "feed" tethers, via a connector, to the necklace, which

45

"33-001350 includes the payload. Again, the connector may be detachable for convenience. The necklace may server as an antenna itself.

Turning to Figure 116, an exemplary external auxiliary device is shown in accordance with the teachings of the present invention. In one example, the device communicates information and/or power using electrical coils in substantially close proximity. Other suitable power and communication coupling means are known, such as for instance capacitive and optical coupling. The example shown for clarity depicts a storage case or stand into or onto which the glasses may be placed when not used in the example of coil coupling. The power and communication components in the case or stand shown can be used, for instance, to re-charge the power storage mechanisms in the glasses and/or to perform docking synchronization and data transfer functions between the glasses and the outside world, including downloading/uploading content and updating clocks.

Referring to Figure 116A, an example shows coils optionally located for instance in the temple or around the eye of the glasses; one or both types of locations or other locations are anticipated. Such coils can be formed by printing, etching, winding, or otherwise on substrates or layers within the frame or on its surface or on detachable or permanently affixed modules. Means are know for coupling power and/or information over such single coil pairs. Referring to Figure 116B, an example shows coils included in a glasses case or storage stand. Four coils are shown to illustrate various possibilities. For instance, a single coil in glasses and case is believed sufficient if the case enforces an orientation of the glasses. When stand allows all four orientations (upside down and flipped left-for-right) and the glasses contain both coils (one facing forward and the other backward when folded), the glasses can always be charged. When a case contains four copies of one type of coil (two on bottom, as shown, and two on top similarly oriented) and the glasses contain one instance of that type, any orientation allows coupling.

Turning to Figure 117, exemplary detachable accessories are shown in accordance with the teachings of the present invention. While various configurations and styles of accessories are readily appreciated, some examples are shown in the form of a "temple attachment" to illustrate the concept. It will be appreciated that some of the examples include configurations where the glasses frame does not anticipate the attachment and the attachment is therefore generic and applicable to a wide range of frames. Adhesives, fasteners, clamps and the like for such fixing of the attachment are not shown for clarity. The attachment means anticipated in some frames preferably includes coupling for power and data transfer, such as by galvanic, inductive, and/or capacitive connection.

Referring to Figure 117A, the temple attachment is shown attached to the temple arm but not to the front frame. Other examples include attachment to the front frame.

Referring to Figure 117B, an example temple attachment means is shown comprising fasteners on the arm and on the attachment. For instance, snaps are an example as are magnets as well as hooks and loops. The fasteners are shown as an example as one part on the arm and the mating part on the attachment. The attachment is shown flipped top-for-bottom so as to expose the fasteners. Also, shown in the example is a camera and/or light source, as described elsewhere here such as with reference to Figure 113.

Referring to Figure 118C, another example attachment means is illustrated in section perpendicular to the main axis of the temple arm. The attachment fits over and/or clips onto the arm.

Turning finally to Figure 118, replaceable arm configurations are shown in accordance with the teachings of the present invention. In some settings there may be advantage in the wearer or at least a technician being able to replace one or both temple arms with arms having an accessory or different accessory function. Referring to Figure 1108, two temple arms and a frame front that they optionally attach to are shown. The lower arm, which includes the example projector/camera, will be considered an accessory arm while the upper arm is a plain arm, although both could be different accessory arms. The hinge is detachable. The example configuration for achieving this shown includes two hinge "knuckles" mounted on the frame front. These are preferably electrically isolated so that power and/or signals can be supplied over them. The mating structure on the frame includes the middle knuckle, which preferably includes a spacer formed from an insolating material so as not to short circuit the electrical paths provided. In one example, when the two parts of the hinge are interdigitated they snap together. For instance, the outer knuckles are urged towards the middle knuckle, such as by

47 133-001350 spring force owing to the way they are formed and mounted. A detent, such as a ball mating in a curved cavity, not shown for clarity, snaps the two together as would be understood.

Referring to Figure 118B, a detail of the exemplary hinge structure is shown in the connected configuration.

Steering a beam from a variety of launch positions can be accomplished through a large steerable mirror and front optic mirrors can be arranged in zones only one of which is used per rotational position of the human eye. A potential disadvantage of steering a large mirror is that it tends to be slow owing to its mass. Also, potential disadvantage of exclusively using one zone per eye position is that for eye positions near the edge of a zone maximum-sized mirror may be required from two or more zones within a limited space, resulting in reduced maximum mirror size or increased mirror spacing: the former can result in decreased spot size on the retina and the latter in increased mechanism size. Accordingly, and in keeping with other aspects of the inventive concepts disclosed in co-pending applications by the present applicant, all of which are included here by reference, various inventive aspects are disclosed here to address the above and provide further objects, features and advantages as will be appreciated from the following and the figures and description. Referring now to Figure 119, an example array of mirrors and its use in steering beams to front-optic mirrors is shown in accordance with the teachings of the invention. A combination plan, schematic, and orthogonal projection view is provided in Figure 119A and a corresponding section, through a cut plane indicated by direction lines Q-Q, is shown in Figure 119B. A single mirror is shown as an example of the source origin of the beams, each impinging on the mirror array at substantially the same mirror locations and from the substantially the same angle, as shown, and resulting in substantially the same beam "footprint" on the mirror array. By varying the angle, in general both so-called "tip" and "tilt," the beams at different points in a so-called "time sequential" process are directed substantially at different of the front optic mirrors, as can be seen by the different focus points shown.

This and other drawings generally, as will be appreciated, are not to scale for clarity and practicality. Also, this and other drawing, as will be appreciated, show

48 c™)33-001350 substantially a two-dimensional slice of structure, even though the full three- dimensional generalization of the structure is anticipated.

The steering of the mirrors comprising the array is illustrated in section 19B where some mirrors are shown as "unused" and others as "used." The position of the unused mirrors is believed inconsequential and shown in as a "neutral" horizontal position. The angular orientation of the used mirrors is shown so as to direct the incident beam at one of the corresponding focus points or mirrors on the front optic. The mirror array acts on parts of the beam and its efficiency is believed related to the so-called "fill factor" of the array. Advantages of the structure are believed to include that the smaller mirrors are capable of faster motion and may be more readily fabricated and take up less vertical space.

Referring now to Figure 120, an exemplary configuration including two different zones for the same eye position is shown in accordance with the teachings of the invention. Two different beam footprints are shown, at substantially opposite locations on the mirror array. As will be understood, when the eye is near the boundary between two zones for example, then one of the footprints will be towards the edge of the array and, if the array is suitably sized as shown, this can bring the footprint needed to reach an adjacent cell of another zone onto the mirror array. This mirror of the second zone when illuminated from this position on the mirror array is believed to be able to provide spot locations on the retina that are adjacent to those provided by the mirror of the first zone. The example sourcing of light to the array shown is by a "supply" mirror that is rotatably positioned to reflect light from a fixed source location. Such mirrors generally here can also be formed as mirror arrays. Turning now to Figure 121, an exemplary front optic configuration and use is shown in accordance with the teachings of the invention. Three example zones are included in the portion of the front optic shown. Other portions may have a single zone, the interface between exactly two zones, or an interface between four or more zones. Of course all manner of mirror shapes and sizes are anticipated within the scope of the invention, but without limitation for clarity and simplicity in exposition an example comprising two round mirror sizes is described here. What will be called a "major" mirror may be of a diameter preferably on the order substantially of a millimeter or two. What will be called a "minor" mirror may be on the order substantially of a tenth to a half millimeter in diameter.

49

~..— „ - ""'"133-001350 Accordingly, as will be seen, there are three groupings of major mirrors, one corresponding to each of a first, a second and a third zone. Corresponding minor mirrors of the respective zones are also indicated, preferably covering with a substantially uniform spacing. Some additional minor mirrors are shown without indicating their zones, as more than three zones may be included. (Mirrors of a zone, as has been described in the co-pending applications included as mentioned already, are substantially directed at the eye position range corresponding to that zone.)

In the example shown, the major mirror labeled "a" at the center of the rings shown is substantially aligned with the optical axis or the foveal axis of the eye. In order to obtain spot sizes on the retina adequately small for the concentric radial distance corresponding to the ring "b" of major mirrors shown, such as for a "foveal" or para-foveal region of the retina, additional surrounding major mirrors are used as indicated. Some of these surrounding mirrors are from the same third zone as the mirror "a." Others of the mirrors of the ring are from other zones, the first and second in the example. (It will be appreciated that in a case when "a" is surrounded by mirrors of its own zone, a simpler configuration results as anticipated in earlier disclosures and a single footprint on the steering mirror array may be used.) As has been described with reference to Figure 125 already explained in the example of two zones, the sourcing of light to the front optic from sufficiently differing angles can result in contiguous regions being covered on the retina by the major mirrors of the adjacent but differing zones. When desired a second ring, such as may be referred to as a "macular" ring concentric with the first ring may similarly be employed, as may subsequent concentric rings. Turning now to Figure 122, an exemplary combination schematic and section of a front optic and steering configuration is shown in accordance with the teachings of the invention. This exemplary embodiment includes a front optic arrangement aspect as well as a steering aspect. The front optic "eyeglass lens" comprises one "mirror" or the like per angular position from the eye for some regions and two mirrors for other positions in a second example region.

The mirrors that are used for a single angular position are illustrated with two beams of the same width for clarity. The outermost of the two beams for a mirror, the beam with the larger included angle, has its right leg incident substantially on the

50

.-...„ *^r.~~~~~~ ' ""'"033-001350 example pupil position with the eyeball rotated forty-five degrees clockwise; similarly, the rightmost leg of the inner beam is incident on the example pupil position at forty- five degrees counterclockwise. The left legs of these beams are placed on the mirror array at positions determined by the mirror angle, which is chosen somewhat arbitrarily but so that they land fully on the mirror array, taking into account a substantially larger "beam" width or cone that can be anticipated and depending on the sourcing of light to be described. By varying the location of the footprint on the mirror array between these two extreme positions, as will be understood, the right leg can be steered to anywhere on the pupil between the two extreme positions. Using more than one front-optic mirror per angular range, such as two in the example, as will be appreciated, provides a savings in terms of the effective size of the mirror array used, since it is believed that different ranges can be covered using different front-optic mirrors. (The division of the ranges between the mirrors can, of course, be varied but preferably result in substantially contiguous coverage.) The two mirrors are shown substantially overlaid, so that four beams are shown for each mirror location. The wide beam is shown to highlight its overlaying two beams using the other mirror, one on each leg. The narrow beam overlays a beam that uses the other mirror, but the overlay is on just one leg. Again, for each mirror there are two beams representing the range of points on the eye that mirror covers. Each beam is shown with uniform width but not all beams having the same width. The point on the eye where one range ends and the other takes over is where the medium beam is overlaid on the widest beam. (In the examples this transition point on the eye has been chosen somewhat arbitrarily so that the extreme point on the mirror array is the same for both mirrors.) The steering mechanism is shown as an array of mirrors, as described elsewhere here, fed by "source beams" directed by active mirrors. The source beams are illustrated as substantially wider to indicate that a wider beam or cone may be used. The active mirrors are illustrated as two example positions, believed extreme positions approximately twenty-degrees apart and with a pivot point offset substantially from the center. These are merely examples for concreteness and clarity. A passive so-called "folding" mirror is included merely to illustrate an example technique that may be useful in some example packaging configurations.

51

"133-001350 In operation, modulated source beams are developed and directed at the corresponding steering mirrors and sequentially steered to the front-optic mirrors. The source beams are provided in some examples using a "spatial light modulator," such as a ferroelectric LCOS or OLED. The small arrays of pixels resulting form the modulator are combined by an optical system, such as a preferably variable focus lens, such as that sold by Varioptic of Lyon France. (Shutters or active mirrors, as examples, may limit the source beams shown to a single such source beam at a time, or in other examples multiple source beams may be processed at overlapping times.) The particular active mirror receiving the source beam steers it by reflecting it so that it impinges on the mirror array at the location, such as stored in a table. For a particular mirror on the front optic next in sequence, the active mirror receives a beam and directs it at the portion of the mirror array dictated by the angle required for the corresponding mirror on the front optic in order to reach the pupil at the corresponding eye rotation sensed, as will be understood. The sequence of mirrors on the front optic is optionally varied in order to minimize perceivable artifacts.

In other examples, not shown for clarity, a row of one or more single-pixel light sources, such as are generated from a laser or row of separate modulators/lasers, is scanned by a so-called "raster scan" resonant mirror across the surface of the active mirror. The scanning function and so-called "dc" steering function are believed combinable into a single mirror, such as the active mirror shown in the illustration; or, the functions can be performed by separate mirrors or the like. During the scan, which is relayed by the mirror array to a particular mirror on the front optic, the light sources are modulated to produce the portion of the image rendered on the pupil that corresponds to the particular front-optic mirror. In the case of multiple modulators or sources, it is believed preferable to arrange the sources substantially perpendicular to the direction of the scan so that they in effect each paint a row of pixels as the combination of them is scanned across. Thus, in such examples each mirror on the front optic can receive a single scan per "frame" interval and the scan comprises in parallel multiple scan lines, one line per modulated light source. In other examples, a two-dimensional array of light sources is used, and they can be flashed, such as multiple times per mirror.

Turning now to Figure 123, a combination schematic, layout, and block diagram of an arrangement for communicating for display and for displaying of

52 133-001350 foveated data is shown in accordance with the teachings of the invention. The source of foveated image data is any type of communication, processing or storage means, such as for example, a disc, a communication receiver, or a computer. The data is shown comprised of two portions, both of which may be combined in a typical communication architecture, such as one being considered data and the other being considered control. The communication shown may be comprised of a very highspeed single serial line or a bus structure comprising several high-speed lines and optionally some ancillary lines, as is typical of such data communication structures. The raw data, no matter how communicated, comprises two related components. The actual image data, such as for each pixel of a so-called "key frame," comprises a collection of "levels" for each of several colors, such as RGB used to reconstruct the color image in the particular color gamut. The foveation level indicator provides information related to the raw image data and relates to the level of resolution involved in that particular region of the data. For example, a portion of the raw pixel data in a foveal region may be indicated by the foveation level indicator as having high resolution, whereas a portion in a substantially peripheral region may be indicated as low resolution.

The foveated display driver receives the two inputs, however encoded, and stores the resulting image in local storage structures for use in driving the actual display array. Preferably the storage structures are flexibly allocated so that the low- resolution data is not "blown up" to occupy the same storage space as the equivalent region of high-resolution data. For example, a general purpose memory array is adapted with pointers to the regions, where each region is stored in appropriate resolution. The "pointers" may be in dedicated memory structures or share the same memory as the pixel data. However the structure is implemented, a set of substantially parallel outputs that are used to sequentially drive the actual display array in real time are provided. For example, a dedicated controller or other similar circuit fetches/routes the pixel data to the raw display lines. It may, for instance, sequence through a series of "frames," where each frame corresponds to one of the front-optic mirrors already described with reference to Figure 122, and for each a series of memory locations is read out, translated by the algorithm, and placed in a buffer register ready to be gated onto one or more parallel outputs to the display pixels. This controller means expands, on the fly, the low-resolution pixels to the

53 133-001350 high-resolution format of the raw display. This is accomplished by a suitable algorithm, such as digital blurring or anti-aliasing or the like as are known in the art.

In a preferred implementation, the foveated display driver is integrated into the same device, such as a so-called "chip" or substrate as the actual display array, so that the parallel data paths to the actual display pixels are "on chip." Accordingly, the amount of data communicated and the amount of on-board storage are believed reduced by substantially an order of magnitude.

A particular design for a single eye in an eyeglasses format will be introduced and described in detail with reference to the figures, but the specific dimensions and/or arrangements are given as examples for clarity and should not in any way under any circumstance be interpreted to limit the scope of the invention.

OVERALL DESCRIPTION

A particular exemplary design will now be described in an overall manner without reference to the figures for clarity and without limitation.

The lenses of a pair of eyeglasses include "mirrors" of two diameters, 1650 micron and 125 micron. The mirrors are partially reflective or reflect limited bands of light, so that the wearer can see through them substantially as usual. The coatings are preferably applied over a complete surface for uniformity and the whole mirror structure can it is believed to occupy a layer of about 1000 micron thickness inside the lens. The larger mirrors give a spot size on the retina of about 15 microns and cover a 2700 micron diameter; the smaller mirrors give a 120 micron spot size and cover a 5200 micron diameter. (These numbers assume a minimum 2.7mm pupil diameter, which is believed present for most indoor viewing; however, the numbers do not include any clipping.)

The large mirrors are arranged in a hexagonal grid with 2300 micron center- to-center spacing along three axes. Each large mirror is oriented to reflect from the fixed "source origin" point to the nearest point on the eyeball. This point on the eye is the center of the pupil when the eyeball is rotated so that its optical axis is aimed at the center of the mirror. The set of large mirrors is divided into disjoint "triples" of mirrors in a fixed pattern. The three mirrors of each triple are each adjacent to the other two, their centers defining an equilateral triangle. Each triple has associated

54 133-001350 with it six "clusters" and each cluster contains six small mirrors. (A consequence of this arrangement is that each three-way adjacent large mirror triangle, whether or not it constitutes a cluster, determines a gap that contains a cluster of six small mirrors.)

The coverage of the large mirrors on the retina is believed complete for a circular region around the eye's optical axis. By moving the effective source of light from the fixed source origin point, concentric rings are covered. Larger spot size, corresponding to lower resolution, is used for larger rings, where such lower resolution is believed sufficient.

The large mirrors are used to cover three regions on the retina: a central disc and two concentric bands. The tiling alignment of the central six mirrors is believed the most critical, as it corresponds to the area of the eye with the highest acuity. This is the "foveal" disc, defined here as enclosed by the circle of one degree radius from the center of the retina. The full 1650 micron mirror diameter is used for the foveal disc, giving a spot size on the retina of about 15 microns. Around the central foveal disc there is a second circle, called here the "macular" circle, defined to be a circle on the retina corresponding to two degrees on all sides of the optical axis. A reduced beam size and corresponding spot size of 30 microns could be used for the mirrors that serve the band between the foveal disc and macular ring (called the macular band), but that do not impinge on the foveal disc, although for simplicity this is not considered. The third concentric circle is called here the "paramacular" circle. Two concentric rings of mirrors cover the band between the macular and paramacular circles (called the macular band), as is believed sufficient. The spot size required in the paramacular band is about 60 microns. This is achieved using about a 250 micron diameter eccentric part of some of the large mirrors in the band. The ability to "steer" or move the origin point, about 5,000 microns in all directions, is believed adequate to in effect change its position so that light from the outer ring of large mirrors used for a particular eye position is able to reach the furthest part of the minimal pupil.

A point on the eyeball corresponds to a large mirror that feeds it light from the source origin, but such a point corresponds to a whole set of small mirrors that feed it light from points distributed all over the front optic. Consequently, there are such sets of small mirrors for each of many "zones" on the eyeball. More particularly, considering the set of small mirrors aimed at a single example such zone (one thirty-

55

~...~ „„..,. ""-""133.001350 sixth of all the small mirrors): these mirrors are arranged uniformly across the lens and they provide a substantially uniform coverage of angles from the front optic to the particular zone on the retina and are aimed at the center point of the zone. So as to compensate for any deviation of the optical axis of the eye from the center of the nearest zone, the source origin is offset so that the beams enter the pupil, for which a maximal offset similar to that used for the large mirrors is believed sufficient. For any particular eye rotation, light is provided to all mirrors of a zone, apart from the few mirrors whose retinal surface is fully covered using the large mirrors. Tiling of the small mirror images, which are lower resolution on the retina, is preferably lined up with that of the paramacular band.

DETAILED DESCRIPTION

Turning now to Figure 124, a detailed exemplary plan view of mirrors on the front optic, such as an eyeglasses lens, is shown in accordance with the teachings of the invention. The center-to-center spacing of the large mirrors is shown as is the example hexagonal or honeycomb packing arrangement as will readily be appreciated.

Also shown are smaller mirrors, such as slightly curved so that the pixel size they generate is the desired size smaller than is achievable with the 200 micron diameter. As will be appreciated, the small mirrors are arranged in six triangular clusters, each cluster containing six mirrors, the collection of thirty-six such mirrors being referred to as a "star" of mirrors. The pattern is shown in lighter color repeated across the front optic. It will be seen that each star of mirrors in effect occupies in terms of spacing the same areas as three large mirrors.

It is believed that there are many more small mirrors than needed to cover the retina. An example use for all the small mirrors is to reduce the amount of angle steering required to source light to the front optic. A particular example, which will be described in more detail and elaborated on further later, is to use one complete retina-covering set for each of multiple possible zones of eye rotation. Thus, wherever the pupil is, a retina-covering set of small mirrors should point nearby and

56

"133-001350 the sourcing mechanism need only bring this point closer if needed so as to enter the pupil. In the example there are thirty six such retina-covering sets.

Turning now to Figure 125, a detailed exemplary plan view of macular aspects of mirrors on the front optic is shown in accordance with the teachings of the invention. In particular, only the large mirrors as already described with reference to Figure 124 are shown for clarity. Overlaid on the mirror diagram, for conceptual ease as will be appreciated, are shown various circles indicating the corresponding regions on the retina for a particular instance. The empty circles shown in solid lines correspond to the position of the eye oriented so that the foveal region is centered. The macular band is shown as concentric. An example misaligned case, shown in dotted lines, is believed the worst cast misalignment. The filled discs centered on each large mirror are believed to correspond to the regions on the retina covered by pixels formed by light reflected from the corresponding mirror. The larger mirrors are used fully for these circles, giving a pixel size of about 15 microns, believed substantially adequate for the foveal region. The "macular" band, as it is called for convenience here and only loosely related to the anatomical term, is an area between concentric circles in which a resolution of substantially half that of the foveal disc is believed needed by the eye. The slight gap visible in the upper left in covering the worst-case example is believed readily covered by, for example, use of more mirrors or by extending the range of the mirrors nearby, possibly suffering some clipping by the pupil if it is at a minimal dilation.

Turning now to Figure 126, a detailed exemplary plan view of paramacular aspects of mirrors on the front optic is shown in accordance with the teachings of the invention. The "paramacular" band, as it is called for convenience here and also only loosely related to the anatomical term, is the bounded area beyond the macular ring already described with reference to Figure 125. The resolution believed required by the eye for this band is believed substantially half that for the macular region. This region is believed coverable by use of the same mirrors, but with a smaller spot size, such as about 400 microns, providing the desired pixel size and also a correspondingly larger coverage circle. As mentioned, however, such smaller effective circle sizes may not be used. Again the dotted line shows what is believed a worst-case misalignment and is optionally covered by use of more mirrors or larger circles from the mirrors used. As will be appreciated, not all the mirrors shown are

57 033-001350 used, as will be more clearly seen when the present Figure is compared to that to be described.

Turning now to Figure 127, a detailed exemplary section through the eye and front optic of an example arrangement of beams related to the large mirrors on the front optic is shown in accordance with the teachings of the invention. The front optic is taken for clarity to be a curved transparent "lens" (although shown without any power for clarity) comprised of the large mirrors as shown in one color. An example curvature and spacing from the eye are shown and dimensioned only for clarity, as has been mentioned. The beams impinge on the eyeball substantially perpendicular to it, as will be appreciated, so that they are substantially able to supply pixels to the foveal region when the eye is aimed at them. (It will be appreciated that the optical axis and foveal axis of the eye are not the same, but for clarity here the foveal axis will be considered operative.) The mirrors in the row across the front optic shown are arranged in this example for simplicity to all correspond to substantially the same origin point shown. Other example arrangements wit multiple origin points will be described later.

Turning now to Figure 128, a detailed exemplary section through the eye and front optic of an example viewing instance of beams related to the large mirrors on the front optic is shown in accordance with the teachings of the invention. The mirrors already described with reference to Figure 125 are unchanged but nine example beams are arranged to enter the pupil. Accordingly, as a consequence of the law of reflection obeyed by the mirrors, the origin points of the beams are splayed.

Turning now to Figure 129, a detailed exemplary section through the eye and front optic of an example viewing instance of beams related to the small mirrors on the front optic is shown in accordance with the teachings of the invention. The pupil is shown corresponding to an example rotation of the eye. Only the particular set of small mirrors comes into play to facilitate provision of light to the eye for the region around the parafoveal, as has been explained. In the example, a single source origin is shown for clarity, although multiple such points are considered in later examples.

Turning now to Figure 130, a detailed section of an exemplary light sourcing means and system is shown in accordance with the teachings of the invention. The frame or inertial reference is shown in bold outline, which preferably corresponds to

58 , """-133-001350 the frame of reference of the front optic and provides support, such as portions of or attached substantially to the frame of a pair of eyeglasses. Modulated beam sources, such as lasers or collimated LED's are shown for completeness as will be appreciated; however, variable focusing and other pre-conditioning means for the sources, such as disclosed in co-pending applications including the applicant as an inventor, already included here by reference, are not shown for clarity. The front optic is potentially positioned above, as the output angle boundary lines show the range of angles of light sent upward, and the range of angles is substantially sixty degrees, being substantially that apparently called for in the examples already described with reference to Figure 127 and Figure 128.

So as to allow the angular range required of the large mirrors to be kept substantially within current integrated mirror performance, or at least to be reduced, multiple small galvo mirrors reflect the light from the beam sources to the large galvo mirrors (such as via a beamsplitter). The large galvo mirrors take the various angles input to them and reflect the light out at a modified angle, believed up to about plus or minus ten degrees in the example. The light sent to the front optic in order to create the pixels on the retina is sent from varying angles, as will be understood and described in more detail in co-pending applications already included here by reference. The small galvo and large galvo have cooperating movement so as to create the varying angle at the eye and substantially fixed or potentially moving across the pupil point of entry into the eye. For example, the small galvo launches the beam at varying positions on the large galvo mirror and the large galvo compensates to keep the output beam incident at the desired points. Thus, there is large motion to point towards a particular mirror on the front optic and also small motion to render different pixels using that front optic mirror.

In some settings and example uses the wearer may not return to or keep the eye in a substantially enough fixed relation to the frame. The present steering system can provide the corresponding adjustment by means of an actuator. In the example, the large galvos are attached to a substrate that is in effect a stage that can be translated substantially in its plane by flexing of "posts" that support it in relation to the inertial frame. Such translation stages are known in the art of microscope sample positioning. An example voice coil actuator is shown. This comprises a fixed permanent magnet assembly and one or more moveable voice

-J33-001350 59 coils shown. The coils are attached relatively rigidly to the platform as shown. Current in the coils exerts sideways forces on the stage and the posts bend to allow lateral motion. The motion can compensate for movement of the eye relative to the frame and also, in some examples, and as needed by smaller and faster movements, centering the large galvos in the position needed to make optimal use of the available pupil. Sensors and positive feedback mechanisms not shown for clarity, as will be understood, are employed for controlling positioning the voice stage.

A variety of approaches to allowing the desired mirror size and at the same time keeping mirrors from colliding are anticipated. In some example, adjacent front optic mirrors are oriented slightly differently to use different origin points so that the corresponding mirrors are located substantially beyond range of each other. In other examples, an instance of which will be described, nearby mirrors are oriented slightly differently in order to share a common mirror, thereby reducing the number of large galvos used. The example shown of five large galvos (i.e. steerable mirrors) is a row in the pattern to be described.

Turning now to Figure 131 , a detailed exemplary plan view of an orientation pattern for large mirrors on a front optic is shown in accordance with the teachings of the invention. Each hexagon corresponds to a "large mirror pointing cluster," being a set of large mirrors on the front optic all aimed substantially so that they can obtain light from the same mirror or location of the sourcing origin. Where they deliver the light onto the eye, corresponding to such a sourcing point, can be as already described with reference to Figure 127. In particular, when the eye is substantially aimed in the direction of a particular mirror, but as the eye rotates, the beam may need to originate from different points. However, each successive mirror, as described with reference to Figure 19 at the origin point, comes into play at substantially the same time for all the beams. The lateral displacement of the steering mechanism described is another example way to align beams with mirrors and the pupil. Owing, however, to the reduced lateral distances on the eye compared to on the front optic (about a factor of three in the example here) and to the relatively larger size of the pupil — especially when it is dilated beyond the minimum assumed — such considerations may not come into play in some system or at some times.

60

"033-001350 The particular arrangement of large mirrors already described with reference to for instance Figure 123 are shown as circles, but the hollow circles though shown earlier were not used to create coverage on the retina in the examples so far and may accordingly be considered "skipped" here. It will be seen, then, that the pattern of hexagons can be shifted by a single mirror in any direction along each of the three main axes parallel to the sides of the hexagons, without any non-skipped mirror leaving the enclosing nineteen hexagons. This is believed to mean that no matter which contiguous set of large mirrors on the front optic is closest to cover a the paramacular region for a particular position of the eye, the set of nineteen large galvos in the steering apparatus can source the needed light.

Turning now to Figure 132, a detailed exemplary plan view of an orientation pattern for small mirrors on a front optic is shown in accordance with the teachings of the invention. It will be appreciated that there are about thirty six times more small mirrors than are believed needed and that this overabundance can be used to reduce the range of source origin points provided, as has been mentioned. One way to do this is for each batch, of the total thirty six batches, to be assigned its own "well-spaced" point on the eye and to take light from a single source point for this. In an example improvement, related at least to the steering system already described with reference to Figure 130, not just one source origin point but a collection of origin points is used. This lets the work be distributed among more than one large galvos. (Of course a separate steering mechanism can be employed for the small mirrors, but re-using the mechanism for the large mirrors has apparent economy and efficiency, especially since it is believed that it will be overly capable based on the performance of galvos currently available.)

The example shown is aimed at providing that at least one complete and undivided batch of large galvos applies, no matter how the point of regard is aligned with the pattern on the front optic. This may not be a necessary condition, but if it is satisfied then it is easy to see that all the points on the retina are covered by the single batch, as opposed to having to mix batches. An example construction of this type is illustrated using a pattern similar to that shown in Figure 131. The pattern of Figure 131 is shown as well in dashed lines, with the pattern for the small mirrors being from the seven solid hexagons with the colored hexagon in the middle. It is

033-001350 61 accordingly believed that the effort is divided up among seven mirrors, and that the division is arranged to be substantially even. As will be appreciated, if the pattern is shifted to any aligned pattern (which is assumed workable by the argument mentioned earlier related to the lateral dimensions on the eye being reduced) then at least one complete pattern of a central hexagon and its surrounding six hexagons is included among the positions covered by the large mirrors of the example steering system of Figure 130.

Turning now to Figure 133, a detailed exemplary section through the eye and front optic of an example arrangement of beams related to an example orientation pattern of the large mirrors on the front optic is shown in accordance with the teachings of the invention. Instead of all the beams being splayed substantially evenly as was described with reference to Figure 128, the beams are directed at particular mirror locations. In the example, the mirror locations are according to the pattern described with reference to Figure 131. Accordingly, it will be appreciated that following along a row (any of the three orientations) will result in three mirrors in a row associated with one steering mirror and then two times two mirrors in a row before the pattern repeats. Thus, the pattern shown in the present figures is an example corresponding to the section shown, where the full clusters are in the pattern two, three, two, two, when viewed from top to bottom. Turning now to Figure 134, a detailed exemplary section through the eye and front optic of an example arrangement of beams related to an example orientation pattern of the small mirrors on the front optic is shown in accordance with the teachings of the invention. Each batch of small mirrors is shown oriented so that it shares, instead of a single origin point, a set of origin points. (The example pattern is chosen for clarity such that the beams do not cross between the front optic and origin points, although this is arbitrary.) This allows the sharing of steering mirrors shown in Figure 133. In the example shown, three steering mirror locations are used, corresponding to the maximum number in a single row that the section is through. Turning now to Figure 135, detailed sections through the eye and front optic of exemplary arrangements of reflectors on the front optic for obtaining light from the environment of the wearer are shown in accordance with the teachings of the invention. Two exemplary arrangements are shown, one using more of its own optical paths and a second using more parts of the existing paths.

62 133-001350 Figure 135A shows three example additional reflectors; all the large mirrors on the front optic would preferably actually be accompanied by such large reflectors, but only three examples (a, b, and c) are illustrated for clarity to avoid clutter. As will be seen, the reflectors bring in from the environment the beams of light that are substantially co-linear with the beam from the front optic to the eyeball. When the eyeball is rotated so that the pupil is aligned with one such beam, the light from the reflector impinges on the origin point and is split from the source beam (using a beam splitter, as would be understood and disclosed in other co-pending applications already included here) and detected. Examples of such electromagnetic radiation that can be so reflected and detected include the visible portion of the spectrum as well as parts of the IR and ultraviolet spectrum.

Turning to Figure 135B, another example way to capture energy from the environment that would impinge on the retina is shown that uses more of the already-described optical paths. Again three exemplary mirror positions are shown and an additional reflector is shown included for each. This reflector is oriented substantially perpendicular to the beam from the front optic to the eye, as shown. The result is believed to be that the light from the corresponding points in the environment are reflected substantially back to the mirror of the front optic and from there to the origin point where they are detected as described. In this example, the reflectors for outside light and the mirrors of the front optic preferably reflect a small percentage of the desired radiation, such as a broad spectrum of the visible. However, to reduce interference related to the sourced light, various thin film coatings and the like may be used. For example, when the sourced light is narrow bands of, for instance, RGB, the mirror on the front optic can be coated to reflect these narrow bands very efficiently and to substantially reflect the broader band much less efficiently, as is known. Similarly, the reflector for generated light may in some examples be coated to pass the narrow bands and only reflect the rest of the broader band. The result is believed to be that most of the sourced light is contained by the front optic and not attenuated or reflected substantially by the addition of the reflector for outside light, yet a portion of the outside light (apart from the narrow bands) is reflected to the detectors. Similar techniques can also be applied for the small mirrors but, as will be appreciated, are not shown for clarity.

63

"133-001350 In some examples the overall level of external light is reduced, such as by an LCD shutter or passive neutral density "sun glasses" like techniques. This then allows the sourced light to make up a significant fraction of the light incident on the pupil, without substantially increasing the level of illumination compared to the external environment. In turn, this allows the filling in of images with images of better focus or other enhancement, such as for night vision or the like.

BRIEF SUMMARY OF THE DRAWINGS

An example comprised of 0.5mm mirrors arranged in nine zones is used for concreteness but without any limitation. Figure 136 and Figure 137 illustrate a schematic view of an inventive aspect including a spatial light modulator, each illustrating principle rays for a different example zone. Figure 138 then shows in cross-section an example configuration according to the schematic of Figure 136 and 137 and including rays for both zones.

An aspect, presented in Figures 139 through Figure 142, relates to a variation on the exemplary embodiment of Figure 136 through Figure 138 in which multiple beams occupy similar spatial positions. The concept is introduced by a schematic in Figure 139 and then examples are given for RGB and more general combinations in Figure 140 and Figure 141 , respectively. A corresponding schematic view is presented in Figure 142.

A further aspect, presented in Figures 143 through Figure 145, allows in effect "painting pixels on the retina with a multi-pixel brush." Figure 143 provides several example optical schematics related to the approach and Figure 144 shows examples of patterns on the retina. Finally, Figure 145 indicates the approach to steering taken by some example embodiments.

In one exemplary combined embodiment, the approach described with reference to Figures 136 through Figure 138 would be used for the so-called "peripheral" pixels and that described with dereference to Figure 143 through Figure 145 would be used for the higher-resolution so-called "foveal" pixels.

A still further aspect, described with reference to Figure 146 through Figure 148, includes a light source array that in effect provides cones of light that impinge

64

"133-001350 on the front optic elements at a range of distances and are steered to the front optic by a single large galvo.

Turning now to Figure 136, an optical schematic of an exemplary light delivery mechanism using a spatial light multiplexer is shown in accordance with the teachings of the present invention. The schematic view will be seen to start with the laser in the upper right. The laser, or whatever source, will be assumed to produce the three or more colors of light needed for full color if desired, such as by combining separate LED's or lasers through a beam splitter not shown for clarity or by a tunable laser or LED. The next step in the schematic sequence is a so-called "beam spreader," such as are well known and sometimes formed by in effect operating a telescope in reverse. The spread beam illuminates the surface of the so-called "spatial light modulator" (what will be referred to here as an "SLM"), which may for instance be of the so-called "LCOS" type or more desirably the currently much faster ferroelectric type SLM such as those produced by Display Tech of Longmont Colorado.

Three example principle rays are shown, corresponding to the central zone (zone five in row major order); each one is launched from the center of a pixel on the modulator. Each of these central rays, in the example, define a corresponding collimated beam. Each central ray impinges on a corresponding fixed passive mirror in the passive mirror array shown. These mirrors are each tilted so that they launch the beam incident on them onto the center of the small galvo steerable mirror shown next in the schematic sequence. The motion pattern of the small galvo cooperates with that of the large galvo to keep the beams incident on the front-optic mirrors to be described. Each ray is shown impinging on the large galvo at a different point as they reflect from the small galvo at different angles. Accordingly, the beams land on the center mirror of each set of nine mirrors of the front optic. (The mirrors are shown, in the example configuration, grouped by the parallelogram grouping shape that typically would not actually be physically present on the front optic). These mirrors of zone five send the beams to the center of the pupil when the eye is looking straight ahead.

Turning to Figure 137, an optical schematic of an exemplary light delivery mechanism using a spatial light multiplexer, like that shown in Figure 136 but for different example rays, is shown in accordance with the teachings of the present

65 33-001350 invention. The schematic is substantially the same as that already described with reference to Figure 136 except that the principle rays shown are for zone one and correspond to the eye in an upper left position. The rays up to the point of the small galvo are, as will be appreciated, the same for each zone and thus allow every pixel of the SLM to be used for each zone. The pattern is shifted slightly on the large galvo mirror and/or angled differently, however, causing it to be incident on the front optic mirrors of zone one.

Turning now to Figure 138, an example embodiment of the schematic already described with reference to Figure 136 and Figure 137 is shown in horizontal cross section. The light is sourced from the laser and spread by the beam spreader to illuminate the SLM. The beam from each pixel of the SLM is incident on its own reflector of the passive mirror array, which is tilted to send it to the center of the small galvo. As will be appreciated, in the example mentioned above, there are only a few hundred pixels on the modulator and corresponding passive mirrors in the array, but there are substantially nine times more mirrors on the example front optic. Thus, the small galvo, in combination with the large galvo, selects the zone to be used by a slight offset. Two example zone ray collections in the plane of the section are shown in this example for in effect the same beams (with optionally different modulated intensity) incident on the passive mirror array: one is for zone five, the middle zone, and the other is for the zone six off to its right. Each zone ray collection is shown incident on the eye at its respective center point. (It will be understood that the angular distance between the zones is assumed fixed, not the linear distance; alternatively, however, additional small galvos can be introduced to handle each "grouping" of front-optic elements with suitably close angular distance.) Turning now to Figure 139, a schematic view of an exemplary combining passive mirror array in accordance with the teachings of the present invention is presented. The schematic is similar to that already described with reference to Figure 136, however, it will be seen that four example locations on the passive mirror array combine their outputs to produce what appears to be a single beam or substantially a single or slightly offset overlay of beams, whether combined at the same time or at distinct times. Thus, the output is shown as substantially a single principle ray reflecting from a zone five surface on the front optic and entering the pupil of an eye in central or "zone 5 position."

"33-001350 66 Turning to Figure 140, a combined sectional and schematic view of an exemplary passive combining mirror structure for combining different colors of light is shown in accordance with the teachings of the present invention. A series of beam splitters is in effect created to combine the beams, as will be understood generally, and in this example aimed at combining colors that are modulated by different portions of the spatial light modulator structure. In this exemplary embodiment of the concept illustrated three beams of light are shown impinging on the passive reflector structure from the SLM; each beam can thus be modulated separately by the SLM. The beams are combined, by a prism structure shown, into in effect a single beam. Thus, three pixels of the SLM are used, each modulating a separate one of the red, green, or blue color components (or whatever set and cardinality is used for whatever color gambit) and the combined beam output includes the combined full color.

Turning now to Figure 141, a combined sectional and schematic view of an exemplary passive beam combining structure is shown in accordance with the teachings of the present invention. Multiple beams are shown entering the passive prism beam combiner structure. In this example configuration, each coated surface may be angled somewhat differently; also, as will be appreciated, each beam may impinge on the coated surface at what is in effect a different position relative to the central axis of the structure (as indicated by the "beam boundaries" shown relative to the section for clarity), thereby changing the effective point of origin of the beam. Only changing the angle, keeping the central rays intersecting on the surface, is believed to yield only an angular change and relates to the example considered more specifically with reference to Figure 145 as will be described. Changing both the angle and the intersection point (principle rays meeting, if at all, off the coated surface), is believed to allow the beams to result in nearby if not adjacent pixels on the retina. The passive mirror array, already described with reference to Figure 137through Figure 138, is in the example schematic of Figure 139 in effect adapted to combine beams into substantially the same space. In the particular example shown, a single transparent combiner structure uses coated surfaces to reflect the beams incident from the SLM along substantially the same output axis. It will be appreciated, however, that other configurations of combining prisms, such as a tree

67

"33-001350 structure where two already combined beams are themselves combined, is also an example type of structure included within the scope here but not shown for clarity. Turning now to Figure 142, a combined sectional and schematic view of an exemplary overall system including passive beam combining structure is shown in accordance with the teachings of the present invention. The figure shows as an example combining four beams, but can also be considered an example of the RGB combining (such as with four "primary" colors) as already described with reference to Figure 140 or more general combining as already described with reference to Figure 141. The arrangement of the sectional view is similar to that already describe with reference to Figure 138, except that the passive mirror array is differently configured and only two of its output beams are shown as examples (though, as will be appreciated, two separate instances related to galvo positions are shown). The principle rays of the eight beams when leaving the spatial light modulator are shown substantially parallel and uniformly spaced. Four of the beams impinge in the example upon one prism combiner sequence and the other four adjacent beams impinge on the other combiner structure.

The result from the passive mirror array is shown for clarity here as substantially two beams; it may for instance in some examples as will be appreciated in fact be eight beams with slightly different angles and at least two effective launch points or two full color beams. The output will be regarded, however, as two beams here for purposes of considering how they interact with the front optic and reach the pupil. As will be seen, in one configuration of the galvos the beams are oriented to impinge on the center of zone five and in another configuration of the galvos on the center of zone six. The solid (non-dotted) lines indicate zone five and the dotted lines zone six. The two example front optic regions used can be seen to be nearby each other as suggested by the nearby status of the beams leaving the passive mirror array.

Turning now to Figure 143, schematic views of exemplary vibrated element sources are shown in accordance with the teachings of the present invention. An example approach is shown to increasing the number of pixels rendered on the pupil when the scanning of the large galvo is too slow to allow the desired number of scan lines to be written directly. The small galvo is what will here be called "vibrated," or moved substantially rapidly, so that a pattern of pixels is drawn along the higher-

68

„.„^■ „.,,...,^• "-"^""33._QOi 350 speed path induced by the vibration. The pattern of pixels from one such higher- speed path is then repeated at each interval along the slower-speed path of the large galvo scan. Examples of such patterns on the retina will be described with reference to Figure 144; the present figure shows some non-exhaustive example configurations for generating such patterns.

Referring now to Figure 143A, an example without small galvos is shown. The source, which may in this example radiate a cone of light, is shown on the left. The light leaving it impinges on the large galvo, which moves in a scan pattern, such as horizontal fast and vertical slow. A particular reflective element on the front optic acts as an aperture stop and allows part of the light to impinge on the retina of the eye, where it is believed that substantially a spot results. As the large galvo moves, the effective position of the source is scanned through space and results in the corresponding pixels being rendered on the retina. The optional selective shutter, such as a reflective LCOS or a transmissive LCD shutter is controlled so as to limit the portions of the front optic onto which the light from the source impinges. This shutter may optionally be combined before the large galvo, after the large galvo, or even be integrated as part of the large galvo, as will be understood. In this example a resonant structure, not shown for clarity, can optionally be added, such as by a crystal that bends the light through it or by attaching a resonant surface to the large galvo.

Referring to Figure 143B, a resonant galvo is shown taking light from whatever source, such as an LED or part of a SLM, and sending it on to the larger scanning galvo, where it continues on as in the other examples. A resonant galvo preferably vibrates or resonates at a speed substantially higher than the larger galvo can conveniently be moved, so as to allow for the distribution of additional points on the retina along the slow scans, as has been mentioned and will be illustrated further with reference to Figure 144. It is well known that small galvos can have resonant frequencies in the tens of kilohertz, which may be suitable in some embodiments of the inventive concepts described here. The motion in resonance is small compared to the scan line to achieve the example pattern type to be described. It will be appreciated that this embodiment does not in the example described for clarity correct the origin point but rather varies the angle of origin through the resonance; this is in contrast to the example to be described with reference to Figure 143C. In

69

-...~ ~~~~ ' ""'""33-001350 some examples of this configuration the small galvo performs the steering functions that it performs in embodiments without the vibration and at the same time also vibrates. In other examples, the small galvo is supported on a vibrating structure or includes a vibrating structure along with its other components. Referring now to Figure 143C₁ an exemplary embodiment comprising two vibrating elements is shown. Light from whatever source impinges on a first vibrating element, shown in the example as a small galvo, and is then substantially launched at a rapidly changing angle (or with another parameter, such as polarization, varying rapidly) towards the second vibrating element, which launches the light from a substantially varying position and with a substantially varying angle. In one example, the two galvos vibrate in a cooperating manner substantially similar to that of two galvos controlled directly to keep the beam incident on the center of the front-optic mirror and yet vary its point of origin, as has been disclosed elsewhere in co-pending provisional applications already included here by reference. More generally, the vibratory structures cooperate or are coordinated such that the launched angle and position combination of the resulting beam is such that it can substantially be reflected by a beam-width structure in the front optic or enter the pupil through a limited if not beam-width aperture or some compromise between these as described later. In the particular example illustrated, the resulting beam is moved slowly in effective origin position by being reflected by the large galvo shown.

Referring finally to Figure 143D, a combination of a vibrating structure and a small galvo to steer the resulting beam towards the large galvo and ultimately to the front optic is shown.

Turning to Figure 144, an exemplary plan view of pixels on the retina related to vibratory structures is shown in accordance with the teachings of the present invention. Increasing the number of pixels effectively rendered on the retina for a given large galvo speed and pattern is believed achievable by in effect vibrating one or more elements as has been mentioned and its effect on the image on the retina shown here. In the first example, Figure 144A, the pattern on the retina is created by spots that are included on a sine wave pattern superimposed on the scanning pattern, as will be understood by those of skill in the art with reference to the drawing. In particular, it will be seen that the vibration causes the effective scan line to be wavy and thus have increased length, allowing for more dots to be placed

70 33-001350 without dots substantially overlapping and resulting in more pixels on the retina. (The dots are intended to indicate substantially the center of a spot.) Six scan lines are shown, each comprised of about twenty sinusoidal waves comprising about eight spots each, yielding a substantially square pattern on the retina. Some sinusoidal nodes are called out for clarity and the central scan lines are shown as well.

Referring now to Figure 144B, an alternate pattern is generated where the sine wave is oriented angularly and so that the spots can be rendered on a substantially vertical segment of each three-hundred-sixty degree full cycle. Thus, the individual pixels on the retina can be substantially in a rectilinear pattern because of the way the combined trajectory includes substantially vertical segments. One advantage of such an approach is that the pixel locations are close to the rectangular arrangement, and even the square pixel aspect ratio, currently in use.

Turning now to Figure 145, combination schematic and sectional view of exemplary vibrated-element overall configurations are shown in accordance with the teachings of the present invention. Figure 145A shows the principle rays and is partly overlapped on the same drawing sheet for ease in reading, as will be appreciated, with Figure 145B that shows the corresponding beams. The small difference in angle of the principle rays can be seen to propagate through the mirrors in Figure 145A. It will be seen that already at the point of the front optic the principle rays are too far apart to be incident on the 500 micron passive mirror located there in the example. Nevertheless, as will be readily appreciated in view of Figure 145B the substantially wider beams can impinge on the smaller mirror and thus result in beams of appropriate diameter directed at the pupil of the eye. The beams arriving at the pupil will, it is believed, have diverged and spread to an extent that still allows them to enter the pupil.

Turning now to Figure 146, a schematic view of an exemplary steered array source in accordance with the teachings of the present invention is presented. The source array can be any suitable means of generating the light, in the example shown as pixel source regions on a plane and oriented substantially perpendicular to a plane. One example is an array of light emitting devices, such as OLEDs or whatever other technology. Another example arrangement, as would be readily understood, is a transmissive array that is lighted from the back. A further example arrangement is a reflective array, such as a typical LCOS, preferably using

71 "33-001350 ferroelectric material, such as those sold by Boulder Non-Linear Systems. The light impinges on the large galvo that directs it successively, in operation to the various front optic elements, preferably visiting each once per frame, such as twenty-four to one hundred times per second. The order may optionally be varied from frame to frame for an improved viewing experience for a given speed. The power remaining in the light wavefront impinging on more distant elements is reduced, due to divergence, and this effect is preferably compensated for to produce the perception of a more uniform image. After leaving the front optic, the light enters the pupil of the eye. Turning now to Figure 147, a schematic view of an exemplary steered array source with aperture in accordance with the teachings of the present invention is presented. The schematic plan view of Figure 147A indicates the aperture array imposed in such a way that it blocks light with too much angle deviation from the normal. In one example, it is an opaque structure placed in front of an emissive array. In another example, it is located behind the modulating array in a transmissive arrangement. One advantage of such an arrangement is believed to be the reduction of stray light, for example light that would impinge on front-optic elements other than the one steered to at a particular time or more generally preventing unnecessary scatter of light. Referring to Figure 147B, a side view section is inset for clarity. It indicates an arrangement where the light leaving the array passes through the aperture array, as indicated by the example rays aimed at the galvo.

Turning now to Figure 148, combination schematic and sectional view of a direct source configuration with optional aperture is shown in accordance with the teachings of the present invention. The light source array, as already described with reference to Figure 146 is show launching light at the large galvo. In one configuration of the large galvo, it reflects the light to near the center of the front optic; rotated slightly counter clockwise, it sends the light towards the near corner of the front optic. Both reflections impinge on substantially the center of zone six.

As will be appreciated, but now shown for clarity, the effective cone of light from each pixel on the light source array diverges. In the example embodiment shown, when the cone reaches the center of the front optic, it has a wider spread than when it reaches the shorter distance to the near corner. Accordingly, the cone is cropped more by the one front optic element, when it is the same size, as by the

72

^„.._ "-"'""33-001350 other. This difference in illumination is compensated for in the driving of the source array.

In another set of examples, presented for concreteness and clarity in exposition, the front optic mirrors or dichroics might be on the order of 1mm in diameter and spacing on the front optic on the order of 2.5mm center to center. As another concrete example, pixels of the light source might be on the order of 10 or 20 microns. For efficiency, a nearly collimated beam might be used to generate the light from a spatial modulator, so that the cones are not too divergent.

Many variations and extensions will be understood to be possible in keeping with the spirit of the invention and inventive concepts disclosed, for example:

In general, related to all the front-optic mirror or diffractive structure schemes contemplated here and in any and all of the provisional applications included here by reference, it will be understood that the front-optic structure may in some variations be such that the beams impinge on it in essentially a single point per structure or, in other examples, over a range of central positions so that the beams enter the pupil with their central ray at a central point relative to the pupil. More generally, the point at which the beams converge may be anywhere between the front optic and the pupil (or even between the pupil and the retina). It will further be appreciated that some clipping may be allowed either by the front optic element or by the pupil. In other example variations, instead of a beam spreader, as described such as with reference to Figure 109, one or more conical beam sources, such as LED's or the like are used. The geometry of the passive mirror array, if used, is adapted accordingly. When an array of such sources is present, they can be used to create images that are then allowed through to one front optic at a time by the SLM. For peripheral locations on the retina, a single source (whether one or more LED's) can uniformly supply light, with downstream modulation by the SLM, to all the corresponding points on the front optic; but for higher-resolution portions of the image, the array of sources can be modulated to create the image and this is allowed through just one of the SLM pixels to the corresponding position on the front optic. Thus, time is divided between in effect "broadcast a single pixel at a time to all the peripheral points" and "monopolization of the time slot for a particular front optic element related to the foveal or macular region." As another example, a somewhat sparse array adequate for low resolution is flashed for each peripheral location; but

73

„..„ . ' -""^»33-001350 that pattern is flashed in a set of staggered slight shifts so that an array of pixels results that includes a number of pixels equal to the base number of pixels times the number of flashes. The density of the overall resulting rectangular array of flash points can be adjusted by the offset shift amounts, providing a degree of freedom or two that can aid in the tiling of the images on the retina. Blurring of the peripheral pixels may be provided, for instance, by alternate source LED's or a liquid crystal.

Referring now to Figure 149A, an example of an inductive coil coupling means is shown for clarity and concreteness. Such coils are able to transfer power and high-speed data, such as is known in the art, for instance as disclosed by K. Chandrasekar et al in "Inductively Coupled Board-to-Board Connectors," Electronic Components and Technology Conference, 2005. Such coils can in some examples be "printed," such as by etching away conductive areas on a substrate. Capacitive coupling is also known and potentially used here, but is not shown for clarity. Referring to Figure 149B, an inductive coupling embedded in eyeglasses frame, such as substantially near the end of the sidearm earpiece is shown. Example ways to fabricate such a structure include forming the coil structure by known means and then adhering, laminating or potting it into the sidearm. Again, capacitive structures not shown for clarity are applicable separately or in addition to inductive structures. Referring to Figure 149C, an example mating lanyard end boot is shown fit over the sidearm end. A suitable coil structure is formed within the preferably substantially deformable boot. The boot is shown fit over the end of the sidearm earpiece, presumably so that it is held in place by the elasticity of the material it is made from (and/or the material the sidearm earpiece is made from). The lanyard exits from the end boot. Again, capacitive structures are applicable but not shown for clarity.

Referring to Figure 149D, a section through an exemplary inductive coupling boot surrounding a side arm is shown. The earpiece can be seen surrounded by the lanyard end boot and the cross-sections of the coils, such as printed coils, can be seen arranged substantially near each other.

Turning now to Figure 150, a schematic view of an exemplary surface diffractive grating element is shown for the purpose of characterizing such known types of structures and describing how they can be designed generally. The

33-001350 74 diffractive grating element defines a substantially planar surface assumed in this example to lie in the xy-plane. The diffractive grating element can be characterized by a complex surface having a periodic spatial variation, complex reflectivity denoting reflectivity that includes both amplitude and phase of the reflected light. The surface normal vector of the diffractive grating element N in this example is parallel to the z-axis. In a more general case the diffractive grating element surface can be curved, in which case the grating normal is position dependent and is defined locally relative to a plane tangent to the surface of the diffractive grating element. The reflectivity can vary periodically in amplitude, phase, or both as a function of position on the diffractive grating element surface. As shown in the exemplary diffractive grating element of Figure 150A, reflectivity is substantially invariant with respect to translation parallel to the x-axis and exhibits periodic variation with respect to translation along the y-axis. Regions of constant reflectivity are referred to as diffractive contours, which in the example of Figure 150 are substantially straight lines substantially parallel to the x-axis. The orientation of the diffractive contours in Figure 150 and the reference axes are chosen for expositional convenience only. As described further below, diffractive contours can be straight or can follow curvilinear paths. They can be continuous or they can be dashed, segmented, or otherwise partially written to control overall effective contour reflectivity, to enable overlay of multiple diffractive grating element structures, or for other reasons.

The diffractive grating element can be characterized by a wavevector K₉ which lies in the plane of the diffractive grating element and is oriented perpendicular to the diffractive contours. The magnitude of K₉ is Ma, where a is the spacing between diffractive contours measured along a mutual normal direction. In diffractive grating elements more complex than the example shown, having curved or variably spaced diffractive contours, the wavevector can be defined locally for small regions over which contour spacing and orientation is relatively constant.

Monochromatic light having wavelength λ, incident on the diffractive grating element from some direction, can be assigned a wavevector k_in oriented along a direction normal to its wavefront. In the language of geometrical optics, k_ln is parallel to the ray representing the input light. The wavevector k_in has the magnitude 1/λ. When the input light has a range of spectral components, wavevectors having a

75

_.._ ' ""—33-001350 corresponding range of magnitudes can represent the various spectral components. When the input light has a spatially varying wavefront, the wavevector can be defined locally for small regions over which the wavevector is relatively constant.

The case wherein K₉, k_in, and N are substantially coplanar (i.e., when K₉ lies in the plane of incidence) is schematically depicted in Figure 150B. In that case the diffractive grating element properties, input and output directions, and the

mλ = asmθ_ln -asinθ_mιl , wavelength are related according to the diffractive equation:

where m is any integer (including zero) that provides a real solution for the output angle. The output angle is defined to be positive when on the opposite side of the surface normal relative to the input angle.

In more general cases, when the wavevector K₉ does not lie in the plane of incidence, the output wavevector can be determined by decomposing the input wavevector into components parallel to and perpendicular to the plane of the diffractive grating element. Those components are denoted Hf_n and kf_n , respectively. Analogous components for the output wavevector are kζ_ul and k_o ^z _ut . The values permitted for those quantities are given by the diffractive equations:

where m is any integer including zero that results in a real value for k, out ^■

The diffractive grating elements of the embodiments may be designed using the following approach based on ray optics and the above-specified diffraction equation. First, trajectories of the rays incident on the diffractive grating element and the rays diffracted at each point of the diffractive grating element (on a certain convenient grid) are defined in accordance with the desired functionality. Then

76 33-001350 diffractive equations above are used at each point of the grid to calculate local k- vector of the diffractive element. The local k-vector defines the orientation and the value of the local period of the diffractive element as mentioned at each point of the grid. Thus, the configuration of the diffractive contours of the entire diffractive element may be defined. The approach is viable for designing diffractive grating structures for beam transformation in both one and two dimensions.

Another exemplary approach to designing diffractive grating elements is to use a holographic design approach, as defined in US patent application 11/376,714, based on computed interference between simulated optical signals. While the above mentioned application refers to photoreduction lithography as the preferred fabrication method, other methods including e-beam writing, diamond turning, mechanical ruling with ruling engine, holographic exposure, maskless photolighography and writing with a laserwiter, followed where appropriate by resist development and etch, may be used. Turning now to Figure 151 , a diffractive element and mirror assembly is shown in a projective view that changes divergence in one dimension in accordance with the teachings of the present invention. Two diffractive grating elements 902 and 904 are oriented perpendicularly to each other with independently adjusted galvo steering mirrors 901 and 903 and may change divergence of the light beam in two dimensions, controlling for instance focus and astigmatic properties of the light beam. Mirror 901 is oriented in such a way that beam 905 is perpendicular to the straight diffraction contours of diffractive grating structure 902. Similarly, mirror 903 is oriented in such a way that beam 906 is perpendicular to the straight diffraction contours of diffractive grating structure 904. In this example, a single direction is scanned by the galvo mirror. It will be appreciated that the design techniques described with reference to Figure 150 are an example of procedures suitable for arriving at such diffractives, as would be understood.

Referring to Figure 152, an exemplary known straight line diffractive is shown in section. The desired paths of the rays originate from a point source 1001, diffract on the diffractive element 1002 shown in cross-section and composed of straight line diffractive contours 1003 parallel to each other and having a period a that may have different value along the direction of x-axis, and after the diffraction converge at a single image point 1004. For a ray 1005 incident on the diffractive element 1002 at an angle α and diffracted into ray 1006 at an angle β, the diffraction equation defining the period a of the diffractive grating element at the point 1009, where ray 1006 is incident on the diffractive grating element is

mλ=asin(α)-asin(β).

For a different ray 1007 incident on the diffractive element 1002 at an angle αi and diffracted into ray 1008 at an angle βi, the diffraction equation defining the period a2 of the diffractive element at the point 1010 where ray 1007 is incident on the diffractive grating element is

mλ=d7(sin(αi)-sin(βι)).

For a known m,λ,α_vβ distance between points 1001 and 1009, between points 1004 and 1009 and distance x between points 1009 and 1010, the diffractive equations may be used to find period a as well as period a1 and a function of distance x thus defining the diffractive grating element. For convenience of design and simulation, such dependence may be approximated by a polynomial.

Said diffractive element has the following property useful for the embodiments described herein. If a beam with certain divergence (for example, a collimated beam) is originated from point 1001 and directed at a certain angle to the surface of the diffractive element 1002, it will be directed to point 1004. If the size of the beam on the diffractive element 1002 is less than the area of the diffractive along axis x, the divergence of the diffracted beam will depend on its position on the diffractive element 1002 along axis x. Thus the divergence properties of the beam may be controlled by pointing it to different areas of the diffraction element and as long as it originates from point 1001 , it will be directed to point 1004, where a subsequent directing mirror may be placed.

Turning to Figure 153, actual simulations of the diffractive approach described with respect to Figure 151 and Figure 152 are shown. The simulation was run using optical design software Code V version 9.8, by ORA of Pasadena California.

78

~,. π *~~~r.~~~ , — '""33-001350 In both Figure 153A and Figure 153B, collimated input beam 1101 is incident on steering galvo mirror 1102 and then on diffractive element 1103. Diffractive element 1103 was designed in accordance with the above approach and accordingly directs any beam originating from the center of mirror 1102 to the center of mirror 1104. In Figure 111A, the steering galvo mirror 1102 comprises the angle of 45° with respect to the direction of the input beam as measured from the direction perpendicular to the surface of the mirror. In Figure 153B, that angle is changed to 35°. The divergence properties of the diffracted beam are different, as will be appreciated from the separation in the ray trace shown. In general, the incident beam may not be collimated. Also useful may be a beam that is both divergent and convergent after diffraction on the diffractive element 1103; an example way to achieve this is by inserting a negative power in front of mirror 1102.

Turning now to Figure 154, shown in projection is an exemplary design for a beam-shaping system in accordance with the teachings of the present invention. Control of the cross-section of the beam shape that prepares it so that after diffraction from the front optic it preferably has a substantially circular shape; moreover, the diameter is preferably adequate to achieve a small, and preferably a diffraction-limited, spot on the retina.

The example design, in accordance with the non-holographic diffractive design techniques already described, is composed of two diffractive elements 1202 and 1204 and two steering galvo mirrors 1201 and 1203. As in the case of the diffractive elements for astigmatism/focus control as already described with reference to Figures 150-153, each of the two diffractive grating elements has a separate galvo mirror to adjust beam size in perpendicular dimensions by pointing the beam into a particular section of the corresponding diffractive grating element. Note that mirror 1201 is oriented in such a way that beam 1205 is perpendicular to straight diffraction contours of diffractive grating structure 1202; similarly, mirror 1203 is oriented so beam 1206 is perpendicular to the diffraction contours of grating 1204. Turning finally to Figure 155, exemplary designs for the diffractive gratings of the beam-shaping system of Figure 154 are now described in accordance with the teachings of the invention. The diffractive grating element 1302 is divided into what will be called discrete "sections." Exemplary sections 1304 and 1305 are shown in Figures 155a and 155b, respectively. Each section has straight line parallel 33-001350 79 diffractive contours and these contours are shown perpendicular to the plane of the figure. Each section has a pitch calculated from the diffractive equation to direct the central ray of the input beam after it is reflected from the first galvo mirror 1301 into the center of the second galvo mirror 1303. Due to the difference in the angles of incidence and diffraction, the dimension of the beam in the plane perpendicular to the diffractive contour will change after diffracting on the segment. The difference in beam sizes may be calculated as

cos(αr) a = o o^t" '" cos(/7) "

where a_out is the dimension of the diffracted beam, a_in is the dimension of the incident beam, as shown.

The size of the segment is preferably large enough to accommodate the size of the beam. The adjustment of the beam diameter is in discrete steps and effected by changes in the input angle made by galvo mirror 1301 so that the input beam is substantially fully incident on the corresponding segment. Figure 155A shows reduction in the beam dimensions after the diffraction while Figure 155B shows increase in the beam dimension.

It will be readily understood by those of skill in the art that the arrangements described generally here can be fabricated using volume holograms and that certain advantages and additional capabilities may result.

Turning now to Figure 156, a combination block, functional, schematic and flow, and optical path diagram of an overall exemplary embodiment in keeping with the spirit of the present invention is shown. The "front optic" receives light from the "Front End" combination that includes at least some of the four functions: "Focus Transformation," which optionally adapts to meet the focus needs of the viewer eye and/or the distance to the viewer eye from the front optic and optionally includes astigmatism correction; the "Angle Encoding," which through means such as angle, frequency, or polarization, influences the angle of the light emitted from the front optic towards the eye; "Spot Shaping," which influences the shape of the light incident on the front optic to a desired footprint; and "Position Encoding," which

80 - 33-001350 directs the light form the front end so that it arrives at the desired location on the front optic.

The light input to the front end originates from the "Back End." Three functions comprise the backend. The "Color Modulation" function, as is known from the display art, is preferably performed in the back end by powering the source; for instance LED's are known to be emissive substantially linear in the current through them and are able to handle high bandwidth. The "Source" of light, such as from tunable or monochromatic sources, whether for instance lasers, high-radiance LED's, edge emitting LED's, surface emitting LED's, or organic LED's. The "Beam Collimation" function, preferably downstream from the source of light, is typically performed by conventional lenses or the like but may also include diffractive elements. In some examples, modulation can be downstream from the source, such as by active devices that absorb light or send it in a dead end direction. In some examples the output of the backend is three "beams" of collimated light that are collinear. In other examples, as mentioned, the three beams are not collinear and may optionally be non-parallel.

The "Inputs" section is comprised of three two functions. The first function is "Return-Path Sensing," which preferably receives light from a splitter located at about the interface between the back end and the front end. As mentioned, polarization optionally is used to allow scatter from the system itself to be discriminated from light reflected by the eye. Also, as mentioned, in some examples the sensor detects one or more aspects of the light it receives, such as the degree to which it is concentrated in a spot or spread out due to poor focus. The second function is "Position Sensing," which in some examples is informed by return-path sensing, is aimed at learning the geometry of what the viewer can see and where the front optic is positioned in that geometry. Examples for such sensing include cameras, motion sensors, communication with external devices, and the like, as mentioned elsewhere in more detail. An input to the inputs section, that is presumably not processed but passed through the inputs section, is the content to be displayed. In some examples, all or part of the content is generated locally in the device at some times.

The "Control" section takes its input from the input section. It controls the color modulation of the back end section. It obtains information from the sensing elements

81 33-001350 of the inputs section, optionally under experiments it controls. As a result of programming and calculation not shown for clarity, the control section also controls the angle and position encoding, along with the related spot shaping, depending on the eye position it has calculated and the focus shaping depending on the focus and astigmatism information it obtains from the input section.

Turning now to Figure 157, a combination block, functional, schematic, flow, and optical path diagram of an exemplary safety system in keeping with the spirit of the present invention is shown. Two example substantially independent safety monitors, "Monitor #A" and "Monitor #B," are shown with connection through optional "Opto-lsolation." It will be appreciated that one, two or more such safety monitors may be desired depending on the application and other considerations. When there are more than one, then it is preferable that they are able to communicate and such communication is preferably isolated in some suitable manner so that at least for example the independence of failure modes is easier to verily. A key aspect of a safety monitor system is that it is able to prevent light from damaging the eye of the viewer. To achieve this such a system is comprised substantially of two functions, monitoring and shuttering. The "Fail-Safe Shutter" function is indicated as being applied to the "Non-Safety Front-End/Back-End" rectangle enclosing the "Back End" and "Front End" functions, already described. This is to depict that the failsafe structure preferably operates on one or both of these functions. Examples of failsafe shutters include, but without any limitation: MEMS mirrors that have a safe rest state and means to prevent their powering and being taken from the rest state; flip-flops or the like that hold power to the light sources unless they are reset; and LCD shutters that are interposed in the optical path that block the light and "trap" it when they are returned to their un-powered state.

A safety monitor includes among its inputs photo sensors of two general types. One type of such sensor, a "Sent Energy Sensor," is interposed between the front end and the front optic and receives light indicated by a beam splitter that is directed substantially at the front optic, thereby performing the more general function of measuring the light sent out by the system. A second type of such sensor, a "Returned Energy Sensor," is responsive to light returning through the light path that typically includes reflection from the retina, thereby performing the more general

82

~"~ """13-001350 function of measuring light incident on the retina. An example is shown as interposed between the back and the front end and using a beam splitter configured so that it is responsive substantially to the light being returned.

The operation of an exemplary safety monitor will now be described. One or more conditions are preferably satisfied to prevent the monitor from pulling the enabling signal(s) from the fail-safe shutter. One such condition relates to the dynamic nature of relative sensor measurements. For example, the difference in the light sent and the light received should vary, due to the presence of blood vessels and the like, as a focused spot is scanned across the retina. The safety monitor computes, whether by analog or digital means, this difference from the sent energy sensor and the returned energy sensor and contains structure that allows it to make a determination as to whether there is sufficient variation to indicate that the spot is in fact being scanned. One example type of suitable structure is a filter, whether analog or digital, that passes energy at the expected frequency, and a threshold measuring structure, whether analog or digital, that assesses whether sufficient energy passes. Another example type of structure compares the difference waveform with stored information related to the reflectivity pattern of the particular eye, such as obtained from previous scans.

A second condition, satisfaction of which may keep the enabling signal(s) at the fail-safe shutter, relates to the level of energy being sent and/or the degree of focus of that energy. For instance, if the absolute level of energy as sensed by the sent energy sensor is below a threshold, or it is below a higher threshold related to a lack of focus measured by the returned energy sensor, then the signal remains enabled. Two or more safety monitors preferably communicate to check each other's operation and to leverage each other's resources. As one example, if one safety monitor withdraws enabling for its fail-safe shutter, then it preferably communicates to the other safety monitor a request to do likewise. As another type of example, one monitor preferably at a random and unpredictable time, requests such withdrawal of support by another monitor and then checks that the request was honored and then informs the other monitor that it was only a test. In a further type of example, one monitor requests from the other a sample vector of values recently received by the other monitor from its sensors and then compares these to the sensor values it has

83

„.. „ ' —-"33-001350 received itself, withdrawing enablement if the differences exceed pre-established thresholds.

Turning now to Figure 158, a combination flow, block, functional, schematic, diagram of an overall system in keeping with the spirit of the present invention is shown. There are five parts, Figure 158A-E, the first of which is the initial part and the part that is returned to by the other parts when they recognize that they may no longer be appropriate. The initial state or entry point is shown as "Start" box 500. Two parallel paths are shown originating from this point, to indicate that there are two autonomous so-called "processes" or concurrent interpretation paths in this example. One process is aimed at determining if there has been movement of the viewers head and reporting the relative amount of that movement. It comprises a repeat block 510 and "adjust position relative to head movement" block 511 that is repeated so long as the system continues to run from start 500.

The so-called "main loop" is shown with entry point "Reset" 510. An example initialization is the setting of the "volume" to be searched in to its small initial value. The position of the volume, not shown for clarity, is the last position where the eye was correctly tracked and the initial value is axial. Next repeat box 512 makes an unconditional loop of the remaining parts, with three exit points shown to be described. First within the loop is the "Measure within volume" box 513. This box attempts to locate the center of the viewer eye by searching within the volume. In this is preferably done by searching in order from the more likely locations to the less likely locations, as mentioned elsewhere. The location of the eye, the rotation of the eye, and optionally its focus are potential parameters of the search space. As will be understood, one example way to locate the eye is by identifying the pupil and measuring its location. So-called binary search or simple scan search, for example, may be more effective, depending on the characteristics of the mirrors.

After measurement 513 three tests are performed to determine where and if control should be handed off. The first test shown in the arbitrary but hopefully logical ordering is the "No movement" test 514. It tests for the more or less trivial case that the eye is in the same position as it was measured to be in the measurement preceding that of box 513. In case it is, as indicated by the "Y" for yes, the "Fixation" section will be entered through entry point 520. Similarly, the "Ballistic Motion" test 515 is directed at detecting if they new position of the eye represents an apparently

84 - 33-001350 large-scale ballistic motion from the previously measured position. In case it is, the "Saccade" entry point 540 is transferred to. And again, the previous two tests having failed, the "Eyelid closed" test 516 is performed. If the sensors report that the eyelid is occluding view of the eye, then control is transferred to the "Blink" entry point 560. Having failed the tests 514-516, the volume to be searched is increased, as indicated by box 517. This expansion of the volume is of course limited by the reach of the system. When the loop is again repeated, as indicated by box 512 as already described, the space of the measurements of box 513 is increased. It is believed that in this way the eye will eventually be located and the proper section transferred to. Turning now to Figure 158B, the fixation section is described as reached through entry point 520 already mentioned. The fixation section is a loop, as indicated by repeat block 521. A step includes rendering the image on the viewer's pupil by raster scanning or the like and at the same time measuring the returned energy as shown in block 522. Block 523 uses the returned energy to adjust the focus (or scanning across high-contrast regions repeatedly can be used for this as explained). There are also two tests. If the position has moved only a small amount, then test 524 is satisfied and control transfers to "Pursuit" entry point 580. If not, then a test for substantially zero movement 525 transfers to reset 501 already described. Otherwise, the loop 521 repeats. Turning now to Figure 158C, the "Saccade" section is described as reached through entry point 540 already mentioned. Again the section is a loop, this time headed by repeat block 541. Shown next is rendering 542 on the viewer's pupil of the input image, positioned so that the predicted location of the gaze point on the retina and in the image coincides. The rendering, however, is believed potentially "blurry" as very little detail is believed perceived by the viewer during saccades.

Next some predictions, measurements and adjustments are made. For example first is "Ballistic movement prediction" calculation 543, which attempts to fit the measurements it has collected into a ballistic trajectory and to predict the end point of the trajectory. Historical data related to the particular viewer is preferably used to tune this model. Measurements searching for the orientation of the eye, using the last predicted location as a starting point, are made according to box 544. The prediction is adjusted 545 based on measurement 544. (In subsequent

85

^"33-001350 iterations, prediction and adjustment are preferably combined, not shown for clarity.) Also, the focus is adjusted 546 if measurements and/or predictions indicate this.

Two tests are shown as examples. If the prediction 543 or 545 or measurement 544 determine that the eye has stopped moving, control is transferred by test 547 to fixation entry point 520. If the conclusion from the prediction efforts and measurements is not within parameters prescribed for a saccade, the process returns to the reset point 501.

Referring to Figure 158D, the "Blink" section is described as reached through entry point 560 already mentioned. Again the section is a loop, this time headed by repeat block 541. The search volume is increased 562 at each iteration. The volume is searched 563. If the eye is determined to be "open," that is no blink in progress, then control is transferred to the fixation entry point 565; otherwise, it remains in the loop.

Turning finally now to Figure 158E, the "Pursuit" section is described as reached through entry point 540 already mentioned. It is again a loop and directed at the phenomena known as "smooth pursuit" during which the eye travels slowly, typically following an object that is moving. It will be appreciated that this is an example where information from the content can assist in determining the likely behavior of the eye, although such data are not shown for clarity. The loop header block 581 is shown explicitly, as with the other diagrams.

The render and measure step 582 is similar to that already described with reference to block 522, as box 583 adjusting focus is to box 523. A movement to track the pursuit is indicated in box 584 as well as an adjustment or determination of the amount to move. Then a test 585 is made to determine whether the measured position from box 582 matches up with the predicted position. If yes, iteration of the loop continues; if no, reset entry point 501 is returned to.

Turning now to Figure 159, a combination block, functional, schematic, flow, and process architecture diagram of an overall system in keeping with the spirit of the present invention is shown. In this exemplary embodiment, various aspects of the inventive systems are each represented as an "engine" or substantially autonomous or otherwise separated rectangular "process" block. Exemplary communication paths between the blocks are indicated by slant-boxes, with arrows showing the flow direction(s) and content labels indicating the type of data

86 33-001350 transferred. At the center of the system is the "Control" box 600. It is shown taking input from some boxes, sending output to some boxes, and having bi-directional interaction with other boxes.

One input to control 600 is "Focus Engine" 610. Slant-box 615 indicates a type of message, shown as "focus distance," that is sent from focus engine 610 to control 600. Implicit in this system description is that focus engine 610 has an ability to make the measurements needed to determine changes in focus, and to alter the optical wavefront transformation to make the corresponding accommodation. One example use of this information at control 600 is to calculate so-called "vergence" angles between the eyes, such as when the control 600 for one eye is able to communicate with the control for the other eye of the same viewer, not shown for clarity. Another exemplary use of focus distance is in attempting to determine the landing point of a saccade. The focus distance is also shown being supplied, as a second output of slant-box 615, to the "Input Content" source 690, to be described in more detail below.

A second input to control 600 is "Head Motion Engine" 620. Slant-box 625 indicates a type of message, shown as "displacement," that is sent from head motion engine 620 to control 600. Displacement indicates the difference between viewer head positions relative to some reference position, such as an initial position or incremental re-synchronization position. The human eye is believed to in effect correct for such displacement by eye movement, in an effort to keep the image on the retina substantially unchanged during head movement. The so-called "gaze point," the point the person is looking at in the content is believed preferably to remain unchanged; however, the so-called "clipping" of the image portion displayed in the field of view of the viewer, changes as the field of view is shifted. It is believed that a movement of a spectacle form factor relative to the viewer's head, is also detected by a motion, since it is unlikely that the head will move and the spectacles remain fixed.

An output of control 600 that influences what is displayed, at least in the case when the viewer is looking continuously at a gaze point, is the "gaze point; clipping" output 635. This is shown supplied to both input content 690 and "Render Engine" 630. It substantially indicates the point in the content image the user is looking at and where that point is within the clipped field of view. In some examples, included in the

87

„..-, ,„. . '""33-001350 gaze point is the focus distance, as mentioned with reference to already described slant-box 615. Accordingly, "displayable content" slant-box 695 includes the content that render engine 630 is to display, such that the parts outside the clipping are omitted, the level of detail is adequate for the distance from the gaze point, and the focus distance is optionally accommodated.

A third input to control 600 is "Disruption Engine" 640. Slant-box 645 indicates a type of message, shown as "alerts," sent from disruption engine 640 to control 600. As already described with reference to Figure 157, an aspect of the function of safety engine 640 is to determine if there has been an interruption in the projection of images on the retina. A related kind of disruption anticipated is movement of the pupil or change in the relative position of the system to the viewer head. Such changes are example alerts. Start of continuous viewing is also considered an example type of alert.

A first example engine for which control 600 communicates in the example bi- directionally is "Eye Search Engine" 650. This engine seeks to find the position of the center of the eye. In the example shown, slant-box 655 indicates that the portion of the space, indicated as "volume," over which the search is to be constrained is supplied by control 600 to eye search engine 650. Examples of information characterizing volume include such things as a bounding box, rectangle, or other shape in a two- or three-dimensional coordinate system in the frame of reference of the front optic. Other examples include parameter ranges, such as focus and/or astigmatism ranges. Further examples include hints or clues, such as last know find or projected or likely finds or probability distributions on such finds. For instance, during a blink, if fixation is suspected of being maintained, there is very high probability of a particular location, however, a volume does grow in case of saccade. The result of a successful find of the eye is shown in slant-box 655 as "coordinates," although additional information may be included. In some examples, information may include such things as pupil diameter, degree of eye occlusion, and so forth. The second example engine for which control 600 communicates in the example bi-directionally is "Model Engine" 660. An aspect of the function of model engine 660 is to provide analysis of data related to the position and disposition of elements in the system and related to the viewer, including basing predictions on data collected earlier. For instance, calculating the position of the eye axis and the

88

~ ' ""'""33-001350 distance to the eye and the gaze point and the clipping are examples of functions that can be performed by the model engine 660. Output can, in some examples, include coordinates describing the axis of the eye and the focus distance. In other examples, outputs include probabilities based in some examples on the past behavior of the viewer, such as various speeds and ranges, positions of apparatus relative to the head, and so forth. The historical data base for such probabilities is shown as "Database" 669. The data communicated between database 669 and model engine 660 is shown as the slant box "coordinate and measurement history" 667. As one example, what may be called "zone reflector" schemes particularly well suited to so-called "peripheral" portions of the retina have been proposed by the present applicant. Such schemes handle a fixed number of fixed eye positions each with a different set of mirrors and make adjustments for actual eye positions that lie between those fixed positions. For any particular such actual position the amount of adjustment to the nearest fixed is believed, however, to vary for differing locations on the eyeglass lens, due to asymmetry in the geometry.

As another example, so called "major reflector" schemes, believed well suited to so-called "macular and foveal" portions of the retina, in which reflectors used at particular instants are substantially those located around the line of sight, have been proposed by the present applicant. Adjustment of the launch position, at least in increments, into alignment with the front-optic reflector(s) closest to the point of regard is preferable. Also, for such front-optic reflectors closest to the point of regard, such as are believed applicable for instance to the para-foveal or macular regions of the retina, there is a distance between launch locations and some variation due to geometry that depends on where on the eyeglass lens the reflectors are located.

Various exemplary inventive aspects are disclosed here to address the above and provide further objects, features and advantages as will be appreciated from the description and drawing figures.

Turning now to Figure 160, an exemplary arrangement for sourcing light from varying positions to front-optic mirrors in a reflector zone system is shown in a combination optical schematic and block diagram in accordance with the teachings of the present invention. The "source" is preferably an emitter of light with high radiance, such as for example a laser or so-called vcsel.

89 33-001350 The next element in the chain is the "beam expander," being well known and in some examples acting like a telescope in reverse, producing a substantially collimated beam output (not shown for clarity). In some examples such a beam expander optionally has a variable amount oval correction, such as using variable cylinder lenses or other variable lenses with asymmetry as are know constructed using electrowetting of immiscible fluids. Such oval correction may be desired to compensate for the effects related to obliquity of the eyeglass lens interface relative to the light directed at and reflected from particular reflectors in such front optics. The next element in the example chain is a "spatial light modulator," as are known. The larger beam coming from the output of the beam expander and impinging on the spatial light modulator is accordingly divided into a number of smaller beam portions each potentially separately temporally modulated in an all or nothing or so-called "grey level" fashion, as is known.

Preferably after or in combination with the spatial light modulator is the "first active mirror structure." While this is shown in a transmissive configuration, a reflective configuration is more typical. (Such schematics not being intended, as will be understood, to indicate which of such configurations apply to the components for generality and clarity in exposition.) These mirror structures steer the portions of the expanded beam separately to mirror elements in the "second active mirror structure" as indicated by some exemplary principle rays. This second mirror structure in turn preferably directs the portions of the beam towards the front optic reflectors, as shown. Subsequently, these portions of the beam are preferably reflected by the front-optic reflector structures and directed at least in part into the pupil of the eye not shown for clarity. When multiple mirrors are used to reflect a single collimated beam footprint, as variously contemplated here, it is well known in the art, and famously for multi- mirror telescopes, that the mirrors are preferably arranged at distances that are at least close to a multiple of the wavelength of light involved. So-called mirror "piston" is preferably also controlled to adjust the height of the mirrors accordingly. Without suitable such measures a loss in resolution may be obtained.

In operation, the first active mirror structure selects elements of the second active mirror structure to determine the "origin" or "source location" of the beam portions. Then the second active mirror structures steer the light successively to the front optic mirrors, such as the so-called "minor" mirrors. The motion is preferably "point to point," "continuous scan," and/or "scan with pause," such as are known in the art and depend variously for instance on the amount of time, power, and mirror characteristics. An example scan pattern will be described. The light is shown launching from a range of locations, so as to provide the effect of entering the pupil with a range of angular content sufficient to provide connected images as has been described in co-pending applications as already included herein. In a reflector zone configuration, the mirrors of a single zone are illuminated in order to provide the light that enters the pupil. (As will be appreciated, the spacing of the fixed locations near the pupil of the eye may be such that only one enters the pupil at a time or there may in some example embodiments be multiple fixed locations that can enter a particular pupil location.)

In another operational aspect, the location from which energy is launched may be desired to be varied depending on the geometry of the particular regions of the eyeglasses lens being covered in order to compensate for position adjustment so as to enter the pupil of the eye directly, as mentioned above. Since such changes may be desired to be made substantially during the scanning of minor mirrors, a novel variation may be employed: Additional mirrors in the second active mirror structure track along with the mirrors being used to source the light and they are brought into play, and light shifted to them from some of those being used to steer the light for other portions of the eyeglasses lens, by changing the modulation of the corresponding portions of the beam at the spatial light modulator. Accordingly, as will be appreciated, this is believed potentially to result in rapid changes in the effective launch location of the light during the scanning process and optionally even in a way that does not depend on special mirror movement but rather only spatial light modulator changes, which are believed in at least some technologies to be substantially faster.

Color is preferably provided, such as by multiple sources of primary or other colors composing desired color gamuts and/or the re-use of the optical chain for several color components in parallel or sequentially. (Such color rendering techniques are not shown or described further for this or other embodiments for clarity as they would be understood.)

91

"33-001350 Referring now to Figure 161 , an exemplary arrangement for sourcing light from varying positions to front-optic mirrors in a point-of-regard system is shown in a combination optical schematic and block diagram in accordance with the teachings of the present invention. Those aspects of this figure that are substantially similar to those already described with reference to Figure 160, as will be appreciated, are here abridged or omitted for clarity. For instance, the source, first beam expander and first spatial light modulator are substantially the same as the source, beam expander and spatial light modulator, respectively, as already described with reference to Figure 160 above. The "pre-combining mirror structure," in some exemplary embodiments, is substantially a single active mirror matrix and in other examples, as indicated by the vertical dotted line, may include multiple reflections for each of plural portions of the light transmitted. In a preferred configuration, a multi-pixel "paintbrush" is in effect formed from a substantially linear arrangement of beam origin points where all the beams are aimed substantially at the center of the input of the "second beam expander" to be described. One example structure to deliver such light would be a single row of mirrors, optionally dynamic mirrors that may be used for other purposes at other times. However, a single substantially round source may be more economically fabricated, and the spatial light modulators may be more readily fabricated in structures with more square aspect ratios. Accordingly, a multiple reflector arrangement preferably provides origin points along a line from modulator locations arranged in multiple rows. To accomplish this, a first reflector takes a beam portion to a second reflector, the second reflectors being arranged along a line. In some examples the first reflectors are dynamic mirrors and can be re-purposed for other configurations.

The "second beam expander" expands the input to produce an output beam of substantial width that contains rays with angular components related to the angular content of the input. It is believed, consistent with the so-called optical invariant, that the angles of rays in the output will be substantially smaller than on input. The output beam impinges on the "second spatial light modulator" and the results impinge on the "pre-launch mirror array" as indicated. In the example configuration it is believed that the second spatial light modulator pixels each

92

"13-001350 modulate multiple image pixels and so are used more as a way to gate light to particular pre-launch mirror locations.

The example use of the structures shown is indicated by example beams shown between the pre-launch mirror array and the example front optic mirrors, for clarity. The example shows all the pre-launch mirrors having substantially the same angle and resulting in the "potential beam envelope towards eye" shown as a beam with dotted boundary lines. The potential envelope is limited or gated by the second spatial light modulator to limit the output to what will be called "beamlets" that are in effect portions of or sub-beams of the overall beam envelope. Such gating is aimed at reducing the amount of light that spill onto other mirrors and structures, although other techniques to reduce undesirable aspects of such light may be employed and the need for this gating function removed. Both an example "first beamlet towards eye" and a "second beamlet towards eye" are shown, each impinging on a respective "front optic mirrors" shown. The geometry of such beam steering is also shown further in Figure 162 to be described.

Turning now to Figure 162, exemplary beam steering configurations are shown in a combination optical schematic ray trace diagram in accordance with the teachings of the present invention. An example arrangement comprising two example beamlets is shown in Figure 162A and two different mirror angle examples are compared in figures 162B and 162C. This arrangement was already shown and described with reference to Figure 161.

Referring now to Figure 162A, the "mirror array" is shown reflecting beams of light. The beams are indicated by boundary lines. The "source beam envelope" is shown as a dotted line and including end caps for clarity in the diagram, as will be appreciated. Each mirror in an example mirror array section is shown, although the number of mirrors may be significantly larger than the few shown for clarity in the diagram. The "first beamlet from source" (shown is solid lines [red in color versions]) and the "second beamlet from source" (shown as dashed lines [blue in color versions]) are shown arriving at the same angle and as part of the source beam envelope. This sourcing angle is not changed in the examples described for clarity. After impinging on the mirrors of the mirror array, the "beam envelope towards eye" emerges. Included in the beam envelope towards eye are the "first beamlet towards front optic" and the "second beamlet towards front optic" as shown. Referring to Figure 162B and 162C, two unequal mirror angles θ and θ¹ are shown, respectively. While the input angles are unchanged, as mentioned earlier, the position of the single example beamlet within the input beam envelope is raised by Δ' measured along the vertical shown. This can be seen, through the dotted construction lines [light blue in color versions] provided for clarity, to result in a change in vertical height at the envelope cap distance of Δ.

Turning now to Figure 163, exemplary minor scanning of front optic mirror structure is shown in a combination plan and schematic view in accordance with the teachings of the present invention. The layout already is here depicted here with a particular scan pattern example for a zone. The "scan lines for example minor zone" are shown as solid horizontal bars [light blue in color versions] that cover the minor mirrors of that zone. Of course scan rows can be arranged in various directions and patterns, not described for clarity and without limitation.

Turning now to Figure 164, exemplary major scanning of front optic mirror structure is shown in a combination plan and schematic view in accordance with the teachings of the present invention. Beginning with an initial horizontal "first scan row" and followed by a "second scan row" all the way to a fifth scan row are shown covering the concentric dotted circles and mirror ring labels already described with reference to Figure 163. The scanning pattern is shown with an example "initial position range, shown outlined in dashed lines and shaded deeper [darker blue in color versions' along with a similarly illustrated "final position range." Although the initial and final positions suggest an example "width" of the beam from the second beam expander described with reference to Figure 161, they may not actually be realized as fixed mirror positions in case resonant modes of mirror are used or other dynamic scanning method. It will be appreciated, however, that the starting and final positions are shown as chosen so that even the mirrors on the edge of the concentric circles are able to receive the full range of delivery angles, which is preferred and may be an unnecessary requirement shown for clarity.

Turning now to Figure 165, exemplary tilt pan system is shown in a combination optical schematic, block, and section diagrams are shown in accordance with the teachings of the present invention. Initial, middle, and later views are provided in Figures 165A, 165B, and 65C, respectively. The "active mirror," could for instance be a part of the "second active mirror structure" of Figure

94

^„.._ r. ""«'««33.001350 160 or of the "pre-launch mirror array" of Figure 161. The "spatial light modulator," or SLM for short, could for instance be the "SLM" of Figure 160 or the "second SLM" of Figure 161.

The source region on the SLM is the portion of the SLM that in effect tracks the "target regions" as the active mirror is rotated. In some examples, the SLM is performing a gating function and lets only the light for the "beam" through; in other examples it provides images by modulating individual pixels. In the former case, the pixels are formed by an upstream modulator, such as the "first SLM" of Figure 161, and those pixels preferably "track" or put differently are spatially shifted to follow or pan along in synchronization with movement of the active mirror. In the latter case, the pixels generated by the SLM are moved along its surface so that they remain in substantially the same alignment with the beam over the range of angles introduced by the active mirror. Thus, as will be seen the "target region" receives substantially the same pixels over the range of the scan, from Figure 165A, through Figure 165B, to that of Figure 165C. Operation of the figure will be further described with reference to Figure 166.

Turning now to Figure 166, operation of an exemplary tilt pan system is shown in a combination block and flowchart diagram in accordance with the teachings of the present invention. The chart shows a single instance of the operation for a single mirror and target region. More generally, multiple instances may occur in parallel and or sequentially and/or spatially separated, as will be understood.

The loop or block of operations is repeated some number of times in the example. Each time the mirror angle is moved. Indicated is a stepwise movement of the mirror; however, in many embodiments the mirror inertia makes the steps into a continuous movement. The SLM, typically, moves in discrete steps, although continuous motion may be possible in some technologies. The movement of the two elements is coordinated, such as being controlled from synchronized algorithms, table look-ups, or feedback loops. A description of an exemplary aspect is now provided but without any limitation with reference to Figure 167. The display includes a pixel source and optical elements that provide light with the needed angular content to multiple reflective system elements. Time-division multiplexing by displaying multiple images

Λ3-001350 95 within each frame time interval is accomplished by switching on reflective system elements at the appropriate times to selectively reflect into the pupil of the eye. In some examples a LCD shutter is formed between a first fixed polarizer mounted near the projection system and a separately switchable liquid crystal layer located adjacent to a second fixed polarizing layer. By varying the polarizing effect of the liquid crystal as in known integrated LCD shutters, the light from the source is either allowed to reach and return from the mirror or it is substantially absorbed. Light from the environment is attenuated by the filter it is incident on but substantially not subject to the shutter effect as sourced light. In one example the adjacent reflector system elements provide beams that are smaller than the pupil but substantially parallel when they are at their corresponding extreme positions entering the pupil, thereby providing for the pixels that originate at points at the interface of the two elements. Other pixel origins are included beams at angles ranging between the extremes. The pixel source can be any type of display means, such as OLED, transmissive SLM or reflective SLM illuminated, for example, by LED, VCSEL or laser, shown only in section schematically as a narrow rectangle. Various pixel areas are shown comprising various subsets of the pixels on the pixel source. At a single example instant in time, a single pixel area may be illuminated. Its position on the pixel source determines the angles that correspond to each of its pixels in the beams projected towards the front optic, as will be described, and by varying the placement of the pixel area the angular content of the projected light is varied so that it meets the requirements of the beam to be reflected by the reflector system element into the pupil. The area on the pixel source of some beams may be disjoint and other may overlap, as illustrated by example instances.

Light leaving the pixel source is preferably substantially passes through an optical system shown as a single optical element, a lens in the example. Whatever the optical system its function is to bring the light from each pixel into a substantially parallel beam directed at least toward the relevant reflector system elements. In particular it will be appreciated that the optical element shown provides light corresponding to the each pixel from the pixel source to plural reflector systems at the same time; however, the switching on of a single mirror or a limited number of mirrors is anticipated to control from which reflector light reaches the pupil of the eye.

96 33-001350 After leaving the optical element light is shown passing through a polarizing filter. Of course the filter could be on either side of or included in the optical system, affixed to the pixel source, and/or laminated onto the reflector system. In the example of a very well known type of LCD shutter, the liquid crystal is laminated between two fixed linear polarizers, each oriented the same way as the lines on the corresponding surfaces in contact with the liquid crystal. Application of a voltage perpendicular to the layers untwists the liquid crystal and blocks the light. Many other arrangements are known, including where electrodes are provided at the ends of the layer, and accordingly the electrodes are not shown for clarity. Embedding the reflector system elements preferably into the "lens" or "front optic" of the spectacle is anticipated. In some examples such arrangements are fabricated by steps included forming two separate front and rear halves of the spectacle lens, coating the inner surface of at least one of them and combining them into a single unit such as by application of optical cement or the like. Whatever driving technology may be applied and corresponding conductive paths and optionally active elements are preferably included in the embedded layers. In some examples the switching is controlled and powered by autonomous active elements and the switching is detected by optical feedback sensors located on the return path, such as after a beam splitter located just after the pixels, not shown for clarity. In other examples the pixel source and the switching are controlled and powered by the same system. For instance, conductive paths can be established from the front optic to the frame and to the projection controller. In other examples only two conductive paths are provided, such as one from each surface of the front optic, so as to facilitate interconnection. One or more active controller elements would be located within the front optic and known techniques for providing power and signal over the same pair can be employed as would be understood.

Various patterns of reflector system elements are anticipated. Some examples include more than one size, as indicated. The small size preferably provides low resolution images to the peripheral portions of the retina. More than one collection or "zone" of such small reflectors can be provided and each aimed so as to correspond to a particular eye position. It is believed that in some configurations the small reflectors of a zone can be activated at the same time. The larger reflectors are preferably oriented to associate the angles corresponding to the eye rotated to look

^„..- ~ "_"33-001350 97 directly at it and the center pixels of the display. For a particular eye rotation, the nearby larger reflectors are preferably also used; however, the angular content provided them when they are selected is preferably such that the resulting beam lands in the pupil. In operation, different portions of the image to be projected onto the retina are provided substantially separately to different reflector system elements, in some examples at substantially different time slices within the frame. Frames are preferably every sixty to 120 per second. The "tiling" of the images on the retina is preferably arranged so that a seamless foveated image results, as disclosed in co- pending applications by the applicant already included here by reference. When light intended for a first mirror that impinged on a second mirror would not enter the pupil and vice versa, the system may overlap in time the projection from the two mirrors. Similarly for more than two mirrors.

The example shown is in terms of very standard types of LCD shutters. However, many types of shutters are known and they could readily be employed as would be understood.

Moreover, switchable mirrors are known. For example, there are those based on so-called "Bragg" effect, such as have been disclosed by Hikmet and Kemperman in an article entitled "Switchable mirrors of chiral liquid crystal gels," that appeared in Liquid Crystals, Volume 26, Number 11 , 1 November 1999, pp. 1645-1653 along with a variety of related articles referencing and referenced by this article in the literature, including those based on so-called blue phase, all of which are included here by reference. Other examples are based on other effects, such as those disclosed in US patents 5,251 ,048, 6,359,673, 5,875,012, 5,847,798, and 6,034,752. When a switchable mirror is employed, that can be changed between transparent and reflective, it is believed that an advantage is that less light is blocked since polarizing shutters block half the light as a result of the fixed polarizer that incoming light is incident on initially. Furthermore, when switchable mirrors are employed, more than one mirror may overlap or be layered in the front optic. This is believed to allow higher resolution images with smaller pupils.

As a still further advantage of switchable mirrors, the angular variation or some of the range of variation of the source means is optionally accomplished by the

98

""33-001350 selection of differently oriented mirrors, such as those in substantially the same angular position on the front optic.

Another exemplary embodiment of what may be called a "switchable mirror" is a so-called "electrically switchable hologram." These are known in the art, commercially manufactured, and disclosed for example in US Patent 6,677,086, titled "Switchable volume hologram materials and devices" by Sutherland, et al., issued January 13, 2004, and all the patents that reference it. Such switchable holograms in some example embodiments implement holographic optical elements that perform the function of a mirror, in some instances using what is known as a Bragg mirror structure.

The overlapping patterns shown in illustrate example ways that all points can be covered by at least one mirror, diffractive, hologram and/or other redirector structure in a three layer construction. In one aspect it is believed that such structures allow beams of certain sizes to be in effect re-directed by in effect a single planar structure, for any center point of the beam impinging on the structure. Said differently, it is believed that, for beams of certain diameters and angles, at least one of the three mirror layers will contain a mirror that allows the whole beam to be redirected.

Referring specifically to Figure 168A-C, three example views are provided of so that the exemplary structure of three substantially round redirectors overlapping can be more readily seen, as will be appreciated. Figure 168A shows only of the three layers separately. Figure 168B then shows two of the layers composed, the additional one in dotted lines. And finally, Figure 168C illustrates the composition of the three layers, the additional one in dashed lines. Referring now specifically to Figure 169A-C, three example views are provided so that the exemplary structure of three substantially rectangular redirectors overlapping can be more readily seen, as will be appreciated. Again Figure 168A shows only of the three layers separately; Figure 169B then shows two of the layers composed, the additional one in dotted lines; and finally, Figure 168C illustrates the composition of the three layers, the additional one in dashed lines.

Turning finally to Figure 170, exemplary redirector arrangements are described. When the structure preferably a part of the proximal optic that re-directs the incident light toward the eye — in some examples oriented in a substantial plane physically related to its angle or in other examples formed as a volume hologram and occupying another physical substantial plane such as a continuous surface shared by multiple such redirectors — is wider than the beam width being used at least at a particular time, then it may be advantageous to "walk" the beam across the redirector while limiting the motion of the center of the beam relative to the eye pupil, such as holding it fixed on a desired portion of the eye pupil. An example for clarity shows some exemplary beams for the latter case, where beams are kept at a substantially fixed center point on the pupil.

More particularly, the beams drawn in solid lines impinge on one portion of the redirector structure, while those drawn in dotted lines impinge on another portion, resulting in an angular variation between the beams entering the eye and the potential rendering of different pixels. A preferred embodiment realizes a range of discrete positions for the beam center impinging on the redirecting structure in order to create a corresponding set of pixels on the retina of the eye. In other examples some positions on the redirecting structure have beams of multiple launch angles impinging on them, resulting in multiple beam center points on the plane of the eye pupil. Such arrangements in some examples comprise discrete steps with multiple angles on the redirector and/or discrete steps in the plane of the eye pupil with multiple differing positions on the redirector and/or unique points on the redirector and on the eye pupil plane per beam.

The projection structure for sourcing these beams is, as will readily be appreciated and understood by those of ordinary skill in the optical arts, substantially similar to that disclosed elsewhere here for maintaining the center of the beam at a fixed location on the redirector structure. Said differently, the beam is projected as if the distance to the redirector structure is the distance to the pupil of the eye and the beam center is fixed there. In other example embodiments a redirector structure acts as an aperture for a wider beam incident on it whose angle is varied, resulting in a wider beam impinging on the eye and in some instances such beams may be partly occluded by the iris and sclera of the eye.

The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention, in the use of such terms and expressions, to exclude equivalents of

100

~..— "'""33-001350 the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims that follow:

101

"33-001350

Claims

CLAIMSWhat is claimed is:

1. A display system for projecting an image onto the retina of an eye, comprising: a light source that produces at least one beam of light modulated in accordance with image data received thereby; at least one optical scanner that receives said beam of light and retransmits said beam with a selected trajectory; a proximal optic disposed at a distance from said optical scanner, said proximal optic comprising a diffractive optic having a plurality of distinct redirection areas that correspond to and intersect respective trajectories of said beam and retransmit said beam with respective selected altered trajectories; and a frame adapted to hold said proximal optic in a determined position in front of and relative to an eye and to said optical scanner, said optical scanner and said eye being disposed on the same side of said proximal optic, so that said beam can be directed from said light source by said optical scanner and said proximal optic into the eye with said selected altered trajectories through the pupil thereof.

2. The display system of claim 1 , wherein said beam of light is substantially collimated from said scanner to the cornea of the eye.

3. The display system of claim 2, wherein said light produced by said light source has a narrow bandwidth.

4. The display system of claim 3, wherein said distinct redirection areas comprise Bragg reflectors.

5. The display system of claim 4, wherein said light source comprises one or more lasers.

6. The display system of claim 5, wherein said lasers comprise laser diode.

7. The display system of claim 1 , wherein said scanner comprises a first tillable mirror and a second tiltable mirror, said first tiltable mirror receiving said beam of light and reflecting said beam of light toward said second tiltable mirror and said second tiltable mirror reflecting said beam of light along said selected trajectory.

8. The display system of claim 7, wherein said beam of light is substantially collimated from said scanner to the retina of the eye.

9. The display system of claim 8, wherein said beam of light produced by said light source has a narrow bandwidth.

10. The display system of claim 9, wherein said distinct redirection areas comprise Bragg reflectors.

11. The display system of claim 10, wherein said light source comprises one or more lasers.

12. The display system of claim 11 , wherein said lasers are laser diode.

13. The display system of claim 1 , wherein said distinct redirection areas comprise Bragg reflectors.

14. The display system of claim 13, wherein said diffractive optic is a volume hologram.

15. The display system of claim 14, wherein said beam of light is substantially collimated from said scanner to the cornea of the eye.

16. The display system of claim 15, wherein said light produced by said light source has a narrow bandwidth.

17. The display system of claim 13, wherein said redirection areas comprise respective interference patterns that retransmit said beam of light with an altered trajectory that is dependent on the position and angle at which said beam of light with said first trajectory strikes a given redirection area.

18. The display system of claim 17, wherein said light produced by said light source has a narrow bandwidth.

19. The display system of claim 18, wherein said beam of light is substantially collimated from said redirector to the cornea of the eye.

20. The display system of claim 19, wherein said beam of light comprises plural multiple wavelength components and said diffractive optic is a volume hologram.

21. The display system of claim 1 , wherein said light source produces a first beam of light modulated in accordance with image data received thereby and a second beam of light modulated in accordance with image data received thereby, and said system comprises a first optical scanner that receives said first beam of light and retransmits said first beam with a first selected trajectory, and a second optical scanner that receives said second beam of light and retransmits said second beam with a second selected trajectory, said proximal optic having a first plurality of distinct redirection areas that correspond to and intersect respective trajectories of said first beam and retransmit said first beam with first selected altered trajectories, and having a second plurality of distinct redirection areas that correspond to and intersect respective trajectories of said second beam and retransmit said second beam with second selected altered trajectories, said second beam being directed from first light source by said first scanner and said proximal optic into the eye with said first selected altered trajectory through the pupil thereof at a first selected angle relative to the optical axis of the eye, and said second beam being directed from second light source by said first scanner and said proximal optic into the eye with said second selected altered trajectory through the pupil thereof at a second selected angle relative to the optical axis of the eye.

22. The display system of claim 21 , wherein said second plurality of distinct redirection areas are smaller in area than said first plurality of redirection areas, and said first plurality of redirection areas are distributed among said second plurality of redirection areas.

23. The display system of claim 22, wherein said first beam is substantially collimated from said redirector to the cornea of the eye and said second beam has a smaller diameter than said first beam and diverges significantly from said scanner to the retina of the eye so as to produce a spot size on the retina larger than the spot size of said first beam.

24. The display system of claim 23, wherein said light produced by said light source has a narrow bandwidth.

25. The display system of claim 24, wherein said first and second redirection areas comprise Bragg reflectors.

26. The display system of claim 25, wherein said light source comprises one or more lasers.

27. The display system of claim 26, wherein said lasers comprise laser diode.

28. The display system of claim 23, further comprising a first control system for causing said first scanner to vary said first selected trajectory so that said first selected altered trajectory scans a foveal region of the eye.

29. The display system of claim 28, wherein individual redirection areas of said first plurality of redirection areas correspond to respective tiles of retinal pixels on the foveal region of the eye, and said first selected trajectory may be varied to intersect an individual redirection area so as to illuminate individual pixels within the corresponding tile.

30. The display system of claim 29, further comprising a second control system for causing said second scanner to vary said second selected trajectory so that said second selected altered trajectory scans a peripheral region of the eye.

31. The display system of claim 30, wherein individual redirection areas of said second plurality of redirection areas correspond to respective tiles of retinal pixels on the peripheral region of the eye, and said second selected trajectory may be varied to intersect an individual redirection area so as to illuminate individual pixels within the corresponding tile.

32. The display system of claim 28, further comprising a second control system for causing said second scanner to vary said second selected trajectory so that said second selected altered trajectory scans a peripheral region of the eye.

33. The display system of claim 32, wherein individual redirection areas of said first plurality of redirection areas correspond to respective tiles of retinal pixels on the foveal region of the eye, and said first selected trajectory may be varied to intersect an individual redirection area of said first plurality of redirection area so as to illuminate individual pixels within the corresponding tile, and individual redirection areas of said second plurality of redirection areas correspond to respective tiles of retinal pixels on the peripheral region of the eye, and said second selected trajectory may be varied to intersect an individual redirection area of said second plurality of redirection areas so as to illuminate individual pixels within the corresponding tile.

34. The display system of claim 32, further comprising a photodetector positioned so as to detect light reflected from the retina of the eye so as to determine the direction of the optical axis of the eye, and a pupil-tracking control system responsive to said photodetector for enabling said first optical scanner and said second optical scanner to select the first selected trajectory and the second selected trajectory, respectively, so as to take into account the direction of the optical axis of the eye.

35. The display system of claim 34, wherein the light detected by said photodetector to determine the direction of the optical axis of the eye is light from one of the said first beam or said second beam reflected from the retina.

36. The display system of claim 1, further comprising a scanner control system for causing said optical scanner to vary said selected trajectory so that said selected altered trajectory scans a region surrounding the fovea of the eye.

37. The display system of claim 36, wherein individual redirection areas of said plurality of redirection areas correspond to respective tiles of retinal pixels on the foveal region of the eye, and said selected trajectory may be varied to intersect an individual redirection area so as to illuminate individual pixels within the corresponding tile.

38. The display system of claim 1, further comprising a scan control system for causing said optical scanner to vary said selected trajectory so that said selected altered trajectory scans a peripheral region of the eye.

39. The display system of claim 38, wherein individual redirection areas of said second plurality of redirection areas correspond to respective tiles of retinal pixels on the peripheral region of the eye, and said second selected trajectory may be varied to intersect an individual redirection area so as to illuminate individual pixels within the corresponding tile.

40. The display system of claim 1 , further comprising a photodetector positioned so as to detect light reflected from the retina of the eye so as to determine the direction of the optical axis of the eye, and a pupil-tracking control system responsive to said photodetector for enabling said optical scanner and said to select the selected trajectory so as to take into account the direction of the optical axis of the eye.

41. The display system of claim 40, wherein the light detected by said photodetector to determine the direction of the optical axis of the eye is light from one of the said first beam reflected from the retina.

42. The display system of claim 1, wherein said frame is an eyeglasses frame, said proximal optic is attached to the surface of an eyeglass lens closest to the eye is when the frame is worn, and the optical scanner is attached to an arm of the eyeglass frame.

43. A display system for projecting an image onto the retina of an eye, comprising: a light source that produces at least one beam of light modulated in accordance with image data received thereby; an optical scanner that receives said beam of light and retransmits said beam with a selected trajectory; a proximal optic disposed at a distance from said optical scanner in the trajectory of said beam, said proximal optic having a plurality of distinct redirection areas that receive said beam, where the beam has a selected first trajectory corresponding to a selected one of said respective redirection areas, and retransmits said beam with a selected altered trajectory; and a frame adapted to hold said proximal optic in a determined position relative to an eye and to said optical scanner so that said beam can be directed from said light source by said scanner and said proximal optic into the eye with said selected altered trajectory through the pupil thereof at a selected angle relative to the optical axis of the eye.

44. A display system for projecting an image onto the retina of an eye, comprising: a light source that produces a first beam of light modulated in accordance with image data received thereby and a second beam of light modulated in accordance with image data received thereby; a first optical scanner that receives said first beam of light and retransmits said first beam with a first selected trajectory; a second optical scanner that receives said second beam of light and retransmits said second beam with a second selected trajectory; a proximal optic having a first plurality of distinct redirection areas that correspond to and intersect respective trajectories of said first beam and retransmit said first beam with first selected altered trajectories, and having a second plurality of distinct redirection areas that correspond to and intersect respective trajectories of said second beam and retransmit said second beam with second selected altered trajectories; and a frame adapted to hold said proximal optic in a determined position in front of and relative to an eye and to said optical scanner, said optical scanner and said eye being disposed on the same side of said proximal optic, so that said first beam can be directed from first light source by said first scanner and said proximal optic into the eye with said first selected altered trajectory through the pupil thereof at a first selected angle relative to the optical axis of the eye, and said second beam can be directed from second light source by said first scanner and said proximal optic into the eye with said second selected altered trajectory through the pupil thereof at a second selected angle relative to the optical axis of the eye.

45. The display system of claim 44, wherein said second plurality of distinct redirection areas are smaller in area than said first plurality of redirection areas, and said first plurality of redirection areas are distributed among said second plurality of redirection areas.

46. The display system of claim 45, wherein said first beam is substantially collimated from said scanner to the retina of the eye and said second beam has a smaller diameter than said first beam and diverges significantly from said scanner to the retina of the eye so as to produce a spot size on the retina larger than the spot size of said first beam.

47. The display system of claim 46, wherein said light produced by said light source is narrow bandwidth.

48. The display system of claim 47, wherein said first and second redirection areas comprise Bragg reflectors.

49. The display system of claim 48, wherein said light source comprises one or more lasers.

50. The display system of claim 49, wherein said lasers comprise laser diode.

51. The display system of claim 44, further comprising a first control system for causing said first scanner to vary said first selected trajectory so that said first selected altered trajectory scans a foveal region of the eye.

52. The display system of claim 51 , wherein individual redirection areas of said first plurality of redirection areas correspond to respective tiles of retinal pixels on the foveal region of the eye, and said first selected trajectory may be varied to intersect an individual redirection area so as to illuminate individual pixels within the corresponding tile.

53. The display system of claim 51 , further comprising a second control system for causing said second scanner to vary said second selected trajectory so that said second selected altered trajectory scans a peripheral region of the eye.

54. The display system of claim 53, wherein individual redirection areas of said second plurality of redirection areas correspond to respective tiles of retinal pixels on the peripheral region of the eye, and said second selected trajectory may be varied to intersect an individual redirection area so as to illuminate individual pixels within the corresponding tile.

55. Enhanced eyeglasses, providing a configurable, wearable platform for integrating or attaching electronic devices, comprising: a partially transparent, proximal optic that may accommodate a patterned matrix of integrated optical redirectors for guiding a light beam into an eye; a lens frame for mounting the proximal optic lens, the lens frame accommodating a plurality of integrated or attached electronic devices that may communicate with each other and with the wearer; and an arm attached to the lens frame, the arm capable of positioning the lens frame in proximity to and in a line of sight of the eye, and accommodating a plurality of integrated or attached electronic devices that may communicate with each other, with the lens frame devices, and with the wearer.

56. The enhanced eyeglass of claim 55, wherein the patterned matrix of integrated optical redirectors includes sets of redirectors having a common diameter and guiding propagation of a reflected light beam portion in accordance with sensed eye movement and variation in pupil aperture size.

57. The enhanced eyeglass of claim 55, wherein each reflected light beam portion forms an image tile on the retina of the eye, the redirector diameter determining a resolution, a field of view, and an associated spot size of said image tile.

58. The enhanced eyeglass of claim 55, wherein the integrated or attached electronic devices include lasers, light beam projectors, optical elements for directing light beams, electronic sensors, transducers, piezoelectric devices, audio transmitters, micro-power switches, electrical wiring or other means of signal interconnection.

59. The enhanced eyeglass of claim 55, wherein the integrated electronic sensors include pressure sensors for receiving data input.

60. The enhanced eyeglass of claim 55, wherein an integrated or attached electronic device includes a mechanical switch that activates power to other such devices in response to manipulating a portion of the frame or arm.

61. The enhanced eyeglass of claim 55, wherein the redirectors are selected from the group of integrated mirrors, partially-silvered mirrors, Bragg mirror structures within a volume hologram, and dichroic lenses.

62. The enhanced eyeglass of claim 55, wherein lens further comprises one or more holographic materials.

63. The enhanced eyeglass of claim 56, wherein the types of redirectors include foveal redirectors and peripheral redirectors.

I l l

64. The enhanced eyeglass of claim 63, wherein tiles formed on the retina by said foveal redirectors and by said peripheral redirectors have different sizes and resolutions.

65. The enhanced eyeglass of claim 63, wherein tiles formed on the retina by said foveal redirectors and by said peripheral redirectors occupy different spatial regions of the retina associated with different levels of visual acuity.

66. A display system for providing images to at least one eye, comprising: at least one source of light; means for modulating the light from the source responsive to input image data; means for projecting the modulated light separately to each of plural redirecting structures; each of the plural redirecting structures arranged to receive beams of the modulated light directed as input to it by the projecting means and to form from those input beams corresponding output beams; each of the plural redirecting structures configured to be positionable in view of the at least one eye; so that almost every output beam of a redirecting structure has an angle different from the angle of the corresponding input beam to that redirecting structure; so that the redirecting structures differ in that if each is provided the same input beam angle each would produce a different output beam angle; and so that when the plural redirecting structures are positioned in view of the at least one eye and at least some of the plurality of redirecting structures receive input light from the projecting means, the redirecting structures receiving input light each redirect at least a portion of that input light to form corresponding output beams at least a portion of which are directed into the pupil of the at least one eye and those portions of the output beams forming, responsive to the modulation, images on the retina of the at least one eye.

67. The system according to claim 66, wherein said plurality of redirecting structures are non-overlapping as determined from the center of rotation of the human eye and, for at least some diameters of the pupil of the human eye, light output from the plural redirecting structures results in images perceived as continuous on the retina of the human eye.

68. The system according to claim 66, wherein at least some of said plurality of redirecting structures include at least pair-wise overlap as determined from the center of rotation of the human eye.

69. The system according to claim 66, wherein the plural redirecting structures differ in size, with larger redirecting structures resulting in smaller spot sizes on the retina in locations with higher acuity so as to obtain spot sizes on the retina related to differing acuity capabilities spatially on the retina.

70. The system according to claim 69, wherein said redirecting structures are of substantially two sizes and the larger size corresponds to points on the retina closer to the fovea than the smaller of the two sizes.

71. The system according to claim 66, wherein the effective angle of at least some of said redirecting structures are arranged such that they redirect light substantially from a common launch location into the direction of the center of rotation of the eye.

72. The system according to claim 71 , wherein said at least some redirecting structures launch beams that impinge on higher-acuity regions of the retina.

73. The system according to claim 66, wherein at least some of the redirecting structures are comprised of at least two groups, the first group redirecting light from a common launch location to a first eye pupil location and corresponding output beams being from a range of angles comprising a field of view and the second group redirecting light from a launch location to a second eye pupil location with corresponding output beams being from a range of angles comprising substantially the same field of view, and the first eye pupil location not overlapping with the second eye pupil location for at least some range of actual eye pupil diameters.

74. The system according to claim 73, wherein said at least two groups of redirecting structures direct beams impinging on regions of the retina beyond the fovea.

75. The system according to claim 66, wherein said projection of light to the redirecting structures includes light being reflected from a steerable mirror means to impinge on at least one of the redirecting structures to render a pixel on the retina of the eye.

76. The system according to claim 74, further comprising plural separately modulated sources of light directed at said steerable mirror and each such source resulting in a separate tile being projected on the retina of the at least one eye.

77. The system according to claim 66, wherein said projection of light to said redirecting structures includes light being reflected from plural steerable mirrors and each of the plural steerable mirrors directs light substantially directly to at least one of the redirecting structures.

78. The system according to claim 77, wherein each of said plural steerable mirrors are located at a different angle relative to at least one of said redirecting structures to cause a beam input to the redirecting structure to render a different pixel on the retina of the eye.

79. The system according to claim 77, further comprising plural separately modulated sources of light impinge on each of said plural steerable mirrors, so that a wider angular range is served by said plural steerable mirrors.

80. The system according to claim 77, wherein said plural steerable mirrors are positioned moveably substantially perpendicularly to the angle of beams directed as input to said redirecting structures.

81. The system according to claim 66, wherein said projection of light to said redirecting structures includes light being reflected from a first steerable mirror to impinge on a second steerable mirror and the at least two steerable mirrors cooperating so as to launch light from plural angles to impinge on substantially the same at least one of the redirecting structures so as to result in the rendering of plural pixels on the retina of said at least one eye.

82. The system according to claim 66, wherein said projection of light to said redirecting structures includes spatial light modulator means and optical elements for transforming the resulting substantially parallel spatially-modulated beams into beams of differing angles and for providing the resulting beams of differing angles as input to at least one of the redirecting structures, so that beams output from the at least one of the redirecting structures have differing angles and enter said at least one eye and render spatially differing pixels on the retina of the eye.

83. The system according to claim 66, wherein said projection of light to said redirecting structures includes a changeably steerable optical element structure configured to steer multiple pixel bundles so as to provide at least one change of angle of the bundle and cause pixels resulting from the bundle at a first angle that appear at first positions on the retina of said at least one eye to appear at second positions on the retina, the first and second positions on the retina being substantially separate.

84. The system according to claim 83, wherein said changeably steerable optical element structure comprises an array of steerable mirrors and plural of the mirrors of the array used to steer at least one of said pixel bundles and the mirrors of the array of steerable mirrors having controllable piston motion so as to allow substantially consistent phase between the light paths of the mirrors of the array of steerable mirrors.

85. The system according to claim 66, wherein said projection of light to said redirecting structures includes structures configured to changeably steer a launch angle of a beam substantially wider than a one of the redirecting structures that it impinges on as input to the one of the redirecting structures and for the angle of the corresponding output beam resulting from that redirecting structure to be varied by the launch angle of the substantially wider beam.

86. The system according to claim 66, wherein said source of modulated light comprises at least one laser.

87. The system according to claim 86, wherein said at least one laser comprises means for modulating its output power responsive to said image data.

88. The system according to claim 86, wherein said redirector means are volume holograms.

89. The system according to claim 88, wherein said volume holograms are multiplexed to include a separate structure for different colors comprising a color gamut.

90. The system according to claim 88, wherein said volume holograms are stacked to include a separate structure for different colors comprising a color gamut.

91. The system according to claim 88, wherein said volume holograms are multiplexed to include a separate structure for spatially overlapping said redirector structures.

92. The system according to claim 88, wherein said volume holograms are stacked to include a separate structure for spatially overlapping said redirector structures.

93. The system according to claim 88, wherein said laser comprises at least one color band that is of bandwidth less than 5 nanometers.

94. The system according to claim 91, wherein said bandwidth is less than 1 nanometers.

95. The system according to claim 92, wherein said bandwidth is less than one tenth of one nanometer.

96. The system according to claim 66, wherein at least one of said redirector structures configured to produce output beams that enter the pupil of said eye resulting from different input light beam angles where each of the light beams with different input angles has a different center location relative to the at least one redirector structure.

97. The system according to claim 96, wherein the output beam of said at least one redirector structure is configured to at least substantially fill the pupil of said eye.

98. The system according to claim 96, wherein the output beam of said at least one redirector structure is at least 1.5mm in diameter when entering the pupil of said at least one eye.

99. The system according to claim 98, wherein the output beam of said at least one redirector structure is at least 2mm in diameter when entering the pupil of said at least one eye.

100. The system according to claim 98, wherein the output beam of said at least one redirector structure is at least 2.5mm in diameter when entering the pupil of said at least one eye.

101. The system according to claim 66, wherein at least one of said redirector structures is selectable such that it redirects light when selected and substantially does not redirect when not selected.

102. The system according to claim 101 , wherein said at least one selectable redirector structures comprises an active shutter layer.

103. The system according to claim 102, wherein said active shutter layer comprises liquid crystals.

104. The system according to claim 101, wherein said at least one selectable redirector structure comprises an electrically-switchable hologram.

105. The system according to claim 66, wherein selectable beam shaping is inserted in the optical path between said source of light and at least one of said redirector structures.

106. The system according to claim 105, wherein a first steerable mirror directs a beam of light from said source of light to one of plural locations on a diffractive optical element and the diffractive optical element diffracts at least a portion of that light onto a second steerable mirror and the second steerable mirror in turn directs the light along at least one optical path toward at least one of said redirector structures.

107. The system according to claim 106, wherein the shape of a beam of light directed at said at least one redirector structures is selected from plural beam shapes corresponding to said plural respective locations on said diffractive optical element.

108. The system according to claim 106, wherein an aberration correction of a beam of light directed at said at least one redirector structures is selected from plural aberration corrections corresponding to said respective plural locations on said diffractive optical element.

109. The system according to claim 106, wherein wavefront curvature of a beam of light directed at said at least one redirector structures is selected from plural wavefront curvatures corresponding to said respective plural locations on said diffractive optical element.

110. The system according to claim 66, comprising sensing means for eye tracking to capture at least the angular position of the eye at least relative to the tracking means and the at least angular position determined influences the images projected onto the retina of said eye.

111. The system according to claim 110, wherein said projection of light to said redirecting structures is responsive to said captured rotational position of said at least one eye.

112. The system according to claim 110, wherein said modulation of light is responsive to said captured rotational position of said at least one eye.

113. The system according to claim 110, wherein said sensing means comprising means for sensing blinking of said at least one eye.

114. The system according to claim 110, wherein said sensing means comprising means for sensing the diameter of the pupil of said at least one eye.

115. The system according to claim 110, wherein said sensing means comprising means for sensing the shape of the pupil of said at least one eye.

116. The system according to claim 66, wherein said projection of light to said redirecting structures includes varying the amount of light projected so as to compensate for light clipped by the iris of said at least one eye.

117. The system according to claim 66, wherein said redirecting structures are formed on a material that is substantially opaque.

118. The system according to claim 66, wherein said beams of light projected have wavefront curvature and variable focus means varies the wavefront curvature of the beams so as to correspond with different accommodation of said at least one eye.

119. The system according to claim 118, wherein said wavefront curvature is varied so that the accommodation required of said at least one eye is substantially the same as for corresponding portions of the environmental scene on which said images are superimposed.

19

120. The system according to claim 118, comprising means for measuring angular difference between two eyes and means for varying said wavefront curvature depending on the measurement.

121. The system according to claim 118, comprising means for measuring accommodation of the at least one eye and means for varying said wavefront curvature depending on the measurement.

122. The system according to claim 66, wherein light from the environmental scene is directed back toward the light source means and sensed in order to develop information about the scene that influences the images projected onto the retina of the eye.

123. The system according to claim 122, wherein provision is made for light to be directed back toward the light source through at least some of the redirecting structures.

124. The system according to claim 122, wherein provision is made for light to be directed back toward the light source through additional structures positioned substantially near the corresponding redirecting structures.

125. The system according to claim 122, comprising means for measuring the point of regard of said at least one eye and means for measuring the distance to objects in the actual scene located near the point of regard and means for varying the curvature of the wavefront projected depending on the measurement, so as to provided substantially similar focus for said image rendered onto the retina of the eye as for the objects in the actual scene located near the point of regard.

126. The system according to claim 66, wherein sensor means are arrange to receive light directed back toward said light source from the retina of said at least one eye by said redirector structures.

127. The system according to claim 126, wherein said light sensed is from a first of said redirector structures different from a second of said redirector structures, the second of the redirecting structures being used to project light onto the retina of said eye, and the distance between the spots resolved by the first and the second redirector structures on the retina of the eye being determined by calculation means.

128. The system according to claim 127, wherein said light directed back toward the light source from the retina of said at least one eye by said redirector structures is used to improve focus at least for a particular angle into the eye.

129. The system according to claim 127, wherein said light directed back toward the light source is received by sensor means, and said light sensed is from a first of said redirector structures different from a second of said redirector structures, and the second of the redirector structures being used at a particular instant to project light onto the retina of said at least one eye, and the wavefront curvature on the output of the first redirector structure different from the wavefront curvature on the second redirector structure, and the difference in spot size resolved by the first and the second redirector structures on the retina of the eye resulting in different reflection efficiency, and the reflection efficiency being measured, and the wavefront curvature on beams directed into the pupil of the eye adjusted responsive to the efficiencies measured.

130. The system according to claim 66, comprising additional said projecting and redirecting structures for projecting images into two of said at least one eyes, both the eyes of a person.

131. The system according to claim 66, comprising positioning said redirector structures in front of at least one eye and said projecting of light as said input to said redirector structures being through a waveguide.

132. The system according to claim 66, comprising positioning said redirector structures in front of at least one eye and said projecting of light as said input to said redirector structures from the side of the redirector structures opposite the side of the at least one eye.

133. The system according to claim 66, comprising positioning said redirecting structures in front of at least one eye by means of combining the structures with at least one lens of a pair of eyeglasses.

134. The system according to claim 133, wherein the at least one lens of said pair of eyeglasses is substantially shaped from an optically transmissive material such that it provides optical correction for the wearer when viewing the environmental scene.

135. The system according to claim 133, wherein said input to said redirector structures is directed from a position substantially along a temple sidearm of the frame of said eyeglasses.

136. The system according to claim 133, wherein said redirecting structures are embedded in said at least one lens of said eyeglasses.

137. The system according to claim 133, wherein said redirecting structures are comprised of a thin layer on said at least one lens of said eyeglasses.

138. Eyeglasses, comprising: one or more lens and a frame for holding the one or more lens and the sensing means.

139. Eyeglasses according to claim 138, comprising feedback means to said wearer responsive to said means for sensing gestures of the wearer.

140. Eyeglasses according to claim 138, wherein means for sensing touch of said wearer are configured to sense at least one touch of the wearer fingers at a time.

141. Eyeglasses according to claim 138, wherein said means for sensing touch of said wearer are configured to separably sense plural touches at the same time.

142. Eyeglasses according to claim 138, including means for sensing proximity of fingers of said wearer.

143. Eyeglasses according to claim 138, wherein at least a portion of said sensing means are configured along the temple sidearm of the frame of said eyeglasses.

144. Eyeglasses according to claim 138, wherein said sensing means comprising means for sensing fingers of the wearer and means for creating a keyboard effect.

145. Eyeglasses according to claim 138, wherein at least a portion of said sensing means are configured around the eye of the frame of said eyeglasses.

146. Eyeglasses, including audio transducer means for conducting audio signals through bones of the wearer.

147. The eyeglasses according to claim 146, wherein at least one audio signal transducer is located so as to communicate with the bridge of the nose of said wearer.

148. The eyeglasses according to claim 146, wherein at least one said audio transducer is located on the temple of the frame of said eyeglasses.

149. The eyeglasses according to claim 146, wherein at least one said transducer communicates audio information to said wearer.

150. The eyeglasses according to claim 146, wherein at least one said transducer communicates at least audio audible utterances from said wearer.

151. An eyeglasses frame, comprising: at least one sensor for detecting at least two different angular positions of the sidearm relative to the front face of the eyeglasses frame and the powering of the eyeglasses provided responsive to one of the at least two sensed positions.

152. An eyeglasses frame, comprising: mechanical switch means adapted so as to switch power on when at least one sidearm of the eyeglasses frame is moved from the folded into the wearable position.

153. An eyeglasses frame, comprising: at least one hinge arranged for transferring energy between a temple sidearm of the eyeglasses frame and the front face of the eyeglasses frame.

154. The eyeglasses frame according to claim 153, wherein said energy comprises power.

155. The eyeglasses frame according to claim 153, wherein said energy comprises signals.

156. The eyeglasses frame according to claim 153, wherein at least one flexible conductor of energy is connected from a temple sidearm of said eyeglasses frame, through a hinge of the eyeglasses frame, to the front face of the eyeglasses frame.

157. The eyeglasses frame according to claim 153, wherein said at least one flexible conductor of energy comprises at least one optical waveguide.

158. The eyeglasses frame according to claim 153, wherein said at least one flexible conductor of energy comprises at least a conductor of electrical signals.

159. The eyeglasses frame according to claim 153, wherein said energy travels through contact between relatively movable elements of said hinge.

160. The eyeglasses according to claim 159, wherein the temple sidearm of said eyeglasses frame is separable at said hinge of the eyeglasses frame from the front face of the eyeglasses frame.

161. An eyeglasses frame connection system, comprising proximity contactless energy transfer between the eyeglasses frame and at least one external system by means of cooperating structures for energy transfer in each of the eyeglasses frame and the at least one external system.

162. The eyeglasses frame connection system according to claim 161 , wherein said energy comprises power.

163. The eyeglasses frame connection system according to claim 161 , wherein said energy comprises signals.

164. The eyeglasses frame connection system according to claim 161 , wherein the energy transfer by at least one clip on at least one temple arm of said eyeglasses frame.

165. The eyeglasses frame connection system according to claim 161, wherein said external system comprises storage means for storing said eyeglasses frame.